FE AT URE
A perfect storm is giving rise to a surge in artificial intelligence research and KAUST aims to be at its forefront. The quest to design machines with humanlike intelligence has a long history. The term artificial intelligence (AI) was coined in the 1950s, but realizing its expected potential has not been straightforward. Scientists describe two AI winters, in the 1970s and 90s, when commercial and scientific activities in the field declined significantly. Then, from 2010, a new subset of AI, called machine learning, and an even more specialized field, called deep learning, burst onto the research scene, bringing AI closer to the realm of almost limitless possibility. “It’s like a perfect AI storm,” explains KAUST president, Tony Chan. “We have very inexpensive and universally available data-collection devices, like cell phones. We also have fast, powerful hardware and very clever researchers with interesting models and fast algorithms. When everything works together, and you get a killer app, like face recognition, you suddenly see results.” Facial recognition, as when social media websites suggest you tag a person detected in an image you’ve uploaded, is just one example of what can be achieved by training machine-learning algorithms. KAUST researchers are already involved in many aspects of theoretical and applied
38
F a l l 2 0 19
artificial intelligence research. Chan wants to see even more. Theoretical AI A good place to start is with experienced AI researchers. Peter Richtarik joined KAUST in 2017, the same year Google released its federated learning platform, a paradigm-shifting approach to machine learning. Richtarik’s group worked with colleagues at Google to design the platform, which enables developers to design apps that can learn from the private data stored on mobile devices without it having to be uploaded to the cloud. Richtarik is now using his experience to design new and improved algorithms that can train machine-learning and deep-learning models. “I work on the theoretical foundations of machine learning, but my work has many applications,” he explains. One of Richtarik’s 50 active machinelearning-related research projects focuses on distributed machine learning with compression. Training machine-learning models requires huge amounts of data located on tens to even thousands of computers. Communication among them can be very slow, acting as a key bottleneck in the training process. Richtarik and his team develop new training algorithms that work, even when part of the data is discarded, to reduce the amount communicated, a process called lossy compression. “All of the algorithms I design also use randomization. The methods flip coins at every step to make fast and simple decisions during training. This enables them
to learn much faster,” explains Richtarik. “Without randomization, it wouldn’t be possible to have deep learning, machine learning or artificial intelligence—without it we couldn’t train the huge models now used in industry. KAUST colleague Marco Canini works to improve the efficiency of processing huge amounts of complex data through thousands of computers. He and his team collaborated with academic and commercial partners to design a system to speed up the processing per second of training samples by a machine-learning model by 300 percent. He is also studying machinelearning models to develop a tool that can help users understand what they do and how to solve problems when they arise. Training more sophisticated machinelearning models with larger amounts of data is a major trend in the current era of deep learning. Yet, the lack of thorough characterization of the uncertainty of these models limits this technology from becoming more ubiquitous and applicable in life-critical decision-making applications, such as self-driving cars, self-flying drones/planes and automated medical diagnosis. Many deep models can reach very high average performance— super-human in some cases—in many challenging AI tasks, yet they fail in very trivial scenarios: scenarios so trivial that the average human would rarely get it wrong. Such scenarios could be behind the mishaps and fatal accidents that have occurred recently. Bernard Ghanem and his team at