Andrew Ng

Andrew Ng (born 1976) is a machine-learning researcher best known, beyond his research, as an educator and entrepreneur who spread AI. At Stanford he showed how GPUs could speed up deep learning; he was founding lead of Google Brain, Google's deep-learning project; he co-founded the online-course company Coursera; and he led the AI group at China's Baidu. By the count on his own site, more than eight million people have taken an AI class from him.

Career

(Education and current roles from his site; dates from the sources cited for each.)

GPUs and deep learning (2009)

In 2009 Ng and his students Rajat Raina and Anand Madhavan presented "Large-scale Deep Unsupervised Learning using Graphics Processors" at ICML. The deep belief networks and sparse coding then attracting attention were too slow to train at scale. The paper argued that graphics processors far surpassed multicore CPUs, reported DBN training up to 70 times faster than a dual-core CPU for large models, and cut the time to train a four-layer network with 100 million parameters from several weeks to about a day. Training deep networks on GPUs would become the mainstream of image recognition three years later with AlexNet.

Google Brain and the "cat" (2012)

Ng describes himself as the founding lead of the Google Brain project. His title on Google's 2012 blog post was Visiting Faculty, and with Jeff Dean and others he built infrastructure for training large neural networks distributed across Google's data centres.

On 26 June 2012 Dean and Ng presented the results on Google's official blog. According to the ICML 2012 paper, led by Quoc Le, a nine-layer network with a billion connections was trained for three days on a cluster of 1,000 machines (16,000 cores) using ten million unlabelled 200×200 frames taken from random YouTube videos. Without ever being told whether an image contained a face, it developed a neuron that detected faces, and the same network had neurons sensitive to cat faces and human bodies. Using the learned features, it reached 15.8% accuracy in recognising ImageNet's 22,000 categories, a 70% relative improvement on the previous best. The blog said the network had "discovered" what a cat looked like by itself.

That autumn, AlexNet, trained on labelled ImageNet data on two GPUs, won the image-recognition challenge. 2012 was the year in which training deep networks with large data and large compute paid off along both routes at once, unlabelled and labelled.

Coursera (2012-)

According to Coursera's tenth-anniversary post, on 16 August 2011 a group of Stanford professors announced three massive open online courses, and within weeks each had more than 100,000 enrolments. Two of them, Daphne Koller and Ng, launched Coursera with Stanford, the University of Pennsylvania, the University of Michigan and Princeton as founding partners. According to TechCrunch, Ng's machine-learning class became Coursera's first course in 2011, and by 2017 more than 1.5 million students had taken it.

Baidu (2014-2017)

In May 2014 the Chinese search company Baidu announced it had hired Ng as chief scientist. According to The Stanford Daily he would head its AI labs in Beijing and Sunnyvale, California, remaining Coursera's chairman while stepping away from day-to-day work there.

On 22 March 2017 Ng announced he was leaving Baidu. By his account, the AI group had grown to about 1,300 people, including the 300-person Baidu Research; its software was used every day by hundreds of millions of people; and in each of the previous two years his team had birthed a new business unit — autonomous driving and the DuerOS conversational computing platform. A heading in the same post read "AI is the new electricity": just as electricity transformed many industries about a hundred years ago, AI would change nearly every major industry, a theme that became a staple of his talks and courses. He wrote that through Google Brain and Baidu he had played a role in turning two leading technology companies into "AI companies".

DeepLearning.AI and ventures (2017-)

After Baidu, Ng put his weight on education and company-building rather than a research lab. He announced DeepLearning.AI in June 2017 and on 8 August launched a five-course deep-learning specialisation on Coursera (TechCrunch), covering neural networks, backpropagation, convolutional and recurrent networks and more, developed with two Stanford graduate students. He also founded LandingAI, which offers AI such as document processing to enterprises, and chairs it; and he runs AI Fund, a venture studio that co-founds AI companies from the ground up.

He was also a panellist at the January 2017 Asilomar conference on principles for AI research and development (see the Asilomar AI Principles).

The arc

The thread through Ng's career is scale. In 2009 he made training faster with GPUs; in 2012 he made it bigger with 16,000 cores; and in the same year he made a single course reach students by the hundred thousand. He did not invent the central algorithms of deep learning, but he pushed back the limits of scale in compute, data and people, and carried laboratory techniques to large companies and to millions of learners — a distinctive place in the field's history.