Google's Deep Learning System That Thinks

If this doesn't terrify you... Google's computers OUTWIT their humans

Last Friday at a machine learning conference in San Francisco, Google software engineer Quoc V. Le explained how Google's "deep learning" system works.

"Deep learning" requires large computer clusters to absorb data (such as images) and automatically classify them. Google's Android voice-controlled search, image recognition, and Google Translate all use this technology.

In June 2012, the New York Times reported on how Google taught its virtual neural network system key features of cats by feeding it millions of YouTube videos through a system called "DistBelief." The innovation of this system lies in its ability to independently summarize the characteristics of the concept of a cat without prior information about "cat feature descriptions," meaning DistBelief has self-learning capabilities. Of course, the computational power of this system is also very vast, consisting of 1000 machines with 16,000 cores, processing parameters as high as 1 billion.

This system operates using a hierarchical mechanism; the lowest layer of the neural network can detect changes in image colors, the next layer identifies specific types of contours. After adding several subsequent analysis layers, different branches of the system generate detection methods for objects such as faces, rocking chairs, and computers.

However, what shocked Quoc V. Le was that the machine learned to recognize things even humans find difficult to distinguish—like paper shredders. We know what a paper shredder is because we have seen it, but Google's monster had never seen one.

Quoc V. Le explained that learning how to summarize the characteristics of a paper shredder is an extremely complex matter, and he himself couldn't figure it out after thinking about it for a long time.

Even when Quoc showed pictures of the paper shredder to many of his colleagues, they also struggled to identify it. On the contrary, the success rate of this system’s recognition was higher, and Quoc himself could not be sure if he could write a program to achieve this. Quoc explained that this is because people need data rather than themselves to summarize characteristics.

In other words, this means that now Google researchers cannot clearly explain how exactly this system recognizes specific objects, because the program seems to have developed independent thinking abilities, its complex cognitive process being unpredictable, although this "thinking" ability is still limited to a very narrow scope.

However, Google does not expect the deep learning system to independently evolve into a mature new artificial intelligence system. The research director said earlier this year:

Will AI (artificial intelligence) emerge on its own? I am very pragmatic—we make things happen by doing them.

Nevertheless, Peter Norvig, Google's AI lead, believes that such dense statistical data models used by Google are the best hope for tackling thorny issues like reliable voice recognition and understanding, which contrasts with Noam Chomsky's views.

Deep learning is very attractive to Google because it can solve problems that the company's own researchers cannot, and it allows Google to hire fewer incompetent people. We know that Google is renowned for hiring top talent.

Delegating tasks to machines is something Google has done plenty of before. For example, resource management in Google's many data centers is handled by Brog and Omega. These cluster management systems allocate workloads like "biological" entities.

Considering Google's ambition to "organize the world's information," naturally, the fewer people employed, the better. Quoc said that by developing these "deep learning" systems, Google can hire fewer human experts.

He added:

Machine learning is difficult because even though theoretically algorithms like logistic regression can be applied, in practice, we spend a lot of time on data processing and mining features. You have to hire domain experts for every problem. ... So Google hopes machines can do those things.

By working hard, giving machines greater capabilities, and limited local intelligence, Google can solve classification problems that human experts cannot. Will it develop into Skynet? The answer is no. But it can develop into expert machines. Fortunately, machines are still cooperative for now.