Google Brain taught a neural network to spot cats without ever labelling them
In short: In 2012, a Google research team trained a large neural network on frames from 10 million YouTube videos. Nobody labelled the cats for it, yet it learned to pick out the concept on its own. That project came from a group whose later work shaped translation tools, free software and the design behind today's large language models.
The group began in 2011 at Google X, the company's experimental research lab. Jeff Dean and Greg Corrado from Google worked with Andrew Ng, a Stanford professor, on a collaboration that paired open-ended machine learning research with large information systems and computing resources. The cat result stood out because the network picked up high-level concepts from raw video rather than following rules that engineers had written by hand.
Several of the team's tools reached well beyond the lab. In 2013 Tomáš Mikolov and his colleagues built word2vec, a technique that learns vector representations of words from large bodies of text, and it was released as open-source software that spread widely through research and industry. In 2015 Google made TensorFlow freely available to the public, a toolkit that lets anyone build and train their own neural networks. In 2017 researchers described the Transformer in the paper Attention Is All You Need. It was first designed for machine translation and later became the leading design for large language models.
The translation work shows both the gains and the costs of this approach. Google Neural Machine Translation, launched in September 2016, looks at groups of words in the context of the whole sentence instead of matching words one at a time. Compared with the older phrase-based system, it was reported to score a 24 percent improvement in similarity to human translation, along with a 60 percent reduction in errors. The trade-off was speed: translation time grew sharply as sentences got longer, so the team added 2,000 more processors to keep the process fast and reliable.
The group's history is also contested. In December 2020 the AI ethicist Timnit Gebru left Google, and the exact nature of her departure is disputed. In March 2022 the researcher Satrajit Chatterjee was fired after he questioned the findings of a 2021 Nature paper on chip design, whose performance claims were contested. In April 2023 Google Brain was combined with DeepMind into Google DeepMind, part of the company's wider push on AI.
Opens the Intellect+ feed with this card first. Swipe up for the next one.
Sources
- Summary by Intellect · source: Google Brain
To cite this card, link to https://intellectplus.co.uk/app/science-google-brain.
How we source, check and correct cards: Editorial standards.
General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.
Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?
Related
More in Science · Latest · A–Z · All topics