Google DeepMind's AI program learns human navigation skills

Google’s AI beat humans at a game that involved racing around an unfamiliar virtual environment

Notch up another win for the robots: the latest program from Google’s artificial intelligence group, DeepMind, has trounced experts at a maze game after it learned to find its way around like a human.

Scientists noticed that when they trained the AI to move through a landscape, it spontaneously developed electrical activity akin to that seen in the specialised brain cells that underpin human navigational skills. So-called ‘grid cells’ were only identified in animals in 2005 in work that earned researchers a Nobel prize.

The latest breakthrough reveals the potential for human brain-like activity to emerge from scratch in AI systems. Beyond making smarter programs, it paves the way for computer engineers to build models that help neuroscientists better understand the human brain.

After discovering the AI-made grid cells, the DeepMind researchers made a beefed-up version of the program. It went on to beat experienced players at a game that involved racing through rooms to find a prize after being dropped into the virtual environment at a random location. Equipped with the artificial grid cells, the AI was faster and found shortcuts that would occasionally crop up when doors in the environment suddenly opened.

“It is doing the kinds of things that animals do and that is to take direct routes wherever possible and shortcuts when they are available,” said Dharshan Kumaran, a senior researcher at DeepMind. “With the grid cells, its performance is markedly enhanced to the point that it surpasses an expert human player.”

The feat marks a milestone in the field of artificial intelligence. Until now the technology used has proved itself to be superhuman at object recognition and games such as chess, Go, and poker, but not at the very different cognitive challenge of efficient navigation.

But the research opens up fresh possibilities too. Because the AI program developed brain-like activity from scratch, scientists believe that a similar approach could shed light on other mysterious processes in the human brain, such as how limbs are controlled. And it could do so without any need for animal or human experimentation.

“There is no reason why we couldn’t use this to understand different functions that the brain performs. It could be a testbed for experiments you wouldn’t otherwise do,” said Caswell Barry, a neuroscientist at University College London who worked on the project.

Writing in the journal Nature, the scientists describe how they built a ‘deep neural network’, a computer program that uses multiple layers of artificial neurons to process information. They then taught the program the basics of navigation by feeding it the kinds of signals that encode speed and direction in the brains of foraging rats. With feedback on its performance, the AI got better and better at predicting where it was when it moved around a virtual environment.

The scientists hoped that with training the AI program might develop its own grid cell-like activity, and that is precisely what they found. A quarter of the artificial neurons in one layer of the deep neural network had begun firing like biological grid cells. In other words, the AI hit on the same strategy to map out the world as the human brain did long ago. “We were surprised how well it worked,” said Caswell. “The degree of similarity is absolutely striking.”

Grid cells are fundamental for navigation in humans and other mammals. They behave as if an invisible mesh of hexagons has been laid over the land, firing rapidly when an animal crosses from one hexagon to the next. In reality, some hexagons are big, others are small, and many overlap. Neuroscientists suspect that this imaginary mesh helps all mammals work out where they are and calculate the shortest path to their goal.

An animation that shows how grid cells fire in a hexagonal pattern.
An animation that shows how grid cells fire in a hexagonal pattern. Photograph: DeepMind

In the next stage of the project, the scientists went on to build a more sophisticated AI that incorporated the artificial grid cell network. They then let it loose on a virtual reality maze game. Tests in the game showed that the AI used grid cells not only to track its position, but also to work out the direction and distance to its goal, so it could take the most direct route.

“We tend to think of navigation as a very everyday thing, but it’s something we don’t understand very well,” said Kumaran. “What our study does is shed light on the neural mechanisms that underlie human navigation.”

Francesco Savelli, an expert in artificial intelligence and neuroscience at Johns Hopkins University in Maryland, said that the magic of AI systems like the one used is that they come up with their own ways of processing data to solve the task at hand. “The emergence of grid cells in this model is an impressive demonstration of exactly that,” he said.


Ian Sample Science editor

The GuardianTramp

Related Content

Article image
'It's able to create knowledge itself': Google unveils AI that learns on its own
In a major breakthrough for artificial intelligence, AlphaGo Zero took just three days to master the ancient Chinese board game of Go ... with no human help

Ian Sample Science editor

18, Oct, 2017 @5:00 PM

Article image
Google develops computer program capable of learning tasks independently
‘Agent’ hailed as first step towards true AI as it gets adept at playing 49 retro computer games and comes up with its own winning strategies

Hannah Devlin

25, Feb, 2015 @6:36 PM

Article image
Google creates AI program that uses reasoning to navigate the London tube
Combining external memory and deep learning, DeepMind’s program learns how to do tasks independently, and could pave the way for sophisticated AI assistants

Hannah Devlin Science correspondent

12, Oct, 2016 @5:04 PM

Article image
Google’s DeepMind makes AI program that can learn like a human
Program brings artificial general intelligence a step closer by using previous knowledge to solve fresh problems

Ian Sample Science editor

14, Mar, 2017 @5:27 PM

Article image
Google's artificial intelligence machine to battle human champion of 'Go'
Lee Se-dol, 33, one of the world’s top players of the ancient Asian pastime, is confident he can beat Alphago. But he hasn’t seen improvements made to the system

Alex Hern

07, Mar, 2016 @2:33 PM

Article image
Google AI in landmark victory over Go grandmaster
Fan Hui, three-time champion of the east Asian board game, lost to DeepMind’s program AlphaGo in five straight games

Alex Hern

27, Jan, 2016 @6:15 PM

Article image
AlphaZero AI beats champion chess program after teaching itself in four hours
Google’s artificial intelligence sibling DeepMind repurposes Go-playing AI to conquer chess and shogi without aid of human knowledge

Samuel Gibbs

07, Dec, 2017 @12:41 PM

Article image
'The challenge was to play like a human': AI takes on the gamers
Google’s DeepMind beat 99.8% of humans at StarCraft II, but bigger challenge was not giving itself away

Alex Hern

30, Oct, 2019 @6:00 PM

Article image
AlphaGo seals 4-1 victory over Go grandmaster Lee Sedol
DeepMind’s artificial intelligence astonishes fans to defeat human opponent and offers evidence computer software has mastered a major challenge

Steven Borowiec

15, Mar, 2016 @10:12 AM

Article image
Google DeepMind is making artificial intelligence a slave to the algorithm | Letters
Letters: Google’s role in university artificial intelligence courses alarms Sheila Hayman


03, Nov, 2017 @6:49 PM