Before you run hurdles, you have to learn to crawl, and before you read William Shakespeare, you need to know the alphabet. Any educator knows the importance of a step-by-step lesson plan for mastering a task. Now, researchers at Uber AI Labs have designed an algorithm that comes up with its own curriculum for teaching simulated robots to cross difficult terrain, without falling flat on their faceless bodies. The algorithm might one day even help autonomous vehicles react in emergency situations.
The new program, called Paired Open-Ended Trailblazer (POET) first comes up with a set of unique terrains, each inhabited by a computer-controlled character. Using only two legs and a laserlike rangefinder, the character must teach itself to walk. After a period of practice, the artificial intelligence changes the challenge—sometimes making it easier, and sometimes more difficult. It might make trenches wider, stumps taller, or the ground more uneven. Occasionally a different walker is swapped in, to see whether the skills learned on one terrain will help on another. This mutating and swapping of obstacle courses creates an unpredictable series of stepping stones on the path to agility.
Using POET, the robot walkers could eventually cover difficult terrain that couldn’t be learned without the earlier courses, the researchers report in a paper posted to arXiv this month. What’s more, POET worked better than a program that simply increased the difficulty of terrain over time, without trying many indirect paths. POET’s circuitous routes of learning paid off again and again. In one example, a bot crouch-walked until it encountered a world with stumps and had to learn to walk upright; it later returned to a flatter world and kept walking upright, completing the course faster than before.
The researchers say POET could one day help real-life robots solve many complex tasks, or even let autonomous cars learn to handle emergencies that programmers hadn’t thought to put in the lesson plan. In an open-ended fashion, POET might even create and solve entirely new problems, in fields from protein synthesis to poetry.