These Virtual Obstacle Courses Help Real Robots Learn to Walk
AI has shown promise for training robots to do real-world tasks that cannot easily be written into software, or that require some sort of adaptation. The ability to grasp awkward, slippery, or unfamiliar objects, for instance, is not something that can be written into lines of code.
The 4,000 simulated robots were trained using reinforcement learning, an AI method inspired by research on how animals learn through positive and negative feedback. As the robots move their legs, an algorithm judges how this affects their ability to walk, and tweaks the control algorithms accordingly.
The simulations ran on specialized AI chips from Nvidia rather than general purpose chips used in computers and servers. As a result, the researchers say they were able to train the robots in less than one-hundredth the time that’s normally required.
Using the specialized chips also presented challenges. Nvidia’s chips excel at calculations that are crucial for rendering graphics and running neural networks, but they’re not well suited to simulating the properties of physics, like climbing and sliding. So researchers had to come up with some clever software workarounds, says Rev Lebaredian, Nvidia’s vice president of simulation technology. “It has taken us a long time to get it right,” he says.
Simulation, AI, and specialized chips have the potential to advance robotic intelligence. Nvidia has developed software tools that make it easier to simulate and control industrial robots using its chips. The company has also established a robotics research lab in Seattle. And it sells chips and software for use in self-driving vehicles.
Unity Technologies, which makes software for building 3D video games, has also branched into making software suitable for roboticists to use. Danny Lange, the company’s senior vice president for artificial intelligence, says Unity noticed how many researchers were using the company’s software to run simulations, so they made it more realistic and compatible with other robotics software. Unity is now working with Algoryx, a Swedish company that is testing whether reinforcement learning and simulation can train forestry robots to pick up logs.