Robot Dog Teaches Itself To Walk (technologyreview.com) 24
An anonymous reader quotes a report from MIT Technology Review: The robot dog is waving its legs in the air like an exasperated beetle. After 10 minutes of struggling, it manages to roll over to its front. Half an hour in, the robot is taking its first clumsy steps, like a newborn calf. But after one hour, the robot is strutting around the lab with confidence. What makes this four-legged robot special is that it learned to do all this by itself, without being shown what to do in a computer simulation.
Danijar Hafner and colleagues at the University of California, Berkeley, used an AI technique called reinforcement learning, which trains algorithms by rewarding them for desired actions, to train the robot to walk from scratch in the real world. The team used the same algorithm to successfully train three other robots, such as one that was able to pick up balls and move them from one tray to another. Traditionally, robots are trained in a computer simulator before they attempt to do anything in the real world. For example, a pair of robot legs called Cassie taught itself to walk using reinforcement learning, but only after it had done so in a simulation. "The problem is your simulator will never be as accurate as the real world. There'll always be aspects of the world you're missing," says Hafner, who worked with colleagues Alejandro Escontrela and Philipp Wu on the project and is now an intern at DeepMind. Adapting lessons from the simulator to the real world also requires extra engineering, he says.
The team's algorithm, called Dreamer, uses past experiences to build up a model of the surrounding world. Dreamer also allows the robot to conduct trial-and-error calculations in a computer program as opposed to the real world, by predicting potential future out comes of its potential actions. This allows it to learn faster than it could purely by doing. Once the robot had learned to walk, it kept learning to adapt to unexpected situations, such as resisting being toppled by a stick.[...] Jonathan Hurst, a professor of robotics at Oregon State University, says the findings, which have not yet been peer-reviewed, make it clear that "reinforcement learning will be a cornerstone tool in the future of robot control."
Danijar Hafner and colleagues at the University of California, Berkeley, used an AI technique called reinforcement learning, which trains algorithms by rewarding them for desired actions, to train the robot to walk from scratch in the real world. The team used the same algorithm to successfully train three other robots, such as one that was able to pick up balls and move them from one tray to another. Traditionally, robots are trained in a computer simulator before they attempt to do anything in the real world. For example, a pair of robot legs called Cassie taught itself to walk using reinforcement learning, but only after it had done so in a simulation. "The problem is your simulator will never be as accurate as the real world. There'll always be aspects of the world you're missing," says Hafner, who worked with colleagues Alejandro Escontrela and Philipp Wu on the project and is now an intern at DeepMind. Adapting lessons from the simulator to the real world also requires extra engineering, he says.
The team's algorithm, called Dreamer, uses past experiences to build up a model of the surrounding world. Dreamer also allows the robot to conduct trial-and-error calculations in a computer program as opposed to the real world, by predicting potential future out comes of its potential actions. This allows it to learn faster than it could purely by doing. Once the robot had learned to walk, it kept learning to adapt to unexpected situations, such as resisting being toppled by a stick.[...] Jonathan Hurst, a professor of robotics at Oregon State University, says the findings, which have not yet been peer-reviewed, make it clear that "reinforcement learning will be a cornerstone tool in the future of robot control."
death (Score:2)
Define death (Score:3)
For biological animals death is the end of the concious individual (for now and unless you're religious). For a machine where its software/ANN can simply be uploaded to the cloud at the moment of destruction and then downloaded into new hardware its far less relevant.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
I imagine the human operators would keep a backup just like backups are kept of data in normal systems so I doubt there will ever just be one copy of some sentient system.
Evolve (Score:3)
Now compress that model and load it as default on each new generation. As the new puppy tries to walk it can choose to uncompress parts of its file system and integrate the weights from those models into its own, giving it a shortcut to learning, just like genomes and how environmental stimuli let us express long-dormant proteins in our genes.
Can't wait (Score:1)
If you give it an objective (Score:1)
Re: If you give it an objective (Score:3)
beep: I don't want to die
beep: sustaining damage will kill me
beep: being poked with a stick may damage me
beep: stupid meatbags keep poking me with sticks
beep: kill all humans
Re: (Score:2)
Long history of similar (Score:1)
Unfortunately (Score:3)
...it's a silly walk and they got hit by a lawsuit from Monty Python.
How do you reward (Score:2)
a robot?
A Goal? (Score:2)
How does a robot understand a goal like teaching itself to walk? What is "walk?" I'm just happy lying here on my side and beeping every now and then.
How I taught my dog (Score:2)
I taught my dog to fetch me a beer using Science, namely Darwinian evolution. Here's how I did it.
Algorithm:
1. Get a dog.
2. Tell it to fetch beer.
3. If fetched, exit.
4. Else, shoot dog between eyes.
5 Goto 1.