Visual foresight is the technical term for the new learning technique developed by researchers at U.C Berkeley. Robots used in the trial could accurately predict what they would see after they performed a certain action. This initial prototype deals with learning basic manual skills for now meaning that the imaginations of the robots is relatively basic at the moment. Inferences on future events can only be made a few seconds into the future, but it is still a giant leap for robot-kind, which also means that they closer to outshining humankind who can barely see the present picture.
What is even more amazing is that these robots don’t need any help from humans in making their predictions since it is all based on one conforming algorithm that allows robots to do what is needed given a specific condition or situation. The robot also does not need any knowledge of physics to perform the tasks and see into the future.
The visual foresight method is one that involves the robot actually exploring its environment in an unsupervised way. So the robot would first manipulate objects on a table in order to be able to formulate a predictive model that can surprisingly be used for objects the robot has never touched before.
"In the same way that we can imagine how our actions will move the objects in our environment, this method can enable a robot to visualize how different behaviors will affect the world around it," told Sergey Levine, assistant professor in the Department of Electrical Engineering and Computer Sciences at Berkeley. "This can enable intelligent planning of highly flexible skills in complex real-world situations."
The program is essentially made from a deep learning model based on dynamic neural advection, or DNA. These DNA models predict the change of pixels based on the actions of the robot. Currently, the models are making big steps in complexity, allowing the robots to be able to perform more complex actions in order to make more complicated predictions. The robots are currently able to move around multiple objects and change the order of objects in a sequence.
"In that past, robots have learned skills with a human supervisor helping and providing feedback. What makes this work exciting is that the robots can learn a range of visual object manipulation skills entirely on their own," noted Chelsea Finn, a phD student in Levine's lab.
With the DNA based models, robots will reposition objects and then make a model for other objects with the same action. Such a model would concern how the robot would see an arbitrary object prior to moving it and then the transformation of the object in the 3D plane after the action. The robot would also learn by viewing and analyzing raw camera footage in order to understand how objects are affected by various manipulations.
A graduate student in the lab, named Frederik Ebert, said "Humans learn object manipulation skills without any teacher through millions of interactions with a variety of objects during their lifetime. We have shown that it possible to build a robotic system that also leverages large amounts of autonomously collected data to learn widely applicable manipulation skills, specifically object pushing skills."
The only reason the robots are able to apply a model created for one object for multiple items is because the method of learning is specifically through observation. If the robot learned only through physical manipulation of every single object, then its models would not be so generally applicable.
The Berkeley students plan to continue to work on this project in order to refine the learning technique to allow the robots to perform more sophisticated actions.