Can Robots Learn to Set Their Own Goals?

Overview

A project funded by the European Union (EU) aims to improve the ability of robots to set their own goals and develop autonomous skills. According to the European Commission, this would increase EU’s competitiveness in the field of artificial intelligence and advanced learning. The research team has already demonstrated the basic open-ended learning process for solving tasks using Goal-based Open-ended Autonomous Learning Robots (GOAL-ROBOTs). Could this technology soon be used in the manufacturing environment?

Key Findings

• We cannot overstate the future importance of artificial intelligence in manufacturing.
• Researchers and the EU are taking steps to ensure that robotic industrial processes in Europe are not left behind the innovation curtain.

• Open-ended learning robots will advance the fundamental understanding of continuous autonomous learning in which robots can acquire flexible skills in conditions not predicted at design, while requiring little human interference.

Moving the GOAL Posts

Researchers from Italy, Germany, and France are working on a project called GOAL-ROBOTS, which marks a conceptual shift to develop autonomous learning robots. The project focuses on two main outcomes.  First, to achieve autonomous, open-ended learning, robots should be capable of identifying their own goals, then generating the tasks needed to accomplish those goals. Second, newly learned algorithms use these self-generated goals to speed up skill development. The final aim is to allow GOAL-ROBOTs to build computational architectures that support self-generation of goals without human assistance and use these goals to autonomously learn a large library of skills. When new skills are acquired, similarities between the goals can be used to speed up additional skill learning.

The basic open-ended learning process uses a four-level architecture called GRAIL (goal-discovering robotic architecture for intrinsically-motivated learning). This architecture allows the robot to:

• Autonomously discover changes in the environment, store them as potential goals, and formulate these events
• Set its own goals based on “competence-based intrinsic motivations”

• Monitor goal achievement and build a learning indicator for achieved goals

In the first demonstration, we saw that goals are formulated using environmental changes captured by the robot’s visual input (cameras). The before-and-after images of the state were used to enable the robot to self-determine the target, check whether or not the goal had actually been achieved, and improve skills training. When there is a match between the presentation of a selected goal and the event, the system autonomously recognizes this achievement and generates an indicator, which is then used to reinforce the different learning processes in the architecture.  As soon as the robot learns to execute a goal properly, it moves on to train a different skill or goal. Results from the process show how GRAIL allows the robot to cope with changing environments where possible goals become available later.

In the next demonstration, the researchers tested how autonomously acquired skills from intrinsic learning are used to help enable an artificial system perform externally assigned tasks. Using a two-phased (intrinsic and extrinsic) object displacement task, the robot can explore its environment autonomously.  Whenever the robot moves the object, the system recognizes the change (difference in visual inputs) and stores the image of the change together with the actions that generate it (e.g., right-moved object may be accompanied by actions like movement of right robotic arm to a certain angle away from a fixed point or body).

After the researchers allowed the robot to discover a predefined number of object positions, they moved into the extrinsic phase.  In this phase, the robot is shown different positions of the object they want the robot to achieve.  The system performs the assigned tasks using the outcomes discovered autonomously during the intrinsic phase. Specifically, the system matches each unknown task with the closest of the outcomes discovered during the intrinsic learning phase and then modifies this behavior using the reinforcement learning algorithm.  According to the researchers, the results show an autonomously developed skills library able to achieve externally assigned tasks.  You can view two different video demonstrations of the experiment at  https://www.youtube.com/watch?v=VCqxoXlnrAw and https://www.youtube.com/watch?v=O7i1qttva2I .

Can Robots Learn?

In the final demonstration, the researchers implemented explicit learning goals and allowed the robots to practice the skills required to reach these goals in an unstructured natural human environment. This phase is supported by a framework of probabilistic online motion planning with online adaptation using a stochastic recurrent neural network.  The researchers evaluated this phase using a Kuka robotic arm to demonstrate the ability of the robot to adapt motion plans to a realistic dynamic and constrained work situation in an efficient way. Starting with an online, pre-trained network simulation of motion plan, the robot avoids the first obstacle presented to it (see figure 1 below).  After 20 seconds, the robot collides with the second obstacle, and then learns to avoid the second obstacle within five seconds while finishing the task assigned (to arrange the table as seen in figure 2 below).

ARC Advisory Group clients can view the complete report at ARC Client Portal 

If you would like to buy this report or obtain information about how to become a client, please Contact Us  

Keywords: Artificial Intelligence, Robots, Industrial Process, Autonomous Learning, ARC Advisory Group.