Robot Learning by Visual Observation

1

Introduction

Robot programming is the specification of the desired motions of the robot such that it may perform sequences of prestored motions or motions computed as functions of sensory input (Lozano‐Pérez, 1983).

In today’s competitive global economy, shortened life cycles and diversification of the products have pushed the manufacturing industry to adopt more flexible approaches. In the meanwhile, advances in automated flexible manufacturing have made robotic technology an intriguing prospect for small‐ and medium‐sized enterprises (SMEs). However, the complexity of robot programming remains one of the major barriers in adopting robotic technology for SMEs. Moreover, due to the strong competition in the global robot market, historically each of the main robot manufacturers has developed their own proprietary robot software, which further aggravates the matter. As a result, the cost of robotic tasks integration could be many folds of the cost of robot purchase. On the other hand, the applications of robots have gone well beyond the manufacturing to the domains such as household services, where a robot programmer’s intervention would be scarce or even impossible. Interaction with robots is increasingly becoming a part of humans’ daily activities. Therefore, there is an urgent need for new programming paradigms enabling novice users to program and interact with robots. Among the variety of robot programming approaches, programming by demonstration (PbD) holds a great potential to overcome complexities of many programming methods.

This introductory chapter reviews programming approaches and illustrates the position of PbD in the spectrum of robot programming techniques. The PbD architecture is explained next. The chapter continues with applications of PbD and concludes with an outline of the open research problems in PbD.

1.1 Robot Programming Methods

A categorization of the robot programming modes based on the taxonomy reported by Biggs and MacDonald (2003) is illustrated in Figure 1.1. The conventional methods for robot programming are classified into manual and automatic, both of which rely heavily on expensive programming expertise for encoding desired robot motions into executable programs.

Figure 1.1 Classification of robot programming methods.

(Data from Biggs and MacDonald (2003).)

The manual programming systems involve text‐based programming and graphical interfaces. In text‐based programming, a user develops a program code using either a controller‐specific programming language or extensions of a high‐level multipurpose language, for example, C++ or Java (Kanayama and Wu, 2000; Hopler and Otter, 2001; Thamma et al., 2004). In both cases, developing the program code is time‐consuming and tedious. It requires a robot programming expert and an equipped programming facility, and the outcomes rely on programmer’s abilities to successfully encode the required robot performance. Moreover, since robot manufacturers have developed proprietary programming languages, in industrial environments with robots from different manufacturers, programming robots would be even more expensive. The graphical programming systems employ graphs, flowcharts, or diagrams as a medium for creating a program code (Dai and Kampker, 2000; Bischoff et al., 2002). In these systems, low‐level robot actions are represented by blocks or icons in a graphical interface. The user creates programs by composing sequences of elementary operations through combination of the graphical units. A subclass of the graphical programming systems is the robotic simulators, which create a virtual model of the robot and the working environment, whereby the virtual robot is employed for emulating the motions of the actual robot (Rooks, 1997). Since the actual robot is not utilized during the program development phase, this programming method is referred to as off‐line programming (OLP).

The conventional automatic programming systems employ a teach‐pendant or a panel for guiding the robot links through a set of states to achieve desired goals. The robot’s joint positions recorded during the teaching phase are used to create a program code for task execution. Although programming by teach‐pendants or panel decreases the level of required expertise, when compared to the text‐based programming systems, it still requires trained operators with high technical skills. Other important limitations of the guided programming systems include the difficulties in programming tasks with high accuracy requirements, absence of means for tasks generalizations or for transfer of the generated programs to different robots, etc.

The stated limitations of the conventional programming methods inspired the emergence of a separate class of automatic programming systems, referred to as learning systems. The underlying idea of robot learning systems originates from the way we humans acquire new skills and knowledge. Biggs and MacDonald (2003) classified these systems based on the corresponding forms of learning and solving problems in cognitive psychology: exploration, instruction, and observation. In exploration‐based systems, a robot learns a task with gradually improving the performance by autonomous exploration. These systems are often based on reinforcement learning techniques, which optimize a function of the robot states and actions through assigning rewards for the undertaken actions (Rosenstein and Barto, 2004; Thomaz and Breazeal, 2006; Luger, 2008). Instructive systems utilize a sequence of high‐level instructions by a human operator for executing preprogrammed robot actions. Gesture‐based (Voyles and Khosla, 1999), language‐based (Lauria et al., 2002), and multimodal communication (McGuire et al., 2002) approaches have been implemented for programming robots using libraries of primitive robot actions. Observation‐based systems learn from observation of another agent while executing the task. The PbD paradigm is associated with the observation‐based learning systems (Billard et al., 2008).

1.2 Programming by Demonstration

Robot PbD is an important topic in robotics with roots in the way human beings ultimately expect to interact with a robotic system. Robot PbD refers to automatic programming of robots by demonstrating sample tasks and can be viewed as an intuitive way of transferring skill and tasks knowledge to a robot. The term is often used interchangeably with learning by demonstration (LbD) and learning from demonstration (LfD) (Argall et al., 2009; Konidaris et al., 2012). PbD has evolved as an interdisciplinary field of robotics, human–robot interaction (HRI), sensor fusion, machine learning, machine vision, haptics, and motor control. A few surveys of robot PbD are available in the literature (e.g., Argall et al., 2009). PbD can be perceived as a class of supervised learning problems because the robot learner is presented with a set of labeled training data, and it is required to infer an output function with the capability of generalizing the function to new contexts. In the taxonomy of programming approaches shown in Figure 1.1, PbD is a superior learning‐based approach. Compared to the exploration‐based learning systems (as an unsupervised learning problem), PbD systems reduce the search space for solutions to a particular task, by relying on the task demonstrations. The learning is also faster because the trial and errors associated with the reinforcement methods are eliminated.

In summary, the main purpose in PbD is to overcome the major obstacles for natural and intuitive way of programming robots, namely lack of programming skills and scarcity of task knowledge. In industrial settings, this translates to reduced time and cost of programming robots by eliminating the involvement of a robot programmer. In interactive robotic platforms, PbD systems can help to better understand the mechanisms of HRI, which is central to social robotics challenges. Moreover, PbD creates a collaborative environment in which humans and robots participate in a teaching/learning process. Hence, PbD can help in developing methods for robot control which integrate safe operation and awareness of the human presence in human–robot collaborative tasks.

1.3 Historical Overview of Robot PbD

Approaches for automatic programming of robots emerged in the 1980s. One of the earlier works was the research by Dufay and Latombe (1984) who implemented inductive learning for the robot assembly tasks of mating two parts. The assembly tasks in this work were described by the geometric models of the parts, and their initial and final relations. Synthesis of program codes in the robotic language was obtained from training and inductive (planning) phases for sets of demonstrated trajectories. In this pioneering work on learning from observation, the sequences of states and actions were represented by flowcharts, where the states described the relations between the mating parts and the sensory conditions.

Another early work on a similar topic is the assembly‐plan‐from‐observation (APO) method (Ikeuchi and Suehiro, 1993). The authors presented a method for learning assembly operations of polyhedral objects. The APO paradigm comprises the following six main steps: temporal segmentation of the observed process into meaningful subtasks, scene objects recognition, recognition of performed assembly task, grasp recognition of the manipulated objects, recognition of the global path of manipulated objects for collision avoidance, and task instantiation for reproducing the observed actions. The contact relations among the manipulated objects and environmental objects were used as a basis for constraining the relative objects movements. Abstract task models were represented by sequences of elementary operations accompanied by sets of relevant parameters (i.e., initial configurations of objects, grasp points, and goal configurations).

Munch et al. (1994) elaborated on the role of the teacher as a key element for successful task reproduction. The learning was accomplished through recognition of elementary operations for the observed tasks. The demonstrator supervised and guided the robot’s knowledge acquisition by (i) taking into considerations the structure of robot’s perceptibility sensors when providing examples, (ii) taking part in preprocessing and segmentation of the demonstrations, and (iii) evaluating the proposed task solution.

Ogawara et al. (2002a) proposed to generate a task model from observations of multiple demonstrations of the same task, by extracting particular relationships between the scene objects that are maintained throughout all demonstrations. Each demonstration was represented as a sequence of interactions among the user’s hand, a grasped object and the environmental objects. The interactions that were observed in all demonstrations were called essential interactions, whereas the variable parts of the demonstrations called nonessential interactions were ignored in the task planning step. Generalization across multiple demonstrations was carried out by calculating the mean and variance of all trajectories for the essential interactions. A robot program was generated from the mean trajectory, and mapped to robot joints’ motors using an inverse kinematics controller.

The advancements in the fields of machine learning and artificial intelligence in the past two decades produced an abundance of new methods and approaches. This trend was reflected by the implementation of approaches in robot PbD based on neural networks (Liu and Asada, 1993; Billard and Hayes, 1999), fuzzy logic (Dillmann et al., 1995), statistical models (Yang et al., 1994; Tso and Liu, 1997; Calinon, 2009), regression techniques (Atkeson et al., 1997; Vijayakumar and Schaal, 2000), etc.

Significant body of work in PbD concentrated on utilizing virtual reality (VR) as an interaction medium to substitute the actual workspace (Takahashi and Sakai, 1991; Aleotti et al., 2004). The main advantages of performing demonstrations in VR include the availability of direct information for the positions and orientations of teacher’s motions and the environmental objects during the demonstration phase, the accessible simulation of generated task solutions before the execution in the real world, reduced efforts and fatigue, increased safety of the user compared to physical demonstrations, etc. The concept of virtual fixtures, which refers to the use of virtual guidance and assistance to simplify and improve the tasks demonstration, was employed in Payandeh and Stanisic (2002) via provision of prior task information with an aim to restrict the demonstrated workspace and achieve more consistent performance. Aleotti et al. (2004) used visual and tactile virtual fixtures in PbD context, whereas adaptive virtual fixtures that correspond to different subtasks of a complex task were proposed by Aarno et al. (2005).

The recent progress in the field of HRI was also taken into consideration by several authors as a basis for improving the process of transfer of knowledge through demonstrations. Since building and developing social mechanisms between robots and humans in a PbD setting rely on successful training, Calinon and Billard (2007a) highlighted several interaction aspects that a demonstrator must reflect on before the demonstrations, such as what are the best ways to convey the knowledge considering the robot’s abilities, which parts of the demonstration need special attention, etc. (Figure