Dynamic Movement Primitives (DMPs) are widely used for encoding motion data. Task parameterized D... more Dynamic Movement Primitives (DMPs) are widely used for encoding motion data. Task parameterized DMP (TP-DMP) can adapt a learned skill to different situations. Mostly a customized vision system is used to extract task specific variables. This limits the use of such systems to real world scenarios. This paper proposes a method for combining the DMP with a Convolutional Neural Network (CNN). Our approach preserves the generalization properties associated with a DMP, while the CNN learns the task specific features from the camera images. This eliminates the need to extract the task parameters, by directly utilizing the camera image during the motion reproduction. The performance of the developed approach is demonstrated through a trash cleaning task, executed with a real robot. We also show that by using the data augmentation, the learned sweeping skill can be generalized for arbitrary objects. The experiments show the robustness of our approach for several different settings.
Task-parameterized skill learning aims at adaptive motion encoding to new situations. While exist... more Task-parameterized skill learning aims at adaptive motion encoding to new situations. While existing approaches for task parameterized skill learning have demonstrated good adaptation within the demonstrated region, the extrapolation problem of task parameterized skills has not been investigated enough. In this work, with the aim of good adaptation not only within the demonstrated region but also outside of the region, we propose to combine a generative model with a Dynamic Movement Primitive (DMP) by formulating learning as a density estimation problem. Moreover, for efficient learning from relatively few demonstrations, we propose to augment training data with additional incomplete data. The proposed method is tested and compared with existing works in simulations and real robot experiments. Experimental results verified its generalization in the extrapolation region.
While teleoperation provides a possibility for a robot to operate at extreme conditions instead o... more While teleoperation provides a possibility for a robot to operate at extreme conditions instead of a human, teleoperating a robot still demands a heavy mental workload from a human operator. Learning from demonstrations can reduce the human operator's burden by learning repetitive teleoperation tasks. However, one of challenging issues is that demonstrations via teleoperation are less consistent compared to other modalities of human demonstrations. In order to solve this problem, we propose a learning scheme based on Dynamic Movement Primitives (DMPs) which can handle less consistent, asynchronized and incomplete demonstrations. In particular we proposed a new Expectation Maximization (EM) algorithm which can synchronize and encode demonstrations with temporal and spatial variances, different initial and final conditions and partial executions. The proposed algorithm is tested and validated with three different experiments of a peg-in-hole task conducted on 3-Degree of freedom (DOF) master-slave teleoperation system.
A novel approach ensuing Gaussian Mixture Model is proposed in this research to segregate the inp... more A novel approach ensuing Gaussian Mixture Model is proposed in this research to segregate the input image into assorted strokes. Once the strokes are extracted, they are combined to reproduce the same character using Gaussian Mixture Regression. The resulted learned strokes are then implemented on KUKA Light Weight Robot and the learned skill is further improved using Reinforcement Learning.
This paper addresses the problem of fitting finite Gaussian Mixture Model (GMM) with unknown numb... more This paper addresses the problem of fitting finite Gaussian Mixture Model (GMM) with unknown number of components to the univariate and multivariate data. The typical method for fitting a GMM is Expectation Maximization (EM). There are many challenges involved when applying EM for fitting a GMM; such as how to initialize the GMM, how to restrict the covariance matrix of a component from becoming singular and setting the number of components in advance. This paper presents a simulated annealing EM algorithm and demonstrates its usefulness in avoiding convergence to the boundary of the parameter space. We have introduced a systematic initialization procedure by using the principals of
stochastic exploration. The experiments have demonstrated the robustness of our approach on different datasets.
In learning by exploration problems such as reinforcement learning (RL), direct policy search, st... more In learning by exploration problems such as reinforcement learning (RL), direct policy search, stochastic optimization or evolutionary computation, the goal of an agent is to maximize some form of reward function (or minimize a cost function). Often, these algorithms are designed to find a single policy solution. We address the problem of representing the space of control policy solutions by considering exploration as a density estimation problem. Such representation provides additional information such as shape and curvature of local peaks that can be exploited to analyze the discovered solutions and guide the exploration. We show that the search process can easily be generalized to multi-peaked distributions by employing a Gaussian mixture model (GMM) with an adaptive number of components. The GMM has a dual role: representing the space of possible control policies, and guiding the exploration of new policies. A variation of expectation-maximization (EM) applied to reward-weighted policy parameters is presented to model the space of possible solutions, as if this space was a probability distribution. The approach is tested in a dart game experiment formulated as a black-box optimization problem, where the agent's throwing capability increases while it chases for the best strategy to play the game. This experiment is used to study how the proposed approach can exploit new promising solution alternatives in the search process, when the optimality criterion slowly drifts over time. The results show that the proposed multi-optima search approach can anticipate such changes by exploiting promising candidates to smoothly adapt to the change of global optimum.
This thesis focuses on having a robot learn to play the game of darts. Playing darts involves mul... more This thesis focuses on having a robot learn to play the game of darts. Playing darts involves multiple tasks. For e.g. how to throw a dart and where to target on the board. It has been shown that with a few demonstrations by the user, the robot can learn how to produce trajectories for hitting a given point on the board; with improvement in accuracy along with the experience. On the other hand we showed how a robot can discover regions for hitting on the board so that it can maximize its expected score.
This paper presents a complete Simultaneous Localization and Mapping (SLAM) solution for indoor m... more This paper presents a complete Simultaneous Localization and Mapping (SLAM) solution for indoor mobile robots, addressing feature extraction, autonomous exploration and navigation using the continuously updating map. The platform used is Pioneer PeopleBot equipped with SICK Laser Measurment System (LMS) and odometery. Our algorithm uses Hough Transform to extract the major representative features of indoor environment such as lines and edges. Localization is accomplished using Relative Filter which depends directly on the perception model for the correction of error in the robot state. Our map for localization is in the form of a landmark network whereas for navigation we are using occupancy grid. The resulting algorithm makes the approach computationally lightweight and easy to implement. Finally, we present the results of testing the algorithm in Player/Stage as well as on PeopleBot in our Robotics and Control Lab.
Dynamic Movement Primitives (DMPs) are widely used for encoding motion data. Task parameterized D... more Dynamic Movement Primitives (DMPs) are widely used for encoding motion data. Task parameterized DMP (TP-DMP) can adapt a learned skill to different situations. Mostly a customized vision system is used to extract task specific variables. This limits the use of such systems to real world scenarios. This paper proposes a method for combining the DMP with a Convolutional Neural Network (CNN). Our approach preserves the generalization properties associated with a DMP, while the CNN learns the task specific features from the camera images. This eliminates the need to extract the task parameters, by directly utilizing the camera image during the motion reproduction. The performance of the developed approach is demonstrated through a trash cleaning task, executed with a real robot. We also show that by using the data augmentation, the learned sweeping skill can be generalized for arbitrary objects. The experiments show the robustness of our approach for several different settings.
Task-parameterized skill learning aims at adaptive motion encoding to new situations. While exist... more Task-parameterized skill learning aims at adaptive motion encoding to new situations. While existing approaches for task parameterized skill learning have demonstrated good adaptation within the demonstrated region, the extrapolation problem of task parameterized skills has not been investigated enough. In this work, with the aim of good adaptation not only within the demonstrated region but also outside of the region, we propose to combine a generative model with a Dynamic Movement Primitive (DMP) by formulating learning as a density estimation problem. Moreover, for efficient learning from relatively few demonstrations, we propose to augment training data with additional incomplete data. The proposed method is tested and compared with existing works in simulations and real robot experiments. Experimental results verified its generalization in the extrapolation region.
While teleoperation provides a possibility for a robot to operate at extreme conditions instead o... more While teleoperation provides a possibility for a robot to operate at extreme conditions instead of a human, teleoperating a robot still demands a heavy mental workload from a human operator. Learning from demonstrations can reduce the human operator's burden by learning repetitive teleoperation tasks. However, one of challenging issues is that demonstrations via teleoperation are less consistent compared to other modalities of human demonstrations. In order to solve this problem, we propose a learning scheme based on Dynamic Movement Primitives (DMPs) which can handle less consistent, asynchronized and incomplete demonstrations. In particular we proposed a new Expectation Maximization (EM) algorithm which can synchronize and encode demonstrations with temporal and spatial variances, different initial and final conditions and partial executions. The proposed algorithm is tested and validated with three different experiments of a peg-in-hole task conducted on 3-Degree of freedom (DOF) master-slave teleoperation system.
A novel approach ensuing Gaussian Mixture Model is proposed in this research to segregate the inp... more A novel approach ensuing Gaussian Mixture Model is proposed in this research to segregate the input image into assorted strokes. Once the strokes are extracted, they are combined to reproduce the same character using Gaussian Mixture Regression. The resulted learned strokes are then implemented on KUKA Light Weight Robot and the learned skill is further improved using Reinforcement Learning.
This paper addresses the problem of fitting finite Gaussian Mixture Model (GMM) with unknown numb... more This paper addresses the problem of fitting finite Gaussian Mixture Model (GMM) with unknown number of components to the univariate and multivariate data. The typical method for fitting a GMM is Expectation Maximization (EM). There are many challenges involved when applying EM for fitting a GMM; such as how to initialize the GMM, how to restrict the covariance matrix of a component from becoming singular and setting the number of components in advance. This paper presents a simulated annealing EM algorithm and demonstrates its usefulness in avoiding convergence to the boundary of the parameter space. We have introduced a systematic initialization procedure by using the principals of
stochastic exploration. The experiments have demonstrated the robustness of our approach on different datasets.
In learning by exploration problems such as reinforcement learning (RL), direct policy search, st... more In learning by exploration problems such as reinforcement learning (RL), direct policy search, stochastic optimization or evolutionary computation, the goal of an agent is to maximize some form of reward function (or minimize a cost function). Often, these algorithms are designed to find a single policy solution. We address the problem of representing the space of control policy solutions by considering exploration as a density estimation problem. Such representation provides additional information such as shape and curvature of local peaks that can be exploited to analyze the discovered solutions and guide the exploration. We show that the search process can easily be generalized to multi-peaked distributions by employing a Gaussian mixture model (GMM) with an adaptive number of components. The GMM has a dual role: representing the space of possible control policies, and guiding the exploration of new policies. A variation of expectation-maximization (EM) applied to reward-weighted policy parameters is presented to model the space of possible solutions, as if this space was a probability distribution. The approach is tested in a dart game experiment formulated as a black-box optimization problem, where the agent's throwing capability increases while it chases for the best strategy to play the game. This experiment is used to study how the proposed approach can exploit new promising solution alternatives in the search process, when the optimality criterion slowly drifts over time. The results show that the proposed multi-optima search approach can anticipate such changes by exploiting promising candidates to smoothly adapt to the change of global optimum.
This thesis focuses on having a robot learn to play the game of darts. Playing darts involves mul... more This thesis focuses on having a robot learn to play the game of darts. Playing darts involves multiple tasks. For e.g. how to throw a dart and where to target on the board. It has been shown that with a few demonstrations by the user, the robot can learn how to produce trajectories for hitting a given point on the board; with improvement in accuracy along with the experience. On the other hand we showed how a robot can discover regions for hitting on the board so that it can maximize its expected score.
This paper presents a complete Simultaneous Localization and Mapping (SLAM) solution for indoor m... more This paper presents a complete Simultaneous Localization and Mapping (SLAM) solution for indoor mobile robots, addressing feature extraction, autonomous exploration and navigation using the continuously updating map. The platform used is Pioneer PeopleBot equipped with SICK Laser Measurment System (LMS) and odometery. Our algorithm uses Hough Transform to extract the major representative features of indoor environment such as lines and edges. Localization is accomplished using Relative Filter which depends directly on the perception model for the correction of error in the robot state. Our map for localization is in the form of a landmark network whereas for navigation we are using occupancy grid. The resulting algorithm makes the approach computationally lightweight and easy to implement. Finally, we present the results of testing the algorithm in Player/Stage as well as on PeopleBot in our Robotics and Control Lab.
Uploads
Papers by Affan Pervez
stochastic exploration. The experiments have demonstrated the robustness of our approach on different datasets.
stochastic exploration. The experiments have demonstrated the robustness of our approach on different datasets.