Page - 42 - in Joint Austrian Computer Vision and Robotics Workshop 2020

LearningManipulationTasks fromVision-basedTeleoperation MatthiasHirschmanner,Bernhard Neuberger, Timothy Patten, MarkusVincze Automation and Control Institute, TUWien,Vienna,Austria {hirschmanner,neuberger,patten,vincze}@acin.tuwien.ac.at Ali Jamadi FerdowsiUniversity of Mashhad, Mashhad, Iran a.jamadi@mail.um.ac.ir Abstract. Learning from demonstration is an ap- proach to directly teach robots new tasks without ex- plicit programming. Prior methods typically collect demonstration data through kinesthetic teaching or teleoperation. This is challenging because the hu- man must physically interact with the robot or use specialized hardware. This paper presents a teleop- eration system based on tracking the human hand to alleviate the requirement of specific tools for robot control. Thedatarecordedduring thedemonstration is used to train a deep imitation learning model that enables the robot to imitate the task. We conduct ex- perimentswithaKUKALWRIV+roboticarmforthe task of pushing an object from a random start loca- tion to a goal location. Results show the successful completionof thetaskbytherobotafteronly100col- lecteddemonstrations. Incomparisontothebaseline model, the introduction of regularization and data augmentation leads toahigher success rate. 1. Introduction Robotmanipulationtasksindomesticservicesand industry are highly complex due to the various sys- tem components that are necessary to achieve the goal. As a result, it is difficult to directly program robust robot manipulation strategies. Reinforcement learning is an alternative approach that alleviates the requirement forhumanprogrammingandinsteaden- ables a robot platform to learn from its own experi- ence[4,11,15]. However, thisapproachsuffersfrom substantial training time, with some work reporting training times in the order of months [15]. Learning from demonstration (LfD) is an attractive solution in which a human illustrates how to perform a task and Hand Tracking Demonstrations Learned Policy Figure1.Teleoperating the robotarmusinghand tracking from RGB images. The demonstrations are used to teach apolicy toperforma task (e.g. push thebox to thegoal). the robotattempts to imitate [21,2]. This requiresno humanprogrammingandfarfewertrainingexamples compared to reinforcement learningmethods. Demonstrations for learning are often collected through kinesthetic teaching [1] or teleopera- tion [27]. However, these methods are cumbersome because the human must either physically interact with the robot to generate example motions or con- trol the robot system with specialized hardware that the operator may not have experience with. De- spite the advances of teleoperation systems that en- able novices to improve task performance after only a small number of attempts [8], the hardware is not always readily available. LfD can also leverage sim- ulation [18] or by directly observing human activ- ity [9, 13, 16, 24]. But these approaches demand ad- ditional solutions to transfer acrossdomains. To that end, we present an end-to-end system for LfD through vision-based teleoperation, which al- leviates the necessity for virtual reality and teleop- 42