Seite - 45 - in Joint Austrian Computer Vision and Robotics Workshop 2020

Webcam Hand Tracking Xtion DemonstrationsDemonstrationsDemonstrations Robot State Robot Control Hand Poses End-Effector Poses RGB Images RGB-D Images RGB-D Images Desired End-Effector Poses End-Effector Poses (120, 160, 3) 7x7 Conv 64 /2 Concatenation Current Position 7x7 Conv 16 /2 (120, 160, 1) Final Position Auxiliary Predictions Recent Positions (5 ⨉ 3) (1 ⨉ 3) Figure 2. Overview of the system. The dashed lines show the procedure to collect demonstrations for training. The continuous lines show the information flow duringpolicy execution. the changing lighting conditions in the test environ- ment. Additionally, we added dropout of the recent end-effector positions to avoid the robot following the same trajectory during most executions and not taking theobjectposition intoaccount. Theoverall loss isdefinedas L(θ)=λl1Ll1+λl2Ll2+λcLc+λsLs+λauxΣaL(a)aux. (2) The first two terms are the l1and l2 losses.Lc is the cosine lossandLauxare the l2 lossesof theauxiliary predictions. Compared to [27],weadded the loss Ls=exp(−||πθ(ot)||2) (3) that penalizes very slow speeds. The weights were chosen as λl1=1.0, λl2=0.01, λc=0.05, λs=0.1, andλaux=0.01. 4. Experiments Thissectionpresents theexperimental results. We first describe the setup and procedure for collecting demonstration data. We analyze the performance of ourmethodwith respect to thenetworkdesign. 4.1.ExperimentalSetup All experiments are conducted with a KUKA LWR IV+ [3] robotic arm using the provided con- trol unit. The arm has 7 degrees of freedom and is controlled with position commands for the joints. The arm is mounted on the ceiling with a small ta- ble standing underneath it on which the target object (box) rests. The goal region is marked with tape. An ASUS Xtion RGB-D camera is mounted to the ceil- ing to capture the scene from above. For hand track- ing,aseparatewebcamisusedandfacestheoperator. The algorithms for the hand tracking and the task execution run on a remote PC connected to the KUKA control unit via Ethernet. The communica- tion between the remote PC and the control unit is enabled through thekuka-lwr-rospackage1 using the fast research interface (FRI) [23]. For data collection, the teleoperator directly faces the robot and the webcam. For each demonstration, the box is positioned randomly on the table. The teleoperator moves the box to the goal position us- ing our control scheme. We collected 98 demon- strations with an average length of 42.8s with a rate of 10Hz for our evaluation. That is signifi- cantlyabovetheaveragedemonstration timeper task of [27], which is between 3.7s and 11.6s and neces- sitates our changes to the architecture to deal with these imperfectdemonstrations. 4.2. Results For the evaluation, the workspace of the robot on the table is divided into a grid of 9 different posi- tions with 20cm intervals. Per position, the learned policy is executed for4different rotationsof thebox (−45◦,0◦,45◦,90◦). We measure both if the box is pushed towards the goal (started push) as well as if at least part of it is pushed into the goal (success). If the robot starts to push the box, but loses it, we restart the policy manually and keep the box in the same position when the end-effector stops or leaves the workspace. This could be automated with a sim- pleheuristic. If the taskcanbeachieved inaconsec- utive trial, westill count it as a success. As shown in Table 1, our learned policy started to push the box in the right direction in 86.1% of the cases and reached the goal in 58.3% of the overall attempts. A reason for most failure cases is the grid natureofourworkspaceseparation,whichinherently tests the roboton theedgesof itsworkspacewhere it ismuchmoredifficult toperformthe task. We conducted an ablation study to evaluate our changes to the original architecture of [27]. We re- 1https://github.com/epfl-lasa/kuka-lwr-ros 45