Page - 150 - in Proceedings - OAGM & ARW Joint Workshop 2016 on "Computer Vision and Robotics“

Figure 3- Depth Camera based tracking for Spatial Augmented Reality [17] Figure 4 - projected buttons Tightening order and poses of screws might be programmed by pointing to the screws with the finger [20]. Object and pose recognition introduces TUI to the world of AR. We propose a system that provides spatial detection functionality of teach-in devices (e.g. a spherical marble) that can be manipulated by the human. The system consists of one or more 3D sensors and a calibrated projector. Information on dynamic marble pose (resting marble may denote an underlying screw to be tightened) may be used for the programming of process points. A marble is placed on the cylinderhead cover. A projected interface element is pushed by the programmer who has to hold in order to avoid accidental acknowledgements. Once acknowledged, the 3D pose of the marble and thus the underlying screw is programmed to the system. Figure 5 - system setup Figure 6 - SAR-TUI based process point programming The system architecture (Figure 8) motivates advanced functionality in terms of robot hand guidance. Figure 7 shows first results of the environmental modelling system. An arbitrary placed object is recognized by the 3D camera and a virtual environmental model is updated (in realtime) with the model of the recognized object in its correct pose. Force feedback algorithms are planned to render intuitive feedback for the user according to elements of the environmental model. E.g. boundary surface of volumes containing obstacles should make it impossible for the user to move the robots to or through such areas. Therefore negative force (as far as movement direction is concerned) has to be exerted to the hand guiding user. Positive forces may attract the user to process points contained by the model. projector projected button 3D sensors 150