Page - 52 - in Joint Austrian Computer Vision and Robotics Workshop 2020

3.ControlandStateEstimation The manipulator as a forest crane is vastly different compared to a standard industrial robot: the rather unconventional design requires detailed geometrical knowledge to derive the kinematic model. Also, the hydraulic driving system suffers from heavy vibrations, backlashes and jerks which require detailed dynamic parameters for proper modelling. To capture the complex relationships on existing machines where CAD and dynamic modelsare rarelyavailable,weusemachine learning techniques for the prediction of kinematics and dynamics parameters. Additional inertial and visual sensors further help to re-fine the overall state estimation including the adaptive estimation of the dynamicparametersdefining thesway-motionof the two free joints on the end-effector. This adaptive estimation of the kinematic and dynamic parameters allows a simplified manipulator model for adaptive control schemes when picking and placing logs with swaymotion. 3.1.AutomatedGraspingPointPrediction To find the optimal points for the gripper to grasp a log (or logs), it is necessary to recognize graspable objects in the surrounding area of the robotic manipulator and calculate possible candidates. A candidate is defined as a point/area of the log which can successfully be grasped by the gripper. A ZED camera is used for image acquisition and consists of a stereo camera system capturing high resolution RGB-D images from the scene. Core component of the prediction method is a deep learning approach using a Convolutional Neural Network to predict grasping candidates in 2D image space, similar to [5]. The depth information is used for: 1) Automatic annotation of training data for a deep neural network by leveraging sequential depth data. This method is a step towards continuous learning making it easily possible to generate new ground truth training data during real time system application. 2)Calculationof thefinal3Dpositionof the grasping point from the previously predicted 2D grasping candidate. Fig. 6 shows a sample scenario with some logs remaining in the picking area and marked grasping locationsby theAI method. 3.2.ConclusionandNextSteps We proposed a mechanical setup for training a crane model of the wood industry for automated Figure 6. Model logs in the picking area of the 1:5 scaled model crane with marked grasping positions by our AI basedmethod. Redmarks thedesired locationsof the two grippers in theend-effectorof thecrane. log grasping. The setup allows automated operation such that continuous learning without human intervention can be possible. Retrofittable sensors allow additional sensing capability in order to autonomously control the grasping procedure and to verifycorrectpickingofthedesiredlogs. Thecurrent results show that while the alignment of the desired gripperpositions tograspa log iscorrectlypredicted by the AI, not all suggested locations are ideal in view of the center of gravity. Next steps will include the feedback of the capacitive sensors to correct the AIdecision inanautomated learningprocedure. References [1] H. Gietler, C. Stetco, and H. Zangl, “Scalable retrofit angular position sensor system,” in IEEE International Conference on Instrumentation andMeasurement,Dubrovnik,May2020. [2] L. Faller, C. Stetco, and H. Zangl, “Design of a novel gripper system with 3d- and inkjet-printed multimodal sensors for automated grasping of a forestry robot,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS),Nov2019,pp.5620–5627. [3] T. Mitterer and H. Zangl, “Beyond pure sensing: Ieee 21450 in digitalization of the development cycle of smart transducers,” IEEE Instrumentation & Measurement Magazine, April 2020. [4] Sensors for automated grasping of forestry robots. www.youtube.com/watch?v=B1S46LqfG48. [5] F.-J. Chu, R. Xu, and P. A. Vela, “Real-world multiobject, multigrasp detection,” IEEE Robotics and Automation Letters, vol. 3, no. 4, pp.3355–3362,2018. 52