Page - 67 - in The Future of Software Quality Assurance

Testing Autonomous Systems 67 with traffic regulations, extremely reliable, fast, predictive obstacle detection is required. When a robot interacts with objects, damage can also be caused indirectly (in addition to the danger of damaging the object or robot). The following examples from[10,p.77] illustrate this: • A service robot is instructed to bring the dishes to the kitchen sink. In order to deposit the dishes near to the sink, it recognizes the modern ceramic stove top as preferable surface and deposits the dishes there... If now a cooking plate is still hot, and there is, for instance,aplasticsaladbowl,oracuttingboard amongst thedishes,obviously, some risks arise. The situation in which a plastic orwooden object is located very close oron top of the cooking plate can be considered as not safe anymore, since the risk of toxic vapor or fire by inflamed plastic or wood ispotentially present. The worst case accident can be a residential fire causing human injury or death. The risk isnotpresent inasituationinwhich theseobjectsare locatedapart thecookingplate (witha certain safety margin), independent from the state of the cooking plate. • A service robot is instructed to “watering the plants.” In this connection, it is assumed that a power plug fell into a plant pot ... If the robot is watering the plant, the risk of electrical shock arises, both, for human and robot. The risk factors can be considered to be the following: The object recognition again recognizes the power plug while having the watering can grasped (or any plant watering device) and additionally, it can be detected that there is water in the watering can (or similar device). In consequence, a rule should be integrated that instructs the robot not to approaching too close with the watering can toapowerplug,or the like, inorder toavoid that it is struck byawater jet. In order to be functionally safe, a highly or fully autonomous system must therefore have appropriate capabilities and strategies to identify situations as potentially dangerous and then respond appropriately to the situation in order to avoid imminent danger or minimize13 damage as far as possible. The examples cooking plate and watering the plants make it clear that pure obstacle detection alone is not always sufficient. In complex operational environments with complex possiblemissionsof theautonomoussystem, some dangerscan onlybe recognized if a certain“understanding”ofcause-effect relationships isgiven. Such capabilities and strategies must be part of the “intelligence” of highly autonomous systems. The intended system functionality and the necessary safety functionscannotbe implementedseparately,butare two sidesof thesame coin. 3.2 Safety in FailureMode If parts of the autonomous system fail, become damaged, or do not function as intended (because of hardware faults, such as contaminationor defect of a sensor), 13The media in this context mainly discuss variants of the so-called “trolley problem”, that is, the question ofwhether and how an intelligentvehicle should weigh the injury ordeath of one person or group of persons at the expense of another person or group of persons in order to minimize the consequences of an unavoidable accident (see [11]).