Page - 72 - in The Future of Software Quality Assurance

72 T.Linz • Perceiving:Can it recognizepatternsor situationsbasedonsignals anddata? • Analyzing: Can it identify options for action appropriate to the respective situation? • Planning:Can it select theappropriateorbestoptionsforaction? • Acting:Can it implement thechosenactioncorrectlyandon time? The systematic testing of this chain of tasks requires a catalogue of relevant situations that is as comprehensiveas possible. These situations must be able to be varied in manyparameters (analogousto differentequivalenceclasseswhen testing classic IT systems): For example, the “Mobipick” service robot should be able to detect a closed door as an obstacle under different lighting conditions (daylight, brightsunlight,atnight)andwithdifferentdoormaterials (woodendoor,glassdoor, metaldoor). It must be possible to link the situations into scenarios (successive situations) in order to bring about specific situations in a targeted manner, in order to be able to examine alternative paths of action, but also in order to be able to examine the developmentovertimeforaspecificsituationandthetimely,forward-lookingaction of theautonomoussystem. Such testing of the behavior of a system in a sequence of situations is referred to as “Scenario-based Testing.” [4] proposes “ . . . to transfer relevant scenarios to a central scenario catalogue of a neutral authority in order to create corresponding generallyvalidspecifications, includinganyacceptance tests.”.Thestandardization of formats for the exchange of such scenarios is being worked on. ASAM Open- SCENARIO “ . . . defines a file format for the description of the dynamic content ofdrivingand trafficsimulators. . .. Thestandarddescribesvehiclemaneuvers in a storyboard,which is subdividedin stories, acts andsequences.” [18]. Scenario-basedtestingrequiresthat thesametestprocedureis repeatedina large number of variations of the test environment. When testing classic software or IT systems, however, the test environment is constant or limited to a few predefined variants. If the IT system successfully passes its tests in these environments, it can beconsideredsuitable forusewith loworacceptable risk. If a robot or a self-driving car passes its tests in only one or a few test environments, the system may still be totally unsuitable for real operation, or even pose an extreme safety risk. When testing autonomous systems, the systematic variation of the test environment is therefore an essential and decisive part of the test strategy. 4.4 Requirements for theTest Process The combination of “complex cyber-physical system” with “Mission Complexity” and “Environmental Complexity” leads to an astronomical number of potentially testable scenarios. Each of these scenarios, in turn, consists of situation sequences, with the possibility of variation in the respective system status, the environmental situation and the potential options for action of the system. Since safety require- mentsarenotanisolated“subchapterof thetestplan,”butarepresent throughoutall