Page - 38 - in Joint Austrian Computer Vision and Robotics Workshop 2020

The systems of BMW and IncubedIT have been tested on devices with the following specifications. At BMW an Intel(R) Core(TM) i5 with a 1.70GHz processor and 8GB RAM is used. At IncubedIT an Intel(R)Core(TM)i5-7200Uisusedwitha2.50GHz processorand8GBRAM.OnbothsystemsWindows 10 is installed. Clingo is running in version 5.3.0 withGringo V5.3.0. andClaspV3.3.4. 4.1.EvaluationatBMWGroup In Table 1, the mean value and the standard de- viation of the runtimes for all test scenarios (10 for each scenario) are shown and the number of solved test runs isgiven. If theoptimalsolution isnot found within the BMW-specific time limit of 60 seconds, the solving process is aborted. Consequently, these aborted test runs are not considered in the calcula- tions for themeanandstandarddeviation. Themean performance of the imperative method is for every scenario the best. As shown in the tables, two differ- ent ways of using ASP were tested. In the first one, the solver is directly called inside C#, while in the second we run ASP standalone. The serious perfor- mance issues of the former indicate potential for an improved incorporationof theASPcall inC#. The instances are formed as follow: for the test scenario1,wehave5 tasksand5robots; for the sce- nario2,20tasksand12robots;finally, scenario3has 50 tasks and 30 robots. The positions of the robots and stations of the tasks are randomly placed on a 1000m×1000marea. Looking at the results in Table 1 the imperative solution seems the winner, but in ASP not the Eu- clidean distance for single robot is optimised, but the traveling costs of the whole fleet. So, by using ASP, we are rewarded with far better quality solu- tions, as witnessed by Table 4, where traveling costs for scenario 3 are shown. This scenario is particu- larly interesting, since ASP was not able to find the provable optimal solutions within the time limit. Al- though, while looking for that, solvers like Clingo keep returning thebest solution foundso far, as soon as it finds a better one. Looking at Table 4, we can see that thebestASPsolution foundwithin1second considerably beats the C# solution. However, in this scenario we do not get an improvement with higher time limits. Resultswith theother scenariosare sim- ilar, with the imperative implementation never being close to the ASP traveling distance. This particular problemhighlights theperformance-quality trade-off between the twoapproaches. In Table 2 the mean value and the standard devia- tion of the runtime of every test scenario is shown, considering the charge and park problem. Same rules as before are applied regarding the time limit of 60 seconds. The instances are formed as follow: 2 charging stations (CS), 3 parking places (PP) and 3 robots (R) for scenario 1; 7 CS, 14 PP and 3 R for scenario 2; finally, 17 CS, 33 PP and 30 R for sce- nario3. The imperative C# approach shows for all scenarios a better performance than the ASP- implementations, which, as in the task assignment problem,makesuseofadifferentoptimization,mini- mizing theoverall travellingdistancebetweenrobots and stations, while the C# program prioritizes the robots with the most critical battery level. In con- trast to the task assignment, in this case the problem is too complex to ASP, which does not succeed in findinggoodqualitysolutions (Table4)and, insome cases, it does not succeed to find a solution at all. This observation leads to the assumption that the en- codingof theparkandchargeassignmentproblemin ASP is not optimal, as the performance of the task assignmentencodingforsimilarlyscaled instances is significantlybetter. 4.2.Evaluationat IncubedIT In the IncubedITuse-case, the twoproblems, task assignmentandparkandchargeassignment,arehan- dledtogether,accordingtoourcharacterizationinthe previoussection. InTable3, themeanvalueandstan- dard deviation of the runtimes for the test scenarios solved with the original code and with the in-Java integrated ASP are shown, together with standalone ASP. A timeout is reached when a test run requires more than 30 seconds to find an optimal solution. Test runs that reached the timeoutarenotconsidered in the calculation for the mean and the standard de- viation. The testing environment has a floor area of 100m×86m where the robots are freely movable. The three scenarios we are going to test are formed as follows: for scenario 1, we have 5 robots (R), 3 charging stations (CS), 7 parking places (PP) and 5 tasks (T); 10 R, 6 CS, 14 PP and 10 T for sce- nario 2; finally, for the last scenario we have 30 R, 18CS,42PPand15T. As we would expect from an NP problem solver, the reader can notice from the results that ASP is faster than the Java program while solving small 38