Page - 144 - in Joint Austrian Computer Vision and Robotics Workshop 2020

ties of a skeleton are visible, we employ the follow- ingheuristic rule:CU/max(CL,10−4)>T,withT beinganempiricallydeterminedthresholdof1.5. Fi- nally, positive predictions of the rule-based classifier and thewaterdetector impliesapositivedetectionof apersonstanding inknee-highwater. 3.Datasets Experiments are carried out on two datasets pro- vided by the MediaEval Benchmark Multimedia Satellite Task 2018 (MMSat18) and 2019 (MM- Sat19) respectively [1, 2]. All available data is man- uallyannotated(water/nowater)andusedtotrain the water detector. A total of 13.761 image annotations (5.395water,8.366nowater)alongwithcorrespond- ing image URLs (incl. download tool) as well as our ResNet50modelweights canbeaccessedpublicly1. 4.Results&Discussion For experimental evaluations, we randomly split the MMSat19 data into training (80%) and valida- tion (20%) sets preserving class priors. Testing is performed on the (non-public) test set of the MM- Sat19benchmark. For theglobal approach (GA),we usedthepipeline inFigure1A.First, thewaterdetec- tor is trained only on the MMSat19 data (GA-1) and lateronbothdatasets (GA-2). For the localapproach (LA), we used the pipeline in Figure 1B with MM- Sat19 data. Finally, we apply majority voting to all threeapproaches. Due to the imbalanced data, we used macro aver- aged F1-scores as performance measure. The exper- imental results surpass the random baseline of 0.5, which shows that our models are able to learn use- ful patterns. The results on the test set show only minor differences between the four approaches. The overall performance is similar on the validation and test sets, which indicates a good generalization abil- ity. The classification accuracy of the water detec- tor is quite high with 88% (not shown in Table 1). Themainsourceof failureare falsedetectionsof the pose trackerdue toocclusionsby foregroundobjects and reflections in the water (see Figure 2). Potential improvements identified include the use of several poseestimators trainedoncontent fromdifferent en- vironments, e.g., ruralandurbanareas. Additionally, pixel-wiseclassification (segmentation)ofwaterand humanbodiescouldbeuseful todealwithocclusions and reflection in thewater. 1https://tinyurl.com/waterDetectionDataset Figure 2: Challenges: misleading images (left), wa- ter reflections (middle) andocclusions (right). Approach Validation (P/R/F1) Test (F1) GA-1 0.58 0.67 0.61 0.61 GA-2 0.55 0.60 0.56 0.59 LA 0.58 0.77 0.60 0.59 MajorityVoting 0.59 0.68 0.61 0.61 Table 1: Macro-averaged precision (P), recall (R), and f1-scores forvisualflood level estimation. 5.Conclusion Our experiments show that pose estimation and water detection provide useful clues for the assess- ment of flood levels. By building upon skeletons, the presented approach is invariant to gender, age and height. Main challenges for robust water level estimation represent occlusions and reflections. For future work, a larger, more balanced and more het- erogeneousdataset isneeded. Acknowledgments This work was supported by the Austrian Re- search Promotion Agency (FFG), Project nos. 855784,856333, and865973. References [1] B.Bischke,P.Helber,S.Brugman,E.Basar,Z.Zhao, and M. Larson. The multimedia satellite task at MediaEval 2019: Estimation of flood severity. In WorkingNotesProc.ofMediaEvalWshp.(toappear). [2] B. Bischke, P. Helber, Z. Zhao, J. de Bruijn, and D. Borth. The multimedia satellite task at MediaE- val2018: Emergencyresponseforfloodingevents. In WorkingNotesProc. of theMediaEvalWshp., 2018. [3] Z.Cao,T.Simon,S.-E.Wei, andY.Sheikh. Realtime multi-person 2D pose estimation using part affinity fields. In Proc. ofCVPR, 2017. [4] M. Larson, P. Arora, C.-H. Demarty, M. Riegler, B. Bischke, E. Dellandrea, M. Lux, A. Porter, and G. J. Jones. ”Working Notes Proc. of the MediaEval 2019Wshp. (toappear)”. [5] M. Zaffroni, L. Lopez-Fuentes, A. Farasin, P. Garza, and H. Skinnemoen. AI-based flood event under- standing and quantification using online media and satellite data. In ”Working Notes Proc. of the Me- diaEval2019Wshp. (to appear)”. 144