Page - 143 - in Joint Austrian Computer Vision and Robotics Workshop 2020

HowHigh is theTide?EstimationofFloodLevel fromSocialMedia JuliaStrebl,DjordjeSlijepcevic,ArminKirchknopf, MuntahaSakeena,MarkusSeidl,MatthiasZeppelzauer St. Po¨ltenUniversityofAppliedSciences,Austria {firstname.lastname}@fhstp.ac.at Abstract. The availability of social media data rep- resents an opportunity to automatically detect and assess disasters to better guide emergency forces. We propose a method for flood level estimation from user-generated images to support assessing the severity of flooding events. Furthermore, we provide labeled data for water detection. Results on a public benchmark dataset are promising and motivate fur- ther research. 1. Introduction The visual estimation of flood levels is a novel task. In this paper we aim at detecting images with a certain water level, i.e. where the water is at least knee-high. Our work is based on preliminary work fromtheMediaEval2019SatelliteTask[1]. Ourcon- tribution is twofold: we demonstrate the feasibility of visual flood level estimation by combining a su- pervised water detector with pose estimation and we providenovel imageannotations forwaterdetection. Related work focuses on either visual, textual or multimodal flood level estimation from social media content [1]. Zaffaroni et al. [5], for example, com- bine multiple pre-trained networks for the estima- tion of flood level. Further approaches can be found in[4]. Weaimatpresentingasimpleandefficientap- proach toprovideabaseline for futurecomparison. 2.Methods Input toourapproacharesocialmediaimages. We propose two approaches that build upon three com- ponents: (i) a supervised water detector that predicts whetheracertainimageorimageregioncontainswa- ter, (ii) a pose estimator that detects people and their joints and (iii) a rule-based fusion module that com- bines the information from the water detector and the pose estimator to make a final decision. The first approach (see Figure 1A) aims at detecting wa- terwithin thewhole imageanddetectingat leastone person with concealed lower body parts. The second approach (see Figure 1B) performs water detection locally around each detected human body. If at least for one body the model detects concealed lower ex- tremities and water in the vicinity, the image is as- signed toknee-highwater. class 0 class 1 > T > T Water Detector Rule-based Classfier OpenPose OpenPose class 0 class 1 > T A) B) Water Detector Rule-based Classfier Figure1: Global (A)and local (B)approach. We employ ResNet50 (pre-trained on ImageNet) for water detection. Images are resized to the net- work’s input size (227x227)whilekeeping theorigi- nal aspect ratio. Horizontal flipping, brightness vari- ations and non-uniform re-scaling of the images are applied for data augmentation. The top five layers are fine-tuned (6 epochs, batch size 256) before the whole network is trained using Adam optimizer (10 epochs, batch size 32, learning rate 10−4). We em- ploy OpenPose [3] to detect body joints from de- picted human bodies. To filter out unreliable skele- tons, we exclude those with a confidence score (CU) - calculated from the two most robust upper body parts (head and chest) - below an empirically esti- mated threshold of 0.6. We calculate a mean confi- dence score (CL) over the lower body parts (knees and feet). To determine whether the lower extremi- 143