Page - 162 - in Proceedings of the OAGM&ARW Joint Workshop - Vision, Automation and Robotics

TABLE I THIS TABLE LISTS THE NAME, NUMBER (PERCENTAGE) OF DISOCCLUDED PIXELS AND THE CHARACTERISTICS OF THE IMAGES USED IN THE SUBJECTIVE STUDY. Name Disocclusions Characteristics Arm 54050 (2.6%) low-textured background Bird 29790 (1.4%) moderately textured background Crowd 57711 (2.7%) cluttered repetitive background Edge 51173 (2.5%) highly textured background Flower 50483 (2.4%) repetitive background describes the test material, the inpainting techniques used for comparison and the selected subjective methodology including a description of the test environment and subjects. A. Dataset All inpainting methods are evaluated on footage from a movie sequence. Five still images – termed as Arm, Bird, Crowd,Edge andFlower – have been chosen as test images, with a resolution of 1920×1080 pixels. The selected im- ages cover different image characteristics including varying densities of background texture and diverse amounts of disoccluded pixels, as summarized in Table I. B. Algorithms We compare our depth-guided PM inpainting approach (DPM), which was described in Section 2, against our implementation of PM [2] with constant patch sizes of 51×51 pixels, the image completion function content-aware fill (CAF) of Adobe’s Photoshop CS53, which does not use depth information, and horizontal background replication (HBR) [7]. We use the following, same constant parameter settings to generate the results: {τ1,τ2}={10%,10%}. The thresholds have been chosen to provide a small but reason- able amount of valid pixels to be used for patch matching while preventing target patches from becoming too large, which would lead to blurrier inpainting results and increase the overall runtime of the algorithm. C. Subjective assessment procedure The Pair Comparison (PC) method has been chosen to quantify the subjective ratings [12]. In the PC method, a pair of stimuli is compared and the subjects are asked to rate the quality of the stimuli in terms of preferences using a ternary scale (i.e., stimulus A is preferred, stimulus B is preferred, or stimuli A and B are equally preferred). Particularly, using 4 inpainting approaches and 5 images, a total number of 30 pair comparisons had to be performed by eachsubject.Eachpairwaspresentedsuccessively in random order. The subjects were allowed to switch interactively between the two stimuli of a pair. Moreover, each subject performed a trial run in which the test methodology was introduced. We compute the quality score for each method by increas- ing its respective counter by 1 in case of a preference and 0.5 in case of an equal valuation. The accumulated value is 3http://www.adobe.com/technology/projects/content-aware-fill.html Fig. 2. Pair comparison scores of the subjective study. then divided by the number of comparisons per method and by the total number of participants. Hence, the final score shows the percentage of comparisons “won”, e.g., a value of 100 indicates that this method has always been preferred over any other approach. The test sequences were displayed on a 23.6′′ stereoscopic display (i.e., Acer GD245HQ) with a native resolution of 1920×1080 pixels and the NVIDIA 3D vision controller. To provide an ideal test setup, the room was darkened to avoid external visual disturbances and the viewing distance was set to one and a half times the screen size. Seventeen non-expert observers (six female and eleven male observers aged between 17 and 49) participated in the study. All of the subjects were screened for visual acuity, color vision and stereo vision according to ITU-R BT.1438 recommendation [12]. IV. RESULTS AND DISCUSSION In Fig. 2, the PC scores obtained for the five test images are presented, grouped by the evaluated inpainting methods. Our proposed approach DPM performs best and is preferred on average in 72.75% of all comparisons. In contrast, the other PatchMatch-based inpainting methods PM and CAF attain significantly lower average PC scores of 34.51% and 38.43%, respectively. Fig. 3 offers a closer look at some examples of inpainted regions. Regarding our approach, the study participants re- marked a clear delineation of the foreground objects. A possible explanation is the reduction of artifacts caused by foreground color blur, which are mainly perceived as unnatural shadows of the objects (cf. DPM and PM in the second and third row of Fig. 3). Additionally, it can be seen that for holes at the image border, it is possible to inpaint coherent information by using adaptive instead of fixed-size patches. The lower score of our approach (60.78%) compared to HBR (74.51%) for the image Flower may be caused by significant inaccuracies of the corresponding depth map. In 162