Page - 114 - in Joint Austrian Computer Vision and Robotics Workshop 2020

FrameBorderDetection forDigitizedHistoricalFootage HelmDaniel,PointnerBernhard,KampelMartin TUWien, Institute forVisualComputingandHumanCenteredTechnology {daniel.helm,martin.kampel}@tuwien.ac.at,bernhard.pointner@student.tuwien.ac.at Abstract. Automatic video analysis of digitized his- torical analog films is influenced by video quality, composition and scan artifacts called overscanning. This paper provides a first pipeline to crop the main frame window by detecting Sprocket-Holes and in- terpreting the geometric hole layout to distinguish between two different film reel types (16mm and 9.5mm). Therefore, an heuristic approach based on histogram features is explored. Finally, our results demonstratea firstbaseline for future research. 1. Introduction In the age of digitization analog film collections are digitized by using modern technologies and pro- cesses1. During these processes the frame content as wellas theareaaroundtheexposedframeisscanned. This area includes black borders of the film reel, Sprocket-Holes (SH) or parts of the next or previ- ous frames. This effect is called overscanning and is needed to ensure preservation of significant infor- mation (see Fig.1-a). Furthermore, it is a fundamen- tal procedure for sustainable film digitization and archival. However, for developing automatic video analysis toolsof scannedhistorical analogfilms, this additional information is undesirable and can influ- ence the performance of those systems [1, 3, 4]. The project Visual History of the Holocaust (VHH)2 has been funded in order to digitize analog media col- lections related to the liberation phase of the Nazi concentration camps. These collections are used for furtherexplorationsonautomaticvideocontentanal- ysis. However, they do not include annotated meta- datasuchas thefilmreel typeormaskedoverscanar- eas. Therefore, automatic mechanisms for detecting and removing overscans in film reels such as 16mm 1https://dft-film.com/products/archive-challenges-and- solutions.html - last visit: 2020/02/08 2https://www.vhh-project.eu/en/ - last visit: 2020/02/08 or 9.5mm (see Fig.1-b) can be used to provide more efficientways for exploringanalogfilms. scanwindow overscan a) b) Figure 1. (a) Demonstration of overscanning, (b) real worldexamplesofa16mm(top)and9.5mm(bottom)film reels. Mu¨hling et al. [2] and Zeppelzauer et al. [4] ex- plore the challenges of cinematographic techniques inhistoricalvideos. However, toourbestknowledge no comparable scientific investigation on automat- ically removing overscan information by detecting SH has been published in the last decade. This pa- per proposes a first Frame-Border-Detection (FBD) approach to remove overscan areas in scanned ana- log frames by detectingSHs as well as interpreting the hole geometry and layout. This information is used to classify two different film reel types (16mm and 9.5mm). Moreover, the hole positions are used to extract the final frame window using traditional computervision techniques. 2.Methodology We propose a multi-stage pipeline split into four mainblocks: Threshold-Filtering(THF),Connected- Component-Labelling (CCL), Calculating-Crop- Window (CCW) and Reel-Type-Classifier (RTC). The original input frame is first converted into a grayscale image. In the THF-stage, the input image is thresholded to get a binary mask. The threshold Th is calculated for each input frame dynamically by analyzing the fields 1-6 visualized in Figure 114