Page - 115 - in Joint Austrian Computer Vision and Robotics Workshop 2020

P1 (x1, y1) y1 x1 b) c) Film_Reel_Type: 16mm THF CCL RTC CCW Input Output Scale-factor: 1.33 1 2 34 5 6 5 a) Figure2. Illustrationof (a) thepipeline, (b) thegenerationof thecornerpointand(c)calculationof thefinalcropwindow. Exp. P R Acc mIoU@0.95 IoU@0.70 16mm-fixed 0.95 0.94 0.88 - - 9.5mm-fixed 0.94 0.68 0.82 - - 16mm-dyn. 0.96 0.84 0.90 - - 9.5mm-dyn. 0.97 0.64 0.81 - - overall-fixed 0.948 0.812 0.85 0.747 0.867 overall-dyn. 0.962 0.74 0.86 0.763 0.895 Table 1. Precision (P), Recall (R) and Accuracy (Acc) on the test set - classification of the reel-types 16mm and 9.5mm. mean Intersection over Union scores at thresh- olds: 0.95and0.7. 2-a. After a filtering process, the mask is used in the CCL-stage for labeling all detected potential SH. This step includes a further filtering process to remove outliers. Finally, this step is the base for the CCW- and RTC-stage. In CCW the corner points are calculated as demonstrated in Figure 2-b. The center of the resulting square is used as reference point for the final crop window which is defined with a configurable scale factor (e.g 1.33) to get the correct scaled frame crop related to the original film reel such as 960x720 pixels (see Fig.2-c). In the RTC-stage, our pipeline is able to classify the input frame into the reel types: 16mm and 9.5mm. Therefore, the locations of the labeled holes are analyzed in the masked input image. SHs in the fields 5 and 6 (see Fig. 2-a) are related to the 9.5mm film reel whereas the other ones identify the 16mm reels. 3.Results&Conclusion The evaluation of our pipeline is based on a self-generated dataset including 100 labeled frames randomly selected out of 10 videos related to the National-Socialism3. The dataset contains 50 anno- tated frames for each class: 16mm and 9.5mm reels. The metric mean Intersection over Union (mIoU) is usedforevaluating thepredicted locations. Precision 3http://efilms.ushmm.org/ - last visit: 2020/02/11 and Recall are utilized to evaluate the classification of the two reel types. For the evaluation, two exper- iments have been conducted: a fixed and dynamic Th. The results show that the mIoU scores signifi- cantlydependingontheTHFprocess. Historicalfilm frames include damaged and under-/overexposed ar- eas which make the selection of an optimalTh chal- lenging. Furthermore,SHs are not on the same po- sitions in each frame due to the movements and the varyingspeedofthefilmreelduringthescanprocess. The results are summarized in Table 1. We provide a first baseline for further research. However, opti- mizing our pipeline as well as using Deep Learning- based methods are planned to improve detection and classification results. Acknowledgments The project VHH has received funding from the EU’sH2020researchprogram(GrantNo. 822670). References [1] D. Helm and M. Kampel. Video Shot Analysis for DigitalCurationandPreservationofHistoricalFilms. InS.RizvicandK.RodriguezEchavarria,editors,Eu- rographics Workshop on Graphics and Cultural Her- itage.TheEurographicsAssociation, 2019. [2] M. Mu¨hling, M. Meister, N. Korfhage, J. Wehling, A. Ho¨rth, R. Ewerth, and B. Freisleben. Content- based video retrieval in historical collections of the german broadcasting archive. Int. J. Digit. Libr., 20(2):167–183, June 2019. [3] M. Seidl, M. Zeppelzauer, D. Mitrovic´, and C. Bre- iteneder. Gradual transition detection in historic film material - a systematic study. J.Comput.Cult.Herit., 4(3):10:1–10:18, 2011. [4] M. Zeppelzauer, D. Mitrovic, and C. Breiteneder. Archive film material - a novel challenge for auto- mated film analysis. The Frames Cinema Journal, 1(1), 2012. 115