Seite - 111 - in Proceedings of the OAGM&ARW Joint Workshop - Vision, Automation and Robotics

FORMS – ForensicMarks Search∗ Manuel Keglevic and Robert Sablatnig1 Abstract—The goal of the project FORMS is to support the search and comparison of toolmarks by forensic experts with a semi-automatic system in order to identify and solve connected criminal cases. The proposed methodology uses a neural network with triplet architecture to compute similari- ties between toolmark images. Further, to allow an accurate evaluation under real-world conditions a dataset consisting of more than 3000 images of cylinder locks with toolmarks from real criminal cases is created as part of the project. I. INTRODUCTION Lock snapping is a common way for forced entry in Europe. The unique imprints of the pliers used for these break-ins significantly support the investigation of such of- fenses and are crucial as evidence in the following court cases. However, manual examination of these toolmarks in order to find multiple uses of the same tool is a time consuming task due to the amount of samples. Therefore, the goal of the project FORMS (Forensic Marks Search) is a two-fold solution for this problem: firstly, an application which allows for search and comparison of toolmark images stored in a centralized database. Secondly, a methodology based on state-of-the-art machine learning techniques for an automatic by similarity in order to reduce the amount of images requiring manual examination. The project started in Fall 2015 and is funded by the Austrian Security Research Programme KIRAS. The project partners are the Computer Vision Lab of the TU Wien, the Bundeskriminalamt (Criminal Intelligence Service Austria), the CogVis GmbH, and VICESSE. II. TOOLMARK DATASET Since the validity of comparative forensic examination of toolmarks has been challenged in court, various papers have been published on the comparison of toolmark images [5]. This lead to the development of methodologies for the automatic comparison of striated toolmarks and datasets like the NFI Toolmark Dataset published by Baiker et al. [1]. However, in contrast to forensic images of toolmarks from real criminal cases, these toolmarks were created in constrai- ned environments. Therefore, to allow an evaluation of the real-world performance of toolmark comparison methods, a new dataset was created as part of the FORMS project. This dataset, created by photographing cylinder locks seized duringcriminal investigationsusingamicroscope,consistsof approximately 3000 toolmark images from about 50 different *This work has been funded by the Austrian security research programme KIRAS of the Federal Ministry for Transport, Innovation and Technology (bmvit) under Grant 850193. 1Manuel Keglevic and Robert Sablatnig are with the Computer Vision Lab, TU Wien,mkeglevic@caa.tuwien.ac.at Fig. 1: Image of a broken lock cylinder with toolmarks created by a locking-plier (left). Matching toolmarks on two lock cylinders photographed using a comparison microscope with a magnification factor of 20 (right). crime series. In order to investigate the influence of lighting each of the 154 cylinder was photographed on both sides under 11 different lighting conditions. In Figure 1 a broken lock cylinder with toolmarks is shown on the left side. The appearance of the toolmarks can vary heavily due to material differences in the material, the force applied or the lighting conditions. For example in Figure 1 on the right the appearance difference due to varying depth of the toolmarks is illustrated. III. METHODOLOGY As shown in Figure 1 on the right side, extracting fo- reground (the toolmark) from background (lock cylinder) is challenging due to varying background structure depth of the toolmark. Therefore, the region of interest is marked by the forensic expert by hand in a first step. Local image patches extracted in these regions of interests are then compared using a neural network. The network architecture used is based on triplet learning which has for instance been applied to face detection [4] and local image patches [2]. Further, Keglevic and Sablatnig showed [3] that it can be used to compute similarity measures for striated toolmarks. To capture the unique properties of this problem like varying lighting conditions and background the neural network is trained from scratch. In order to create the necessary training dataa ground-truth toolwas created asa plugin for the image viewer nomacs2. This tool allows the definition and pixel perfect alignment of matching polygons in toolmark images. Using these annotations matching patches for the training and evaluation process can be created along these matching polygons. First results show promising result, however for an in-depth assessment of the performance an evaluation has to be performed as soon as the whole dataset is annotated. 2https://github.com/nomacs/nomacs-plugins 111