Seite - 152 - in Proceedings of the OAGM&ARW Joint Workshop - Vision, Automation and Robotics

Photometric Stereo in Multi-Line Scan Framework under Complex Illumination via Simulation and Learning Dominik Hirner12, Svorad Sˇtolc1, Thomas Pock2 Fig. 1: Visualization of the image stack created by the multi-line scan acquisition. The middle part shows the EPI- lines (here a slice through the image stack). The dashed line represents the read out of one such EPI-line with the respective RGB intensity vector e, which is used in order to infer (by training the network) the surface gradient in transport direction∇x. Abstract—This paper presents a neural network implemen- tation of photometric stereo formulated as a regression task. Photometric stereo estimates the surface normals by measuring the irradiance of any visible given point under different lighting angles. Instead of the traditional setup, where the object has a fixed position and the illumination angles changes around the object, we use two constant light sources. In order to produce different illumination geometries, the object is moved under a multi-line scan camera. In this paper we show an approach where we present a multi-layer perceptron with a number of intensity vectors (i.e. points with constant albedo under different illumination angles) from randomly chosen pixels of six materials with different reflectance properties. We train it to estimate the gradient of the surface normal along the transport direction of the given point. This completely eliminates the need of knowing the light source configuration while still remaining a competitive accuracy even when presented with materials which have non-Lambertian surface properties. Due to the random pooling of the pixels our implementation is also independent from spatial information. I. INTRODUCTION The goal of photometric stereo is to estimate the surface normals (and therefore 3D information) of an object using 1AIT Austrian Institute of Technology GmbH, Vision, Automation & Control, Vienna, Austria {dominik.hirner, svorad.stolc}@ait.ac.at 2Graz University of Technology, Institute for Computer Graphics and Vision, Graz, Austriapock@icg.tugraz.at 2D images. This is done by exploiting Lambert’s cosine law [1], which states that the intensity of the light at a point isdirectlyproportional to thecosineof its surfacenormaland the angle of the incident light (see Eq. (1)). By measuring the light intensity of each point under different known and fixed illumination angles the surface normal of each point can be calculated. This approach was first introduced by Woodham in 1980 [2]. However, this equation only holds with the assumption of a Lambertian surface, i.e. a surface that scatters the light in all directions equally. In case of specular reflections the observed intensity of a point also dependents on the position of the observer and therefore the basic approach of photometric stereo does not hold. In the standard photometric stereo approach the orientation and position of the observer (i.e. camera) is known and fixed. Light-field processing via light-field cameras can be seen as anadd-on to thegeneralphotometric stereo idea.Alight-field is a 4-D radiance function written asL(u,v,s,t), where (u,v) denotes the angle, and (s,t) denotes the position of each light ray respectively. To capture a light-field with a camera, a number of different approaches exist, for instance commer- cially avaliable plenoptic cameras such as the Lytro [3] or by using an array of cameras (multi-camera array) [4]. Using multi-line scanacquisitionwitha light-field inorder tocreate 2.5/3D surface structure was first introduced in [5]. The same multi-line scan light field camera was used in this approach, which acquires multiple single lines (in our implementation 13) with different viewing angles at one time. Between the active lines on the sensor there are a number of predefined inactive lines (in our implementation 40), so that different viewing angles are produced within one acquisition step without the need of placing several cameras (as e.g. in a multi-camera array). In our setup an object is placed underneath the camera and is transported in a defined direction over time with two constant light strips placed orthogonal to the transport direc- tion. Between two acquisition steps si and si+1 the object has to move the distance equivalent by exactly one pixel. After the acquisition process, the single lines acquired by one such step of each active line on the sensor are concatenated and thus all possible lighting angles and a number of different views are created. This produces a 3D light field structure (two spatial and one directional dimension), instead of the usual 4D structure. This 3D light field can be represented as an image stack that can be seen in Fig. 1. This allows for a fast in-line acquisition suitable for industrial inspection. However, since different lighting responses are dependent on the movement of the object, only inference in the transport 152