Seite - 153 - in Proceedings of the OAGM&ARW Joint Workshop - Vision, Automation and Robotics

(a) Surface matte (b) Surface glossy (c) Ground Truth (d) Prediction Lambertian (e) Prediction Semi-glossy (f) Prediction glossy Fig. 2: Yellow pixels show positive and blue pixels show negative gradients. Predictions of the surface gradient in transport direction are shown as follows: (a) Surface of the Lambertian material (first view of the 3D light field data), (b) Surface of the glossy material (first view of the 3D light field data), (c) ground truth surface normal gradient in transport direction∇x used as labels for the regression network, (d) surface normal gradient in transport direction of a Lambertian material learned by the network, (e) surface normal gradient in transport direction of a semi-glossy (gloss = 0.25, roughness = 0.75, see Fig. 4) material learned by the network, (f) surface normal gradient in transport direction of a very glossy material learned by the network. The properties of the different materials can be seen in Fig. 4. One can see that the peaks of the specular lobes (i.e. areas of the biggest negative or positive gradients in (c)) can produce wrong gradient signs. direction is possible. The basic method of photometric stereo uses the fact that the observed intensity (or light response) of a given point is dependent on the surface normal orientation as well as the direction of the light, under the assumption of viewing a Lambertian material and a constant albedo. This can be formulated as follows: e = L·n·a (1) where e=[e1...en]T is a vector of observed intensities, L is a matrix describing the light directions and n denotes the surface normal n=[nx,ny,nz]T. a denotes the albedo which is a scalar value in range a ∈ [0,1]. Inverting this linear equation system yields: n·a = L+ ·e (2) Solving this over-determined least squares problem pro- duces an estimation of the surface normal (Note: L+ is the Pseudo-Inverse of the light direction matrix us- ing, e.g. the Moore-Penrose method [6]). Instead of solv- ing Eq. (1) directly we use a multi-layer perceptron in order to learn a mapping between the intensity re- sponses (eR = [eR1...eR13]T,eG = [eG1...eG13]T,eB= [eB1...eB13]T) in each pixel to the gradient of the surface normal in transport direction∇x = anx/any. Some results of the learned mapping are visualized in Fig. 2. The figure depicts the same small area in all six images. The first two images are examples of how the surfaces that where used for inference from two different material types (matte and glossy) looks like. For both images the first view (i.e. the first illumination angle) of the 3D light field structure was taken. The remaining images show a color-coded visualization of the surface normal gradient ∇x. The intensity vectors eR, eG and eB correspond to the observed intensity values of the different illumination angles (here referred to as views) for each channel of the RGB pixel value respectively. These three vectors (eR,eG and eB) are then stacked vertically for each pixel in order to create the data samples for the network, which then has the formE= [eR1...eR13,eG1...eG13,eB1...eB13]T, where E∈R39 (three color-channels a 13 illumination angles). These data-points of all six datasets are then randomly shuffled in order to avoid a spatial bias due to, e.g. non-constant lighting before presenting it to the network. Since the cumulative number of all points from all datasets is very large (around 3 million samples), a batch based training approach with a batch size of 1000 was used rather then an online learning approach. We used the TensorFlow library [7] for the implementation of the network as well as for the cost function and optimizer. II. RELATED WORK 3Dreconstructionusing2Dimageshasbeenawell studied problem in the field of computer vision. Over the years many different methods to solve this problem arose. In [8] range scanning with stripe patterns were combined with 153