Page - 156 - in Proceedings of the OAGM&ARW Joint Workshop - Vision, Automation and Robotics

TABLE II: MSE of each individual whole dataset applied to the network. On the left we report the accuracy of our neural network, then the accuracy of the Lambertian model when80%of theLambertiandatasetwasused toestimate the illumination matrix L from Eq. (1) as an analogy of learning (L.m.L stands for Lambertian model Lambertian datasets). In the last column 80% of all datasets were used (Lambertian model all dataset). Dataset MSEnetwork MSEL.m.L MSEL.m.a Lambertian 0.01637 0.02435 0.02804 g025r025 0.01537 0.05550 0.03268 g025r075 0.01835 0.02063 0.02741 g075r025 0.01760 0.24619 0.10237 g075r075 0.01795 0.03233 0.02930 glossy 0.03722 0.89302 0.37912 avg. 0.02047 0.21200 0.09982 also be seen in the correlation plots in Fig. (6) where some of the outliers from the glossy dataset also show up in the correlation plot for the whole train and test dataset. We compare our results with the model-based Lambertian approach by solving Eq. (1) for L as an analogy of learning with the samedataset training/testingsplit as forourmachine learning approach. For this the assumption of an constant albedo with a value of 1 was taken. Despite it can be argued that the Lambertian model only works for Lambertian materials. The quantitative results are reported in Table II. It can be seen that the L.m.L. approach completely failed for the glossier material datasets. On the other hand the L.m.a. approach proved to perform in average about twice as good improving significantly especially on the glossy cases. Last but not least, we show that our neural network approach outperforms the traditional photometric stereo by far for the given task, especially for glossier material. IV. CONCLUSIONS AND FUTURE WORK In this paper we showed a neural network based machine learning approach in order to learn a mapping between intensity vectors (i.e different illumination angles) of points with different reflectance properties to a surface normal gradient. We showed that in our approach we do not need to know the position and direction of the light source as well as no spatial information and were still able to produce competitive accuracy. The proposed machine learning ap- proach outperformed the standard photometric stereo based on the Lambertian model by 5-10 times. We tested the network on synthetically generated data and showed that our implementation works well even for very glossy surface properties. In our simulations the train error converges very fast which suggests that we did not yet reach the absolute best accuracy possible and increasing the number of features as well as training the network for longer may still increase the overall prediction of the multilayer perceptron. The mean absolute error (MAE) can be advantageous as it is more robust against outliers [23], however since we excluded strong outliers manually in our datasets beforehand we did not need to use MAE. Nevertheless, exploring this cost function in the future should be done. (a) Train (b) Test (c) Lambertian (d) Semi-Glossy(g025r075) (e) Glossy Fig. 6: (a-e) Show the correlation plot between label and prediction of∇x for the respective datasets of 100 samples uniformly taken from the set. (a) combines 80% of all datasets (which were randomly chosen). (b) combines 20% of all datasets (which were randomly chosen). (e) shows some outliers where the sign of the gradient was wrongly predicted due to the high specular response. The stronger outliers on (a) and (b) also come from this set. For future work we intend to extend this approach to perform material classification (e.g. classify matte, glossy, semi-glossy material etc.) as well as learning the albedo of thecreateddatasets. In thispaperweonlyusedsyntheticdata in order to prove the correctness of the method, however an evaluation on real-world data for the trained networks would be the next step. Additionally, we want to investigate the possibilities of inference on the surface normal gradient orthogonal to the transport direction. REFERENCES [1] J.H. Lambert.Photometria siveDemensura et gradibus luminis, colo- rum et umbrae. Sumptibus viduae Eberhardi Klett, typis Christophori Petri Detleffsen, 1760. [2] Robert J. Woodham. Photometric method for determining surface orientation from multiple images.OpticalEngineering, 19(1):191139– 191139–, 1980. [3] Ren Ng, Marc Levoy, Mathieu Bre´dif, Gene Duval, Mark Horowitz, and Pat Hanrahan. Light Field Photography with a Hand-Held Plenoptic Camera. Technical report, April 2005. 156