Page - 129 - in Proceedings of the OAGM&ARW Joint Workshop - Vision, Automation and Robotics

Fig. 1. Workflow of the processing pipeline for point cloud fusion. As mentioned before, Rothermel etal. [15] fuse depth maps in terms of oriented 3D point cloud generation. They intro- duce a local median-based fusion scheme which is robust to outliers and produces surfaces comparable to the results of the Middlebury MVS. Similar to Fuhrmann and Goesele [3] points are subsampled using a multi-level octree. Favoring points with the smallest pixel footprint, an initial point set is created utilizing nearest neighbor queries optimized for cylindrical neighborhoods, points are then iteratively filtered along line of sight or surface normals. The capability of the fusion strategy for large scale city reconstruction and the straight forward manner for implementation make it particularly interesting for this work. In our work we adopt the concept of the fusion strategy using a weighted median approach favoring high quality disparities assessed by a total variation based classification. III. METHODOLOGY The proposed framework builds upon the Remote Sensing SoftwareGraz (RSG)1.Thephotogrammetricprocessing (i.e. image registration, stereo matching) leads to different inter- mediate results which are utilized in the processing pipeline (seeFig.1).Disparitymapsarederived fromasetofepipolar rectified images using a matching algorithm based on SGM [6]. Forward and backward matching are employed to derive two point clouds via spatial point intersection per stereo pair whose coordinates are stored in East-North-Height (ENH) rasterfiles (i.e. a threebandrasterfileholding thecoordinates in geometry of the disparity map). The advantage of this approach is that coordinates can be accessed directly while preserving the spatial organization, i.e. the structure, of the point cloud. In the next step, surface normals and weights are computed and stored into a compressed LAS file (i.e. a lossless com- pressed data format for point cloud data) [7]. Subsequently, the point clouds are assigned to tiles in order to enable a tile- wise fusion of the data. Fig. 1 depicts the complete workflow of the presented processing pipeline. 1http://www.remotesensing.at/en/ remote-sensing-software.html A. Oriented Point CloudGeneration While in Rothermel et al. [15] normals are derived based on a restricted quadtree triangulation [13], we estimate surface normals in a least squares manner. A moving window operation is applied on the ENH raster files. Normals are derived by locally fitting a plane to the extracted point neigh- borhood. The normal estimation fails in areas with less than three reconstructed disparities. By introducing a threshold defining a minimum number of successfully reconstructed points, we are able to control the robustness of the normal calculation. Inourexperimentsweset thepixelneighborhood to 5 pixels and used a threshold of 3 points for all datasets. B. Disparity Quality Assessment The quality of disparities is affected by many factors like variation of texture strength and surface slant. We assess the quality of disparities in order to derive weights for every single observed point. These weights are later used in the fusion procedure using a weighted-median approach. Kuhn et al. [12] introduced a TV-L2 based classification of the disparities uncertainty. In contrast to many TV-L1 based MVS methods, the L2 norm takes noise and outliers into consideration which is required to measure the quality of the disparities. The TV is calculated over square windows with increasing radiusm resulting in n ∈ [1,20]⊂ N discrete classes.Starting fromaneighborhoodcontaining8connected pixels at a radius ofm=1 it increases by the factor of 8m. The discretization is achieved by introducing a regularization term τ which limits the TV to stay below a certain value. These TV classes describe the degree of the local oscillation of the disparities. The outlier probability can be obtained by learning error distributions from this classification using ground truth disparities. In our case we evaluate the quality of the disparities based on the work of Kuhn etal. [12] using a regularization term of τ=2. Due to the lack of ground truth disparities, we are not able to learn error distributions directly. Therefore, we analyze the quality of the classified disparities in 3D space. Reference data from Terrestrial Laser Scanners (TLS) is used to assess the quality of the raw dense point cloud for every single TV class independently. According to Cavegn et al. [2], vertical Digital Surface Models (DSM) are computed for 129