Seite - 138 - in Proceedings of the OAGM&ARW Joint Workshop - Vision, Automation and Robotics

error terms for the equations above, Sg(v) := n−1 ∑ i=0 m−1 ∑ j=0 ( gi,j−αβv2i,2j−αβ¯v2i,2j+1 −α¯βv2i+1,2j−α¯β¯v2i+1,2j+1 )2 , (14) Su(v) := n−1 ∑ i=0 m−1 ∑ j=0 ( ui,j−α¯β¯v2i+1,2j+1−α¯βv2i+1,2j+2 −αβ¯v2i+2,2j+1−αβv2i+2,2j+2 )2 . (15) Up to constant factors, Sg and Su are just the MSE(g,vg) and MSE(u,vu) from the alignment-MSE definition. Let us therefore now discuss possible constraints for this minimisation problem. These constraints will constitute the class X of images to minimise over that appeared in the definition of the alignment-MSE. Note first that in the equations (11), (12) for subsequent indices i or j the two input images u and g alternate. This suggests that for images u and g that do not perfectly match, solutions of (11), (12) are likely to develop oscillating patterns like stripes of alternating intensity or checkerboard structures, so the discrepancy between u and g can be translated to the image boundary where the first and last row and column of v are linked only to one of the input images and therefore provide degrees of freedom that can absorb the discrepancy. In extreme, this could mean that even for completely mismatching u and g highly oscillatory images v might exist that fulfil (11), (12) without any error. Such solutions should be rejected by a suitable class X. In order to prevent v from developing strong high- frequency structures, a natural requirement could be that v should be essentially interpolating; thus each pixel intensity vi,j should be in the interval bounded by the intensities gbi/2c,bj/2c, ub(i−1)/2c,b(j−1)/2c of the two input pixels it is linked to by (11), (12). Whilst conceptually elegant and free of additional parameters, this constraint turns the minimi- sation of (13) into a quadratic minimisation problem on a highly nonconvex domain. We aim therefore at relaxing this constraint to a convex regularisation that warrants a unique solution as well as a practical minimisation procedure. We extend therefore the energy function (13) to E(v)=Sg(v)+Su(v)+γT(∇v) (16) where T is a regulariser that depends on the derivatives∇v= (vx,vy) of v approximated by finite differences, and γ>0 is a regularisation weight. With regard to the quadratic nature of the mean square error to be measured, a Whittaker-Tikhonov regularisation T(∇v) :=∑ i,j |∇v|2 (17) lends itself as a natural candidate, which yields a convex quadratic minimisation problem, also removing completely the non-uniqueness of the original equations. Minimisers can efficiently be computed using standard iterative solution methods for the linear system of minimality conditions. a b Fig. 3. (a) Superresolution image created in aligning the images from Fig. 2(a) and (d) with Whittaker-Tikhonov regularisation, γ = 0.003. Alignment-MSE measurement with this superresolution image yields a PSNR of 46.05dB. – (b) Same with TV regularisation, γ=0.03, yielding a PSNR of 29.92dB. A further candidate is total variation T(∇v) :=∑ i,j |∇v| . (18) To find minimisers with this regularisation, one can use, e.g., a gradient descent approach where the regularisation is realised via a locally analytic scheme related to single- scale Haar wavelet shrinkage; we use here a variant of the scheme from [19] adapted to the unequal pixel sizes of v. As a general rule, in order to just remove the underdeter- minedness of the equation system (11), (12), it is desirable to keep the regularisation weight γ rather small. V. EXPERIMENTS Weevaluate the regularisedsuperresolutionalignmentpro- cedure from the preceding two sections by the test case from Section II. Starting with Whittaker-Tikhonov regularisation, we observe that for large regularisation weight such as γ=0.3 fairly precise estimates for the displacement can be obtained. However, the superresolution images in this case are severely blurred, leading to overestimation of alignment- MSE and low PSNR. For example, the resulting PSNR for the images from Fig. 2(a) and (d) is 28.61dB. On the other hand, reducing the regularisation parameter to γ= 0.003 yields extremely high PSNR estimates, e.g. 46.05dB for the same two images. The reason is that the superresolution images are far away from interpolating between u and g, showing unnatural oscillations, see Fig. 3(a). In contrast, TV regularisation yields plausible results over a wide range of regularisation parameters, see the exemplary superresolution image in Fig. 3(b). For a more detailed evaluation we focus therefore on TV regularisation. We measure first reconstruction errors for the known exact displacements, see column (a) of Table III. Next we estimate the displacements using the TV-regularised error measure itself, see column (b). Once more the minimisation is done by a grid search with x and y displacements varying from−1 to+1 in 0.01 steps. The TV regularisation weight γ is set to 0.03. As the application of the superresolution alignment in this brute-force minimisation is computationally expensive, we add a third scenario, column (c), in which a faster 138