Seite - 137 - in Proceedings of the OAGM&ARW Joint Workshop - Vision, Automation and Robotics

displacement values, minimisation of the MSE measure is an obvious way to estimate also unknown displacements. For the following, let us consider two images u and g, which are sampled representations of continuous-scale images. To specify the sampling process more precise, we assume that each pixel of g is the integral of the underlying continuous-scale image G over a rectangled region such that all pixels together tesselate (a rectangle of) the image plane: gi,j= ∫ i+1 i ∫ j+1 j G(x,y)dydx , (6) and similarly for u whose grid is of equal resolution but shifted by d=(α,β)∈R2, ui,j= ∫ i+1+α i+α ∫ j+1+β j+β U(x,y)dydx . (7) Without loss of generality, we assume 0≤α,β<1. Whereas in the special case of band-limited images sam- pled with at least their double limiting frequency, Shannon’s sampling theorem guarantees that u and g contain full informationon their continuouscounterparts, this canusually not be expected to hold true for natural images; thus the continuous images U and G are in fact unknown. A good measure for the discrepancy between u and g should essentially measure the discrepancy between their continuous versions U and G. In other words, we do not want to punish reconstructions for badly aligned grids, and formulate therefore an “innocence assumption” (in dubio pro reo – in case of doubt for the defendant): Whatever discrepancy between two images can plausibly be attributed to different sampling, shall not enter the discrepancy mea- sure. In particular, if a sufficiently plausible continuous- scale image V≡U≡G exists from which both u and g can be obtained by sampling, their discrepancy should be measured as zero. Notice that the exact meaning of the word “plausible” remains to be specified later. Our considerations can be boiled down to a discrete image v of size (2n+1)×(2m+1) whose pixels are the intersections of pixels of u and g: vi,j= ∫ ξi+1 ξi ∫ ηj+1 ηj V(x,y)dydx , (8) i=0,...,2n, j=0,...,2m, where ξi = i/2 for even i and ξi= i/2+α for odd i,ηj= j/2 for even j andηj= j/2+β for odd j. Note that pixel (i, j) of g covers the four pixels (2i,2j), (2i,2j+1), (2i+1,2j) and (2i+1,2j+1) of v whereaspixel (i, j)ofucovers the fourpixels (2i+1,2j+1), (2i+1,2j+2), (2i+2,2j+1) and (2i+2,2j+2) of v. The image v is therefore a superresolution image [15] of g and u, albeit with pixels of different sizes. In x direction grid cells of size α alternate with such of size 1−α, whereas in y direction the same is true with β and 1−β. In the general situation when U and G cannot be chosen as equal, we want to retain this idea and construct a super- resolution image v that tries to reconciliate the information of u and g as good as possible. The discrepancy of u and g will then be measured by combining discrepancies between u and v, and between v and g. In the perfectly aligned case,α=β=0, the MSE (2) of images g and u can be combined from the MSEs between each of g and u and their average v := 12(g+u) via MSE(u,g)=2(MSE(u,v)+MSE(v,g)) . (9) Moreover, using the parallelogram identity (or by an easy combination of Cauchy-Schwarz’ inequality with the arithmetic-geometric mean inequality) we see that for any other image v the right-hand side of (9) will be greater than MSE(u,g). This motivates the following definition. Definition. Let images u and g of size n×m sampled as in (6), (7) be given. Let a class X of (2n+1)×(2m+1)- images v sampled as in (8) be given. For each image v∈X, define vu, vg as the downsamplings of v to the grids of u and g, respectively. The alignment-MSE MSEX between u and g with respect to X is defined as MSEX(u,g)=min v∈X 2(MSE(u,vu)+MSE(vg,g)) . (10) Application of this definition requires, first, the specifi- cation of the class X for given images u, g. The class X essentiallydefineswhat areplausible superresolution images. Second, a minimisation method will be needed to find the minimiser. We will turn to these issues in the next section. IV. SPECIFYING CONSTRAINTS Given u and g, a superresolution image v as specified in the previous section must satisfy the equations αβv2i,2j+αβ¯v2i,2j+1 +α¯βv2i+1,2j+α¯β¯v2i+1,2j+1=gi,j , (11) α¯β¯v2i+1,2j+1+α¯βv2i+1,2j+2 +αβ¯v2i+2,2j+1+αβv2i+2,2j+2=ui,j (12) for i = 1,...,n, j = 1,...,m, where we have used the abbreviations α¯ :=1−α, β¯ :=1−β. On one hand, these are just 2nm equations for 4nm+ 2n+2m+1 pixels of v (from which two corner pixels could be eliminated as they are neither covered by g nor by u); additional conditions will therefore be necessary to remove this underdetermination. On the other hand, for images u and g that do not match perfectly, we expect that the equations should be satisfied only approximately. which favours smoothness. Thus, we are led to reformulate our equation system into the minimisation of an energy function E(v)=Sg(v)+Su(v) (13) under suitable constraints, where Sg and Su are quadratic 137