Page - 135 - in Proceedings of the OAGM&ARW Joint Workshop - Vision, Automation and Robotics

a b Fig. 1. (a) Synthetically blurred image with ground-truth PSF, from [11]. – (b) Blind deconvolution result with PSF, from [12]. Note the opposite shifts of image and PSF. no alignment whatsoever is mentioned [6], [8], [9], [10]. Such an evaluation relies implicitly on the assumption that estimated PSFs are aligned with the ground truth PSF; probably this is approximately achieved by some test cases with small PSF support. Efforts to compensate shifts, either for images or for PSFs, are found in [7], [14], [20]. A bench- mark established in [7] is based on simulating camera shake by generated trajectories. Multiple ground truth images are acquired directly along those trajectories, and the best match is used for error measurement. On one hand, a computational alignment step is avoided in this way. On the other hand, the procedure constrains shifts to the ground-truth trajectory which may be insufficient since blind deconvolution methods can well yield translations in which the coordinate origin of the PSF does not happen to be on the (unknown) trajectory that was used to generate the ground truth. The benchmark from [7] is also used in [20] and part of the evaluation in [14]. Further tests in [14] are based on data from [9]. Here, absolute errors of PSFs are measured, namely for “(aligned) blur estimates” with respect to ground truth PSFs. This allows indeed to handle unconstrained displacements. Details of the alignment procedure are not given, however. In the following we discuss how to make precise such an alignment procedure. We focus on a scenario where a ground-truth image and PSF are available, and restoration quality is to be estimated by measuring the error between the ground truth and reconstructed images. In specifying the alignment procedure, some choices have to be made: first, should one register the reconstructed image to the ground truth image, or vice versa, or should perhaps both be transformed? Which interpolation procedure is to be used in the registration process? It is not a far-fetched guess that these details will influence the subsequent error measurements. In fact, we will demonstrate by a simple experiment in Section II that, dependent on details of the registration, the PSNR measures vary by 1.5dB and more. Given the fact that relative improvements of one blind deconvolution method over the other as reported in e.g. [7], [14] often amount to as little as 0.5dB or even less, such a difference is significant. This might be mitigated by using multiple test images and performing statistics on the errors measured for these. How- ever, questions remain: Since errors introduced by interpola- tion can be expected to differ substantially between test cases where the displacement is approximately integer, and test cases where the displacement is near a half-pixel position, results may be strongly biased towards blind deconvolution methods that, for whatever reason, tend to reconstruct PSFs in similar pixel alignment as the ground-truth. Given the complexity of procedures both for constructing apparently realistic test cases, and of the blind deconvolution procedures themselves, it is suchfavourablealignmentsoccurmoreoften for some methods under investigation than for others. In such a case, the bias won’t necessarily average out for larger sample sizes. For this reason, we pursue in this paper the goal to establish an alignment procedure for blind deconvolution results that avoids these pitfalls. We focus here on the MSE, from which (P)SNR can be derived via (4), (5). Structure of the paper. In Section II we evaluate the errors introduced by interpolation-based alignment proce- dures using a simple test case. Section III establishes the fundamentals of an alignment procedure by superresolution in order to avoid these errors. The details of the procedure are discussed in Section IV, followed by experiments on the previously introduced test case in Section V. A short summary and outlook in Section VI concludes the paper. II. ALIGNMENT BY INTERPOLATION To assess the errors introduced by alignment with inter- polation, we set up a simple test case based on a ground truth grey-value image shown in Fig. 2 (a). We blur this image by 16 different PSFs shown in Fig. 2 (b); all these PSFs are downsampled versions of the same high-resolution PSF with horizontal and vertical shifts in 1/4 pixel steps. One blurred image is shown in Fig. 2 (c). Each of the blurred images is deconvolved with each of the 16 PSFs using the non-blind deconvolution method from [18] with the same parameters (α=0.01, 300 iterations). This yields 256 deblurred images with effective shifts w.r.t. the ground truth images from−0.75 to+0.75pixels inxandydirection; one exemplary deblurred image is shown in Fig. 2 (d). We can now measure the MSE (and resulting PSNR) for each deblurred image w.r.t. the ground truth image. In the following we report PSNR values as this is the most familiar measure in deconvolution literature. To reduce the impact of boundary artifacts, a 20 pixel wide margin is excluded from the measurement, thus using a 88×88 central patch of the reference image. We notice first that in the 16 translation-free cases (where the same PSF was used for blurring and deblurring) the PSNR varies between 29.74 and 30.41dB, with an average of 30.07dB and a standard deviation of 0.21dB. Next, we measure PSNR values for the entire set of 256 deblurred images. Here, the ground-truth and reconstructed images are aligned using either bilinear and bicubic interpo- lation with the ground-truth shift values. For the direction of alignment we consider three settings: (a) warping the 135