Christian Lessig

Local Fourier Slice Photography


Christian Lessig

To appear in ACM Transactions of Graphics (presented at Siggraph 2020).




Right: Refocused images obtained with local Fourier slice photography directly from a sparse, compressed wavelet representation of the light field (α is the ratio between the original and the refocused sensor plane distance from the lens). Left: Reconstruction error as a function of the nonzero coefficients (nzs) in the sparse representation, demonstrating that the image fidelity degrades gracefully as storage requirements are reduced. The plot also shows the output sensitivity of our technique, with the largest error obtained when the in-focus region in the image is largest.


Light field cameras provide intriguing possibilities, such as post-capture refocus or the ability to synthesize images from novel viewpoints. This comes, however, at the price of significant storage requirements. Compression techniques can be used to reduce these but refocusing and reconstruction require so far again a dense pixel representation. To avoid this, we introduce local Fourier slice photography that allows for refocused image reconstruction directly from a sparse wavelet representation of a light field, either to obtain an image or a compressed representation of it. The result is made possible by wavelets that respect the ``slicing's'' intrinsic structure and enable us to derive exact reconstruction filters for the refocused image in closed form. Image reconstruction then amounts to applying these filters to the light field's wavelet coefficients, and hence no reconstruction of a dense pixel representation is required. We demonstrate that this can reduce storage requirements and also computation times. We furthermore analyze the computational complexity of our algorithm and show that it scales linearly with the size of the reconstructed region and the non-negligible wavelet coefficients, i.e. with the visual complexity.


Resources

Preprint (low resolution images)

Reference implementation (Mathematica)

Refocusing video dragon scene

Refocusing video photographic light field

Additional videos

Additional experimental results


Acknowledgement

The comments by the anonymous reviewers helped to considerably improve the manuscript, in particular their insistence to try the algorithm on photographic light fields. The Stanford light field group is acknowledged for making the Lytro data sets available to the community. First ideas for the project were developed while the author was a post-doc in Marc Alexa's computer graphics group at TU Berlin. Many thanks also to Eugene Fiume for continuing support.