Christian Lessig

Local Fourier Slice Photography


Christian Lessig

ACM Trans. Graph. 39, 3, Article 24 (April 2020), 16 pages; (presented at Siggraph 2020).




Light field cameras provide intriguing possibilities, such as post-capture refocus or the ability to synthesize images from novel viewpoints. This comes, however, at the price of significant storage requirements. Compression techniques can be used to reduce these but refocusing and reconstruction require so far again a dense pixel representation. To avoid this, we introduce local Fourier slice photography that allows for refocused image reconstruction directly from a sparse wavelet representation of a light field, either to obtain an image or a compressed representation of it. The result is made possible by wavelets that respect the ``slicing's'' intrinsic structure and enable us to derive exact reconstruction filters for the refocused image in closed form. Image reconstruction then amounts to applying these filters to the light field's wavelet coefficients, and hence no reconstruction of a dense pixel representation is required. We demonstrate that this can reduce storage requirements and also computation times. We furthermore analyze the computational complexity of our algorithm and show that it scales linearly with the size of the reconstructed region and the non-negligible wavelet coefficients, i.e. with the visual complexity.


Resources:

Siggraph talk (with videos, use Acrobat for viewing; version without videos)

Reference implementation (Mathematica)

Refocusing video dragon scene

Refocusing video photographic light field

Additional videos

Additional experimental results

Preprint (low resolution images)


Acknowledgement:

The comments by the anonymous reviewers helped to considerably improve the manuscript, in particular their insistence to try the algorithm on photographic light fields. The Stanford light field group is acknowledged for making the Lytro data sets available to the community. First ideas for the project were developed while the author was a post-doc in Marc Alexa's computer graphics group at TU Berlin. Many thanks also to Eugene Fiume for continuing support.