Inverse Rendering

In computer graphics, we pride ourselves on performant and accurate forward simulation of light (rendering). Here, we use graphics as methodology to solve problems beyond the generation of pretty pictures. By marrying extremely efficient forward renderers to numerical optimization methods, we develop novel analysis-by-synthesis approaches that push the boundaries on challenging problems like 3D surface reconstruction, real-time tracking of occluded objects from indirect diffuse light reflections, and light field imaging through uncalibrated free-form optics.

Publications

Non-Line-of-Sight Reconstruction using Efficient Transient Rendering

Julian Iseringhausen, Matthias B. Hullin
ACM Transactions on Graphics 39 (1), 2020. Non-Line-of-Sight Reconstruction using Efficient Transient Rendering

In this paper, we present an efficient renderer for three-bounce indirect transient light transport, and use it to reconstruct objects around corners to unprecedented accuracy.

Deep Non-Line-of-Sight Reconstruction

Javier Grau Chopite, Matthias B. Hullin, Michael Wand, Julian Iseringhausen
Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020. Deep Non-Line-of-Sight Reconstruction

The first deep-learning framework for reconstructing object shapes around a corner.

4D Imaging through Spray-On Optics

Julian Iseringhausen, Bastian Goldlücke, Nina Pesheva, Stanimir Iliev, Alexander Wender, Martin Fuchs, Matthias B. Hullin
ACM Transactions on Graphics 36(4) (Proc. SIGGRAPH), 35:1--35:11, 2017. 4D Imaging through Spray-On Optics

Raindrops on a window heavily distort the view of the scene. We show that a fully calibrated 4D light field can be recovered from a single photograph taken under such adverse conditions.

Machine Learning Assisted Identification of People Hidden Behind a Corner

Piergiorgio Caramazza, Alessandro Boccolini, Gabriella Musarra, Matthias Hullin, Roderick Murray-Smith, Daniele Faccio
Computational Optical Sensing and Imaging, 2017.

We demonstrate the use of machine learning to classify temporal histograms of the light-echoes backscattered from bodies hidden from view around a corner, captured by a SPAD camera.

Tracking Objects Outside the Line of Sight using 2D Intensity Images

Jonathan Klein, Christoph Peters, Jaime Martín, Martin Laurenzis, Matthias B. Hullin
Scientific Reports (Nature Publishing Group), 6, 32491; doi: 10.1038/srep32491, 2016. Tracking Objects Outside the Line of Sight using 2D Intensity Images

We demonstrate the tracking of objects outside the line of sight from 3rd-order indirect diffuse reflections, captured using a regular laser pointer and a 2D camera.