Light in Flight

Transient imaging, or light-in-flight imaging, refers to the capture and analysis of light transport at nanosecond and picosecond scales. We develop devices and methods to enable the capture and processing of such data at low cost and unprecedented speed, and to robustly reconstruct 3D scene information within and even beyond the line of sight.

Publications

Super-Resolution Time-Resolved Imaging Using Computational Sensor Fusion

Clara Callenberg, Ashley Lyons, Dennis den Brok, Areeba Fatima, Alejandro Turpin, Vytautas Zickus, Laura M. Machesky, Jamie A. Whitelaw, Daniele Faccio, Matthias B. Hullin
Scientific Reports (Nature Publishing Group) 11, 1689 (2021), https://doi.org/10.1038/s41598-021-81159-x, 2021. Super-Resolution Time-Resolved Imaging Using Computational Sensor Fusion

Low-Cost SPAD Sensing for Non-Line-Of-Sight Tracking, Material Classification and Depth Imaging

Clara Callenberg, Zheng Shi, Felix Heide, Matthias B. Hullin
ACM Transactions on Graphics 40 (4), Article 61 (Proc. SIGGRAPH 2021), 2021. Low-Cost SPAD Sensing for Non-Line-Of-Sight Tracking, Material Classification and Depth Imaging

Non-Line-of-Sight Reconstruction using Efficient Transient Rendering

Julian Iseringhausen, Matthias B. Hullin
ACM Transactions on Graphics 39 (1), 2020. Non-Line-of-Sight Reconstruction using Efficient Transient Rendering

In this paper, we present an efficient renderer for three-bounce indirect transient light transport, and use it to reconstruct objects around corners to unprecedented accuracy.

Deep Non-Line-of-Sight Reconstruction

Javier Grau Chopite, Matthias B. Hullin, Michael Wand, Julian Iseringhausen
Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020. Deep Non-Line-of-Sight Reconstruction

The first deep-learning framework for reconstructing object shapes around a corner.

Neural network identification of people hidden from view with a single-pixel, single-photon detector

Piergiorgio Caramazza, Alessandro Boccolini, Daniel Buschek, Matthias Hullin, Catherine F. Higham, Robert Henderson, Roderick Murray-Smith, Daniele Faccio
Scientific Reports (Nature Publishing Group), 8, 11945; doi: 10.1038/s41598-018-30390-0, 2018. Neural network identification of people hidden from view with a single-pixel, single-photon detector

We demonstrate a machine learning approach that can locate and identify people from time-resolved single-pixel measurements.

Machine Learning Assisted Identification of People Hidden Behind a Corner

Piergiorgio Caramazza, Alessandro Boccolini, Gabriella Musarra, Matthias Hullin, Roderick Murray-Smith, Daniele Faccio
Computational Optical Sensing and Imaging, 2017.

We demonstrate the use of machine learning to classify temporal histograms of the light-echoes backscattered from bodies hidden from view around a corner, captured by a SPAD camera.

Material Classification using Raw Time-of-Flight Measurements

Shuochen Su, Felix Heide, Robin Swanson, Jonathan Klein, Clara Callenberg, Matthias B. Hullin, Wolfgang Heidrich
Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. Material Classification using Raw Time-of-Flight Measurements

We show that using multi-frequency time-of-flight measurements, five different white materials can be distinguished on a per-pixel basis.

Doppler Time-of-Flight Imaging

Felix Heide, Wolfgang Heidrich, Matthias B. Hullin, Gordon Wetzstein
ACM Transactions on Graphics (Proc. SIGGRAPH), 34 (4), 2015. Doppler Time-of-Flight Imaging

A new computational imaging system that captures metric radial velocity information per pixel -- think of a huge array of traffic speed guns that use light instead of radar.

Diffuse Mirrors: 3D Reconstruction from Diffuse Indirect Illumination using Inexpensive Time-of-Flight Sensors

Felix Heide, Lei Xiao, Wolfgang Heidrich and Matthias B. Hullin
Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014. Diffuse Mirrors: 3D Reconstruction from Diffuse Indirect Illumination using Inexpensive Time-of-Flight Sensors

How to look around a corner using echoes of light, using low-end devices that can't even properly measure such data.

Low-Budget Transient Imaging using Photonic Mixer Devices

Felix Heide*, Matthias B. Hullin*, James Gregson, Wolfgang Heidrich (* joint first authors)
ACM Transactions on Graphics (Proc. SIGGRAPH), 32 (4), 2013. Low-Budget Transient Imaging using Photonic Mixer Devices

A computational method for capturing videos of light in flight using consumer-grade imaging hardware.