Mobile Multi-Bounce LiDAR

Single-photon avalanche diodes (SPADs) are emerging sensors that can measure the propagation of light in a scene, capturing higher-order reflections, shadows, and light transport that ordinary cameras are unable to. Measurement of these multi-bounce light paths is especially useful for non-line-of-sight (NLOS) imaging. The increasing availability of SPAD sensors on mobile devices (e.g. iPhone Pro LiDAR) raises the potential to enable NLOS capabilities on consumer devices in the future. Currently, these sensors are primarily employed for LiDAR-based depth estimation, with untapped potential in other applications. In light of recent advances in SPAD device development, the timing is opportune to revisit the applicability of multi-bounce LiDAR techniques on consumer-grade mobile devices. This thesis extends the applicability of multi-bounce LiDAR techniques from research-grade SPAD hardware to consumer-grade mobile LiDARs. First, we enable single-shot capture of two-bounce signals and remove the need for laser scanning by developing a tomographic formulation for two-bounce non-line-of-sight imaging. Second, we enable real-time non-line-of-sight capture at eye-safe laser power under object and camera motion. Our approach is inspired by principles from burst photography. We implement and evaluate the proposed algorithms in simulations and on experimental SPAD hardware. We also demonstrate real-time non-line-of-sight tracking on a consumer-grade smartphone LiDAR. Potential future applications of our results include "X-ray vision" in AR/VR, full-body tracking for AR headsets, room scanning for hard-to-reach areas, collision avoidance for autonomous vehicles, and robotic navigation.

Name thesis:degree_name

Master

Department dc:contributor.department

Program in Media Arts and Sciences (Massachusetts Institute of Technology)

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2024

Author dc:creator

• Somasundaram, Siddharth

Advisor dc:contributor.advisor

• Raskar, Ramesh

Statement dc:rights

• In Copyright - Educational Use Permitted
• Copyright retained by author(s)

Licence dc:rights.uri

• https://rightsstatements.org/page/InC-EDU/1.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/159263

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/159263