3D spatial layout and geometric constraints for scene understanding

An image is nothing but a projection of the physical world around us, where objects do not occur randomly but follow certain spatial rules. Many existing computer vision approaches tend to ignore this aspect of understanding images. In this work, we build representations and propose strategies for exploiting such constraints towards extracting a 3D understanding of a scene from its single image. We model a scene in terms of its spatial layout abstracted as a box, object cuboids, camera viewpoint, and interactions between them. We take a supervised approach towards estimation, and learn models from training data that is fully annotated with the 3D spatial extent of objects, walls, and floor. We assume the world is populated with axis aligned objects and surfaces, and exploit constrained appearance models which use geometric cues from the scene. Our methods are tailored towards indoor scenes that are highly structured and require careful spatial reasoning. We show that our box layout representation is able to capture the full spatial extent of a 3D scene, which we can successfully estimate even for heavily cluttered rooms. Similarly, by exploiting the geometric constraints offered by the scene, we can approximate the extent of the objects as cuboids in 3D. The box layout provides rich contextual information for detecting objects. We show that modeling the 3D interactions between object cuboids and scene layout improves object detection. Finally, we show how to use our 3D spatial layout models together with object cuboid models to predict the free space in the scene.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Electrical & Computer Engr

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2012

Author dc:creator

• Hedau, Varsha

Contributors dc:contributor

• Forsyth, David A.
• Hoiem, Derek W.
• Ma, Yi
• Huang, Thomas S.
• Hasegawa-Johnson, Mark A.

• Computer vision
• Image Processing
• Scene understanding
• Object recognition
• Single view 3D modeling
• Vanishing points

Statement dc:rights

• copyright 2011 Varsha Chandrashekhar Hedau

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/29773

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/29773