Visual analysis and synthesis with physically grounded constraints

"The past decade has witnessed remarkable progress in image-based, data-driven vision and graphics. However, existing approaches often treat the images as pure 2D signals and not as a 2D projection of the physical 3D world. As a result, a lot of training examples are required to cover sufficiently diverse appearances and inevitably suffer from limited generalization capability. In this thesis, I propose ""inference-by-composition"" approaches to overcome these limitations by modeling and interpreting visual signals in terms of physical surface, object, and scene. I show how we can incorporate physically grounded constraints such as scene-specific geometry in a non-parametric optimization framework for (1) revealing the missing parts of an image due to removal of a foreground or background element, (2) recovering high spatial frequency details that are not resolvable in low-resolution observations. I then extend the framework from 2D images to handle spatio-temporal visual data (videos). I demonstrate that we can convincingly fill spatio-temporal holes in a temporally coherent fashion by jointly reconstructing the appearance and motion. Compared to existing approaches, our technique can synthesize physically plausible contents even in challenging videos. For visual analysis, I apply stereo camera constraints for discovering multiple approximately linear structures in extremely noisy videos with an ecological application to bird migration monitoring at night. The resulting algorithms are simple and intuitive while achieving state-of-the-art performance without the need of training on an exhaustive set of visual examples."

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Electrical & Computer Engr

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2016

Author dc:creator

• Huang, Jia-Bin

Contributors dc:contributor

• Ahuja, Narendra
• Huang, Thomas S.
• Do, Minh N.
• Hasegawa-Johnson, Mark
• Hoiem, Derek

• Computer vision
• Visual synthesis
• patch-based optimization
• image completion
• image super-resolution
• video completion
• visual tracking

Statement dc:rights

• Copyright 2016 Jia-Bin Huang

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/92750

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/92750