Improve OpenMVG and create a novel algorithm for novel view synthesis from point clouds

This thesis presents work to improve open source 3D reconstruction software OpenMVG and to create a novel algorithm to render photorealistic images from new views given a photo collection and 3D point cloud. First, the original OpenMVG is parallelized using GPU and its data structure is optimized. Moreover, we integrated the MatchMiner algorithm into OpenMVG to further improve its efficiency. Last but not least, an initial pair selection formulation and a default focal length setting are introduced and implemented to automize OpenMVG. Then 3D sparse point clouds of construction sites are reconstructed by performing Structure-from-Motion (SfM) with the improved version of OpenMVG and source images (images that are used in SfM) are calibrated and registered to point clouds. Furukawa's Patch-based Multi-view Stereo(PMVS) algorithm is used to reconstruct dense point clouds using calibrated cameras as inputs. With known depth values of 3D points in the dense point cloud, we estimate depth maps of source images using optimization similar to Levin's colorization algorithm. For a novel view of the point cloud, we find source images that share some common elements of the construction site that are also visible to the novel view. Then we warp depth maps of these candidate images to the novel view. We estimate a depth map and label pixels for the novel view by solving a multi-label Markov Random Field (MRF) optimization problem using graph-cuts. We introduce a novel energy minimization formulation exploits both 2D and 3D information. Finally, a photorealistic image of the novel view is rendered by copying pixel colors from selected candidate source images using pixel labels computed with graph-cuts. We experimentally validate our approach on several challenging viewing angles of a point cloud model of a complicate construction site. The rendered results show high photo-realistic synthesis quality in planar scenes.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2016

Author dc:creator

• Tsoi, Ka Wai

Contributors dc:contributor

• Hoiem, Derek W.

• Computer Vision
• Computer Science
• Structure-from-Movtion
• Novel View Synthesis

Statement dc:rights

• 2016 Ka Wai Tsoi.

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/90821

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/90821