ENS-CVXNet: Convex decomposition of complex scenes

Generating accurate convex decompositions of indoor scenes from single RGB images is a challenging task with numerous applications in computer graphics. Current state-of-the-art methods employ encoder-decoder neural networks to convert RGB images into a fixed number of simple primitives (e.g., parallelepipeds). However, these approaches have limitations in capturing long-range dependencies and transferring global information, which can impact their accuracy and generalization capability across diverse indoor environments. To address these limitations, we propose ENS-CVXNet, an ensemble approach that leverages the strengths of various convex decomposition techniques and incorporates additional geometric information as summaries. Our core analysis explores well-established algorithms such as VHACD, COACD, and BSP-Net, as well as the integration of global features extracted by PointNet from the input point cloud. By combining multiple models trained with different summaries and configurations, ENS-CVXNet selects the best-performing model for each input image based on its ability to generate accurate depth predictions, evaluated against ground truth depth maps or predictions from state-of-the-art depth predictors. Through extensive experiments and evaluations on the NYUv2 dataset, we demonstrate that ENS-CVXNet outperforms the baseline method, achieving a 20% decrease in AbsRel from 0.093 to 0.0744, and improving the overall precision and quality of convex decomposition for indoor scenes. Our ensemble approach effectively combines the strengths of various techniques, leveraging geometric summaries and adapting to diverse scene characteristics, results in more accurate and robust 3D geometric reconstruction from single RGB images.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2024

Author dc:creator

• Jain, Seemandhar

Contributors dc:contributor

• Forsyth, David

• 3d Reconstruction
• 3d Vision
• Convex Decomposition
• Ray Tracing

Statement dc:rights

• Copyright 2024 Seemandhar Jain

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/124422

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F124422