Uni4D: Unifying visual foundation models for 4D modeling from a single video

This paper presents a unified approach to understanding dynamic scenes from casual videos. Large pretrained vision foundation models, such as vision-language, video depth prediction, motion tracking, and segmentation models, offer promising capabilities. However, training a single model for comprehensive 4D understanding remains challenging. We introduce Uni4D, a multi-stage optimization framework that harnesses multiple pretrained models to advance dynamic 3D modeling, including static/dynamic reconstruction, camera pose estimation, and dense 3D motion tracking. Our results show state-of-the-art performance in dynamic 4D modeling with superior visual quality. Notably, Uni4D requires no retraining or fine-tuning, highlighting the effectiveness of repurposing visual foundation models for 4D understanding.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois Urbana-Champaign

Year dc:date

2025

Author dc:creator

• Yao, David Yifan

Contributors dc:contributor

• Wang, Shenlong

• Computer Vision
• Machine Learning
• Structure from Motion
• 4D Reconstruction
• Dynamic Modeling
• Video Depth Estimation
• Pose Estimation

Statement dc:rights

• Copyright 2025 David Yifan Yao

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/129198

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F129198