Advancing Mobile Immersive Computing: Systems and User Experience Perspectives

With recent advances in extended reality (XR)—an umbrella term for virtual reality (VR), augmented reality (AR), and mixed reality (MR)—and its growing integration into our daily lives, the study of mobile immersive computing has become increasingly vital. This rapidly evolving field blurs the boundary between the digital and physical worlds, offering interactive user experiences and becoming integral to domains such as education, medicine, and professional training. However, significant challenges arise, particularly in managing high computational demands and bandwidth consumption required for delivering high-quality content to ensure seamless user experiences. Additionally, addressing privacy concerns related to sensitive data collection and processing is crucial for safeguarding user trust and promoting widespread adoption. In this dissertation, I explore two key enabling techniques of mobile immersive computing from systems and user experiences perspectives: volumetric video streaming, which involves delivering 3D video content to resource-constrained mobile devices such as MR headsets, and image-based six degrees of freedom (6DoF) pose estimation, which determines the position and orientation of a device in 3D space based on captured camera views. First, I propose a foveated volumetric content delivery system Theia, leveraging the human visual system's characteristics to significantly reduce bandwidth consumption and on-device computation overhead. Theia prioritizes high-resolution streaming for the foveal region while reducing detail in the periphery, thereby optimizing resource usage without compromising user experience. Second, I introduce a next-generation volumetric video streaming system NeVo to enhance user experience. NeVo delivers high-quality video streaming by leveraging neural content representation, providing users with a more immersive and visually appealing experience. Lastly, I design a practical system PIPE for privacy-preserving, image-based 6DoF pose estimation. PIPE ensures accurate device localization in immersive applications while safeguarding users' privacy by judiciously reducing the transmission of sensitive data.

Author

• Wu, Nan

• Foveated Streaming
• Image-based Localization
• Neural Radiance Fields
• Privacy Leakage
• Volumetric Video Streaming

Identifier

hdl:1920/14382

OAI identifier oai:identifier

oai:MARS:1920/14382