Exploring Bidirectional Interfaces and Precise Input with Personal Devices in Extended Reality

The integration of smartphones with Extended Reality (XR) environments presents significant opportunities to enhance user interactions by leveraging the strengths of both mobile devices and Head-Mounted Displays (HMDs). This thesis addresses critical challenges in cross-device computing, focusing on developing comprehensive frameworks and innovative interaction techniques that facilitate seamless and precise interactions within XR settings. We propose a unified design space that synthesizes fragmented research into a cohesive framework, supporting both phone-centric and HMD-centric tasks. Building on this foundation, we develop novel interaction methods that employ spatial magnification and motion damping to improve target selection precision in virtual environments. Additionally, we introduce dynamic calibration algorithms that significantly enhance touch accuracy by reducing alignment errors between virtual and physical hands. Furthermore, we explore the impact of various interaction modalities and UI representations on user performance and satisfaction, demonstrating the effective adaptation of established 2D interaction techniques to 3D spatial contexts. Through user studies and prototype evaluations, our research validates the effectiveness of these approaches in enhancing usability, precision, and overall user experience. The findings highlight the potential of integrating smartphones with XR technologies to create more intuitive, efficient, and immersive interaction paradigms. This work not only addresses existing limitations in cross-device interactions but also lays the groundwork for future advancements in human-computer interaction within XR environments, paving the way for more sophisticated and user-centric applications.

Department dc:contributor.department

Computer Science

Year dc:date.issued

2025

Author dc:creator

• Zhu, Fengyuan

Advisor dc:contributor.advisor

• Grossman, Tovi

Statement dc:rights

• Attribution 4.0 International

Licence dc:rights.uri

• http://creativecommons.org/licenses/by/4.0/

Handle dc:identifier.uri

https://hdl.handle.net/1807/144903

OAI identifier oai:identifier

oai:utoronto.scholaris.ca:1807/144903