Design and evaluation of extended reality systems

As we enter the era of domain-specific architectures, systems researchers must understand the requirements of emerging application domains. Augmented and virtual reality (AR/VR) or extended reality (XR) is one such important domain that can propel new directions in architecture and systems research, as well as have a profound impact on all aspects of human endeavours. Thus far, conducting end-to-end systems research in XR, and consequently, developing principled design methodologies, has remained a challenge due to a lack of opensource benchmarks and systems. To address this challenge, this thesis develops a novel open-source XR research testbed, proposes several design principles for optimizing the power consumption of XR systems, and demonstrates the efficacy of these design principles in optimizing key XR components. First, we present ILLIXR, Illinois Extended Reality Testbed, the first fully open-source end-to-end XR system with state-of-the-art components, integrated with a modular and extensible multithreaded runtime, providing an OpenXR compliant interface to XR applications (e.g., game engines), and with the ability to report (and trade off) several quality of experience (QoE) metrics. Next, we analyze performance, power, and QoE metrics for the complete ILLIXR system and for its individual components, the first such analysis to be publicly available. Our analysis reveals several properties with implications for architecture and systems research. These include demanding performance, power, and QoE requirements, a large diversity of critical tasks, inter-dependent execution pipelines with challenges in scheduling and resource management, and a large tradeoff space between performance/power and human perception related QoE metrics. We then explore several broadly applicable design principles for optimizing the power consumption of XR systems. We demonstrate the use of distributed on-sensor computing for optimizing head tracking, which reduces VIO subsystem power by 1.3× compared to a centralized baseline. We employ another principle, online frequency control, to optimize scene reconstruction by reducing its frequency, resulting in a 4.9× reduction in power compared to a state-of-the-art baseline. We further reduce the power consumption of scene reconstruction by using scalable hardware mapping, which results in an overall power reduction of 128×–270× compared to an embedded GPU. The end-to-end nature of ILLIXR has enabled research in compute offload, new networking technologies, QoE-driven cross-component approximate computing, QoE-driven scheduling, and automatic accelerator detection. Overall, this thesis presents the research community with a unique testbed that enables architecture and systems research in XR in particular, and domain-specific systems in general. It presents a characterization of the testbed, providing researchers a starting point for conducting XR research. It also presents several generally applicable design principles for optimizing power, and demonstrates their use in optimizing head tracking and scene reconstruction.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2023

Author dc:creator

• Huzaifa, Muhammad

Contributors dc:contributor

• Adve, Sarita
• Chowdhry, Girish
• Fletcher, Christopher
• Hauser, Kris
• LaValle, Steve
• Alaghi, Armin
• Brooks, David

• Extended Reality
• Computer Architecture
• System Architecture

Statement dc:rights

• Copyright 2022 Muhammad Huzaifa

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/120192

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F120192