ILLIXR-A: an open source extended reality system for android platforms

There has been significant progress in virtual, augmented, and mixed reality (VR, AR, and MR) devices, collectively referred to as extended reality (XR). However, until recently, this do- main was dominated by proprietary software, which slowed down research. This motivated the development of an open-source end-to-end XR system called ILLIXR (Illinois Extended Reality testbed). It was developed to democratize XR systems research, development, and benchmark- ing. ILLIXR has been used to provide quantitative characterizations of performance, power, and quality-of-experience metrics for current XR systems and to develop solutions to address the gap between current and desirable XR systems. However, ILLIXR currently only supports the Linux platform, which limits its use in common consumer-grade devices that do not support Linux. The operating system used in most consumer devices, such as smartphones, smart TVs, tablets, and head-mounted displays (HMD), is Android. Android’s open-source nature allowed it to be highly customizable and adaptable for different purposes, contributing to its widespread usage. Additionally, there is little detailed analysis of XR runtimes on the Android platform. Hence, we present ILLIXR-A, a modular, open-source, and end-to-end XR system for the An- droid platform. ILLIXR-A leverages Monado to provide support for OpenXR-compatible applications, which is an emerging open interface between an XR application and runtime. It will enable researchers to utilize the full potential of ILLIXR on a wide variety of Android devices. ILLIXR-A provides full head-tracking capabilities using Visual-Inertial Odometry (VIO) by utilizing the camera and built-in sensors in an Android device. It also includes an IMU Integrator and a pose predictor, which predicts the user’s pose at a faster rate than VIO to compensate for the high latency of VIO. ILLIXR-A also performs Asynchronous Reprojection (Timewarp). Using ILLIXR-A, we evaluated an end-to-end XR system on an Android smartphone, and we present its performance characterization in this thesis. The experimental results highlight the need for efficient algorithms for low-power devices to meet the tight deadlines for a smooth XR experience. We also show how the quality of experience is affected by the complexity of an algorithm and provide future directions for research.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2023

Author dc:creator

• Bhowmik, Madhuparna

Contributors dc:contributor

• Adve, Sarita

• Extended Reality
• Android
• Illixr

Statement dc:rights

• Copyright 2023 Madhuparna Bhowmik

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/121543

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F121543