Automated UI Exploration of VR Environments

Virtual Reality (VR) and Augmented Reality (AR), collectively referred to as Extended Reality (XR), are emerging technologies that have rapidly gained popularity in recent years, due to advancements in headset hardware and hand-tracking software. The spread of Extended Reality (XR), and of the so-called metaverse, is driven by the gaming industry, but these technologies are also increasingly adopted in education, healthcare, training, and industrial design. Being a relatively new technology, with standards, hardware, and capabilities continuously evolving, the ability to perform security testing and analysis of XR applications remains limited, despite the growing number of studies published in recent years. To address this gap, we propose AUTOMATRON, a novel automated testing framework driven by a scripting language designed for automatic interaction with and testing of Unity VR applications for Meta Quest 3, which is currently one of the most widely used headsets for virtual reality. Specifically, AUTOMATRON allows researchers to script application-agnostic sequences of actions at different levels of abstraction, supporting security and analysis tasks, from reproducing XR-based attacks to automating interactions with applications in order to evaluate HTTPS information exposure or uncover crashes and unexpected behaviors. A key advantage over existing tools is that AUTOMATRON allows researchers to write their own scripts, providing flexibility and extensibility for diverse testing needs. We tested the features of our language by checking their effectiveness on a set of applications collected from the official Meta Quest store. In addition, we recorded a few videos to demonstrate the effects of some AUTOMATRON scripts on selected applications.

Author dc:creator

• Marco Conti (348776)

• Automated UI exploration of Unity VR Applications for Meta Quest 3

Statement dc:rights

• In Copyright
• Open Access after 2028-05-01

DOI dc:identifier

https://doi.org/10.25417/uic.32995046.v1

OAI identifier oai:identifier

oai:figshare.com:article/32995046