Sim-on-wheels: Physical world in the loop simulation for self-driving

We present Sim-on-Wheels, a safe, realistic, and vehicle-in-loop framework to test autonomous vehicles’ performance in the real world under safety-critical scenarios. Sim-on-wheels runs on a self-driving vehicle operating in the physical world. It creates virtual traffic participants with risky behaviors and seamlessly inserts the virtual events into images perceived from the physical world in real-time. The manipulated images are fed into autonomy, allowing the self-driving vehicle to react to such virtual events. The full pipeline runs on the actual vehicle and interacts with the physical world, but the safety-critical events it sees are virtual. Sim-on-Wheels is safe, interactive, realistic, and easy to use. The experiments demonstrate the potential of Sim-on-Wheels to facilitate the process of testing autonomous driving in challenging real-world scenes with high fidelity and low risk. Additional results and open-sourced code are available on our project page here: https://sim-on-wheels.github.io/.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois Urbana-Champaign

Year dc:date

2025

Author dc:creator

• Chandaka, Bhargav

Contributors dc:contributor

• Wang, Shenlong

• Safety
• Robot sensing systems
• Real-time systems
• Autonomous agents
• Simulation
• Animation
• Autonomous vehicles

Statement dc:rights

• Copyright 2025 Bhargav Chandaka

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/129207

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F129207