ScenarioXP: A Complete Scenario-Based Testing Framework for the Exploration and Exploitation of Autonomous Vehicle Validation Scenarios

<p>Today is an age of exciting emerging technology where cutting-edge research in autonomous vehicles (AVs) reduces the active human participation in driving and extends awareness beyond human limitations of perception and reaction, improving driving safety and quality of the user experience as a result. The ever-increasing complexity of these autonomous systems poses many challenges towards the validation and verification (V\&V) of these complex systems under time and resource constraints, as the use of artificial intelligence and also the intricacy of the operating environment means that these systems are also black-box and non-deterministic. Scenario-based V\&V testing of such systems, which involves constructing relevant and realistic scenarios to evaluate the performance of a complex system through observation, is a practical and effective means of V\&V testing under time and resource constraints. Furthermore, scenario-based V\&V testing has been the hallmark approach when testing the AV's predecessor, the automobile, as such it is the natural progression to perpetuate scenario-based V\&V testing for AVs. As such, this dissertation proposes an end-to-end scenario-based V\&V testing framework for AV V\&V testing, which intelligently and effectively identifies and proposes solutions to the challenges of testing AVs and other complex systems under time and resource constraints. The dissertation begins by identifying sources of AV V\&V scenarios, then constructing modular and measurable scenarios that are validated through scenario-based tests. Then, challenges in scenario-based testing of highly complex, high-dimensional scenarios are identified through modeling. This dissertation includes numerous studies of sequence sampling and exploration strategies to efficiently explore high-dimensional spaces, and to select high-value and edge scenarios. After the foundations of the proposed framework are validated, the dissertation diversifies: a first-of-it's kind AV scenario-description language for network-based traffic simulation is proposed, a practical means of estimating scenario complexity of AV V\&V scenarios is proposed, and interpretation of the scenario test data by leveraging explainable artificial intelligence is proposed. Finally, the framework is stress tested via highly complex, high-dimensional scenario tests to evaluate the scalability, interpretability, and performance of the dissertation research.</p>

Name thesis:degree_name

Doctor of Philosophy in Electrical Engineering & Computer Science

Level thesis:degree_level

Dissertation - Open Access

Discipline thesis:degree_discipline

Electrical Engineering and Computer Science

Year

2026

Author dc:creator

• Goss, Quentin

• Explainable Artificial Intelligence
• Information Theory
• Complexity Estimation
• Simulation
• Machine Learning
• High-Dimensional
• Scenario Selection
• Scenario Description Language
• Black-Box Testing
• Artificial Intelligence and Robotics
• Data Science
• Numerical Analysis and Scientific Computing
• Other Computer Sciences
• Programming Languages and Compilers
• Theory and Algorithms

Repository record dc:identifier

https://commons.erau.edu/edt/966

OAI identifier oai:identifier

oai:commons.erau.edu:edt-2007