As autonomous vehicles continue to develop, verifying their safety remains a large hurdle to mass adoption. One component of this is testing, however it has been shown that it is impractical to statistically prove an autonomous vehicle’s safety using real-world testing alone. Therefore, simulation tools and other virtual testing methods are being employed to assist with the verification process. Testing in simulation still faces some of the challenges of the real world, such as the difficulty in exhaustively testing the system in all scenarios it will encounter. Manual scenario creation is time consuming and does not guarantee scenario coverage. Pseudo-random scenario generation is a faster option, but still does not ensure coverage of the state space. Therefore, this study proposes the use of Halton sequences to automatically generate scenarios for autonomous vehicle testing in simulation. It compares these scenarios against a set of pseudo-randomly generated scenarios and assesses the performance of each method to cover the simulation state space and provide an accurate depiction of the capabilities of the system-under-test. These tests are carried out in the CARLA simulation environment on an open source, published driving model called “Learning by Cheating” which takes place as the system-under-test. This study concludes that the scenario set generated by the Halton sequence is better at providing an accurate representation of the capabilities of the system-under-test than the pseudo-random scenario generation method.
","abstract_html":"<p>As autonomous vehicles continue to develop, verifying their safety remains a large hurdle to mass adoption. One component of this is testing, however it has been shown that it is impractical to statistically prove an autonomous vehicle’s safety using real-world testing alone. Therefore, simulation tools and other virtual testing methods are being employed to assist with the verification process. Testing in simulation still faces some of the challenges of the real world, such as the difficulty in exhaustively testing the system in all scenarios it will encounter. Manual scenario creation is time consuming and does not guarantee scenario coverage. Pseudo-random scenario generation is a faster option, but still does not ensure coverage of the state space. Therefore, this study proposes the use of Halton sequences to automatically generate scenarios for autonomous vehicle testing in simulation. It compares these scenarios against a set of pseudo-randomly generated scenarios and assesses the performance of each method to cover the simulation state space and provide an accurate depiction of the capabilities of the system-under-test. These tests are carried out in the CARLA simulation environment on an open source, published driving model called “Learning by Cheating” which takes place as the system-under-test. This study concludes that the scenario set generated by the Halton sequence is better at providing an accurate representation of the capabilities of the system-under-test than the pseudo-random scenario generation method.</p>","abstract_has_math":false,"creators":["Ferree, Andrew"],"institution":null,"degree_name":"Master of Science in Mechanical Engineering","degree_level":"Thesis - Open Access","degree_discipline":"Mechanical Engineering","degree_department":null,"school":null,"contributors":[],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2021,"date_issued":"2021-04-10T07:00:00Z","date_published":"2021-04-10T07:00:00Z","updated_at":"2026-07-27T19:25:16Z","subjects":["Autonomous Vehicles","Simulation","Testing","Verification","Scenarios","Halton Sequences","Automotive Engineering","Mechanical Engineering"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://commons.erau.edu/edt/591","outbound_label":"Repository record","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:creator","label":"Author","values":["Ferree, Andrew"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.available","label":"Dc Date Available","values":["2022-05-01T07:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Mechanical Engineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis - Open Access"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Master of Science in Mechanical Engineering"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Autonomous Vehicles","Simulation","Testing","Verification","Scenarios","Halton Sequences","Automotive Engineering","Mechanical Engineering"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://commons.erau.edu/edt/591"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["As autonomous vehicles continue to develop, verifying their safety remains a large hurdle to mass adoption. One component of this is testing, however it has been shown that it is impractical to statistically prove an autonomous vehicle’s safety using real-world testing alone. Therefore, simulation tools and other virtual testing methods are being employed to assist with the verification process. Testing in simulation still faces some of the challenges of the real world, such as the difficulty in exhaustively testing the system in all scenarios it will encounter. Manual scenario creation is time consuming and does not guarantee scenario coverage. Pseudo-random scenario generation is a faster option, but still does not ensure coverage of the state space. Therefore, this study proposes the use of Halton sequences to automatically generate scenarios for autonomous vehicle testing in simulation. It compares these scenarios against a set of pseudo-randomly generated scenarios and assesses the performance of each method to cover the simulation state space and provide an accurate depiction of the capabilities of the system-under-test. These tests are carried out in the CARLA simulation environment on an open source, published driving model called “Learning by Cheating” which takes place as the system-under-test. This study concludes that the scenario set generated by the Halton sequence is better at providing an accurate representation of the capabilities of the system-under-test than the pseudo-random scenario generation method.
"]},{"key":"dc:title","label":"Title","values":["Automated Scenario Generation Using Halton Sequences for the Verification of Autonomous Vehicle Behavior in Simulation"]}]}],"canonical_facts":{"dc:creator":["Ferree, Andrew"],"dc:date.available":["2022-05-01T07:00:00Z"],"dc:description.abstract":["As autonomous vehicles continue to develop, verifying their safety remains a large hurdle to mass adoption. One component of this is testing, however it has been shown that it is impractical to statistically prove an autonomous vehicle’s safety using real-world testing alone. Therefore, simulation tools and other virtual testing methods are being employed to assist with the verification process. Testing in simulation still faces some of the challenges of the real world, such as the difficulty in exhaustively testing the system in all scenarios it will encounter. Manual scenario creation is time consuming and does not guarantee scenario coverage. Pseudo-random scenario generation is a faster option, but still does not ensure coverage of the state space. Therefore, this study proposes the use of Halton sequences to automatically generate scenarios for autonomous vehicle testing in simulation. It compares these scenarios against a set of pseudo-randomly generated scenarios and assesses the performance of each method to cover the simulation state space and provide an accurate depiction of the capabilities of the system-under-test. These tests are carried out in the CARLA simulation environment on an open source, published driving model called “Learning by Cheating” which takes place as the system-under-test. This study concludes that the scenario set generated by the Halton sequence is better at providing an accurate representation of the capabilities of the system-under-test than the pseudo-random scenario generation method.
"],"dc:identifier":["https://commons.erau.edu/edt/591"],"dc:subject":["Autonomous Vehicles","Simulation","Testing","Verification","Scenarios","Halton Sequences","Automotive Engineering","Mechanical Engineering"],"dc:title":["Automated Scenario Generation Using Halton Sequences for the Verification of Autonomous Vehicle Behavior in Simulation"],"thesis:degree_discipline":["Mechanical Engineering"],"thesis:degree_level":["Thesis - Open Access"],"thesis:degree_name":["Master of Science in Mechanical Engineering"]},"updated_at":"2026-07-27T19:25:16Z"}