Scenario Extraction from Egocentric Automotive Datasets

Automated vehicles (AVs) must safely navigate complex and open environments, however, the long tail of edge case scenarios remains a challenge for large scale AV deployment. Scenario-based coverage analysis aims to identify and reduce the space of unknown and unsafe scenarios and is a critical step in the development, validation, and operation of machine learning-enabled automated driving systems (ADS). Enabling scenario-based coverage analysis of large datasets requires temporally and spatially extracting critical scenarios and variations of them. Many of the existing scenario extraction methods are limited to extraction over dataset annotations, rendering them unable to automatically extract the rich semantics that are best inferred by vision, such as weather and road conditions. Additionally, many scenario extraction methods specify scenarios at a lower level of abstraction than that at which scenarios are naturally reasoned about. In this thesis, we present a novel approach to scenario extraction from egocentric datasets. We represent a driving scenario by a sequence of scene graphs, which are a structured semantic representation of a scene. Next, we formally define driving scenarios of interest using Linear Temporal Logic (LTL), then extract all instances of the scenarios from the dataset using an off-the-shelf model checker. We evaluate the approach on the training and validation splits of Argoverse 2, consisting of 850 15-second real-world driving scenarios. We demonstrate the effectiveness of our approach by evaluating against a rule-based benchmark based on dataset annotations, then demonstrate the automatic extraction of scenarios other methods struggle with.

Department dc:contributor.department

Computing and Software

Year dc:date.issued

2026

Author dc:creator

• Ramdhan, Stefan

Advisors dc:contributor.advisor

• Lawford, Mark
• Pantelic, Vera

Statement dc:rights

• Attribution 2.5 Canada

Licence dc:rights.uri

• http://creativecommons.org/licenses/by/2.5/ca/

Language dc:language.iso

en

Identifier URI

https://doi.org/10.71548/427

OAI record oai:identifier

https://macsphere.mcmaster.ca/server/oai/request?verb=GetRecord&metadataPrefix=dim&identifier=oai%3Amacsphere.mcmaster.ca%3A11375%2F33388