Homo prospectus to robo prospectus : towards predictive algorithms for advanced autonomous vehicle perception

Intention and trajectory prediction are key factors in the autonomy stack of self-driving vehicles. They help to avoid potential collisions and plan safe paths.This thesis presents a comprehensive investigation into the enhancement of intention and trajectory prediction in Autonomous Vehicles (AV) through a multi-faceted approach involving enhancing all the autonomy stack's blocks that proceed Prediction, namely, data, detection, and prediction. Firstly, we introduced ROAD-Waymo, an extensive dataset that significantly augments the (US) Waymo Open dataset with a wealth of annotations, enabling the development and benchmarking of algorithms for agent, action, location, and event detection in road scenes. This dataset addresses the need for holistic scene understanding beyond mere object detection, providing a robust platform for domain adaptation and advanced perception in AV systems, aiming to provide high-level understanding to models trained on it. Secondly, we improved object detections under challenging conditions. Addressing the challenge of detection in adverse weather, this thesis demonstrates how synthetic datasets generated using CycleGAN architectures can drastically improve the performance of state-of-the-art detectors in challenging conditions such as night-time and rainy environments. This approach not only enhances detection capabilities but also circumvents the difficulties in collecting extensive real-world adverse condition datasets. Further, the thesis delves into the detection of small objects, a critical aspect of AV perception, especially in high-speed scenarios like autonomous racing. By modifying the popular YOLOv5 object detector, the research introduces the `YOLO-Z' series, significantly improving the detection of smaller objects without substantially increasing inference time. This advancement is pivotal for enhancing contextual awareness in AV systems. Thirdly, we improved the prediction itself by introducing ASTRA (A Scene-aware TRAnsformer-based model), a novel trajectory prediction methodology that models spatial, temporal, and social dimensions simultaneously for precise forecasting. ASTRA's graph-aware transformer architecture marks a significant improvement in trajectory prediction accuracy and efficiency from two different perspectives, demonstrating its suitability for edge computing in AV systems. Finally, the thesis presents Temporal-DINO, a self-supervised video strategy inspired by DINO (self-distillation with no labels), which significantly enhances action prediction in Autonomous Vehicles. This approach demonstrates notable improvements in various architectures, emphasizing its effectiveness in capturing long-term dependencies crucial for AV applications.

Grantor dc:publisher

Oxford Brookes University

Author dc:creator

• Teeti, Izzeddin

Contributors dc:contributor

• Cuzzolin, Fabio
• Bradley, Andrew

Statement dc:rights

• All rights reserved

Language dc:language

en

DOI dc:identifier

https://doi.org/10.24384/zst3-5951

OAI identifier oai:identifier

tle:06fcc3b3-bea4-4c5a-adb6-6f6a1eade520:d6bd9758-527a-46cd-bfe2-c433766e8fca:1