Occlusion-aware tracking for drones using neural methods

This thesis addresses the challenge of gate tracking under occlusion conditions, which remains a significant hurdle for robust autonomous systems. Occlusions occur when tracked objects become partially or completely hidden behind other objects or leave the camera’s field of view, which can cause conventional trackers to lose target identity and accurate positioning. We present a novel approach that combines synthetic data generation and specialized neural network architectures to maintain tracking performance even when the target is partially or completely hidden from view. We implement and experimentally compare four distinct approaches: LiteTracker-CNN with a ResNet18 backbone and specialized point regression for gate corner tracking, LiteTracker- ViT using cross-frame attention to maintain spatial coherence during occlusions, Fine-Tuned CoTracker3 (FT) adapted using synthetic AirSim data, and a semi-supervised fine-tuned CoTracker3 (FT Pseudo) with pseudo-labeling for unlabeled footage. Using Microsoft Air- Sim simulator-generated datasets containing precise ground truth annotations even during occlusion events, we systematically evaluate each approach’s performance across controlled occlusion scenarios. Our experimental results, validated through a benchmark across different occlusion types (non occluded, partially occluded, and fully occluded), reveal that each architecture excels in different aspects: ViT provides the most robust tracking through occlusion events, CNN delivers the fastest processing for resource-constrained applications, and FT offers a balanced middle-ground solution. The ViT implementation specifically achieves a nearly 99% success rate on occluded points and 82% lower error metrics compared to baselines. Analysis of performance degradation showed that ViT experiences only 35.7% error increase post-occlusion compared to CNN’s 47.4% and baseline models’ 540+% degradation. Our supervised fine-tuning approach significantly outperformed the pseudo-label self-supervised method for occlusion handling, with a success rate of 97.11% compared to 46.22% with the pseudo-label approach. This demonstrates that even limited annotated data substantially improves tracking performance through occlusions.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois Urbana-Champaign

Year dc:date

2025

Author dc:creator

• Datta, Akul

Contributors dc:contributor

• Mitra, Sayan

• Drones
• Computer Vision
• Drone Tracking

Statement dc:rights

• Copyright 2025 Akul Datta

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/129249

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F129249