UncL-STARK: An Architecture-preserving Transformer-based Tracker with Uncertainty-aware Layer-skipping

Transformer-based single-object trackers achieve high accuracy but rely on fixed-depth inference, executing the full encoder–decoder stack for every frame regardless of difficulty. This leads to unnecessary computational cost in long video sequences dominated by visually simple frames. This thesis introduces UncL-STARK, an architecture-preserving extension of STARK that enables dynamic, uncertainty-aware layer skipping without modifying the underlying network or adding auxiliary heads. UncL-STARK leverages a low-overhead estimation of uncertainty derived directly from the model’s localization heatmaps to guide adaptive depth selection across frames. This enables a feedback-driven inference strategy in which the computational depth allocated to each frame is adjusted based on the estimated confidence of previous predictions. Extensive experiments on GOT-10k and LaSOT demonstrate that UncL-STARK achieves up to 12% GFLOPs reduction, 8.9% latency reduction, and 10.8% energy savings, while incurring as little as 0.2% average accuracy degradation. Performance remains consistent across both short-term and long-term sequences, in line with the underlying assumption that adjacent frames in tracking sequences exhibit strong visual similarity. Overall, this work presents the first uncertainty-guided, feedback-driven layer-skipping approach for transformer-based tracking, showing that substantial efficiency gains can be achieved without compromising tracking stability or reliability.

Author dc:creator

• Patrick Poggi (24399512)

• Engineering, Electronics and Electrical
• Computer Science

Statement dc:rights

• In Copyright

DOI dc:identifier

https://doi.org/10.25417/uic.32993954.v1

OAI identifier oai:identifier

oai:figshare.com:article/32993954