Joint spatial and layer attention for convolutional networks

In this work, we propose a novel approach that learns to sequentially attend to different Convolutional Neural Networks (CNN) layers (i.e., “what” feature abstraction to attend to) and different spatial locations of the selected feature map (i.e., “where”) to perform the task at hand. Specifically, at each Recurrent Neural Network (RNN) step, both a CNN layer and localized spatial region within it are selected for further processing. We demonstrate the effectiveness of this approach on two computer vision tasks: (i) image-based six degree of freedom camera pose regression and (ii) indoor scene classification. Empirically, we show that combining the “what” and “where” aspects of attention improves network performance on both tasks. We evaluate our method on standard benchmarks for camera localization (Cambridge, 7-Scenes, and TUM-LSI) and for scene classification (MIT-67 Indoor Scenes). For camera localization our approach reduces the median error by 18.8% for position and 8.2% for orientation (averaged over all scenes), and for scene classification it improves the mean accuracy by 3.4% over previous methods.

Name thesis:degree_name

Master of Science (MSc)

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Ontario Institute of Technology

Year dc:date.issued

2019

Author dc:creator

• Joseph, Tony

Advisors dc:contributor.advisor

• Qureshi, Faisal
• Derpanis, Kosta

• Computational Attention
• Convolutional Neural Networks
• Reccurent Neural Networks
• Neural Networks
• Image-Based Camera Localization

Language dc:language.iso

en

Handle dc:identifier.uri

https://hdl.handle.net/10155/1061

OAI identifier oai:identifier

oai:ontariotechu.scholaris.ca:10155/1061