Multi-modal object detection in non-corresponding imagery using unsupervised techniques
type: CreativeWork
description: Substantial advancements have been made in the field of object detection in the past number of years. The presented thesis focuses on multi-modal (RGB-LWIR) object detection for surveillance in a Security and Defence context. At present, the current state-of-the-art in object detection use convolutional neural networks (CNNs). CNNs require large datasets for training; however, there is a severe lack of labelled LWIR datasets available, unlike the RGB modality. Collecting, preprocessing and annotating a large dataset is exceptionally time-consuming and expensive. Therefore, we aim to develop novel techniques to achieve high detection rates in small LWIR datasets using unsupervised techniques. Our ultimate aim is to achieve multi-modal object detection in non-corresponding RGB and LWIR imagery. First, a supervised scenario with only a small low-resolution LWIR dataset available for training is considered. We propose to exploit another larger high-resolution LWIR dataset using transfer learning and fine-tuning techniques usually only used within the RGB domain. This work’s originality lies in our approach focusing on transfer learning across LWIR datasets taken at different locations, angles, resolutions, and target classes at varying distances from the camera. Also, we show that pre-training detection algorithms with the ImageNet RGB dataset significantly improve detection rates in the LWIR imagery by up to ∼ 18%. These results lead us to go a stage further and prove transfer learning between the RGB and LWIR modality is possible. We show that an RGB trained detection algorithm produces some LWIR invariant features. Subsequently, an unsupervised adaptation to this detection network is proposed to increase detection in LWIR imagery. For this adaptation, the distance between the source (RGB) and target (LWIR) distributions is minimised during training to create modality invariant features. This technique can also be applied to non-corresponding imagery, i.e. imagery tha…
abstract: Substantial advancements have been made in the field of object detection in the past number of years. The presented thesis focuses on multi-modal (RGB-LWIR) object detection for surveillance in a Security and Defence context. At present, the current state-of-the-art in object detection use convolutional neural networks (CNNs). CNNs require large datasets for training; however, there is a severe lack of labelled LWIR datasets available, unlike the RGB modality. Collecting, preprocessing and annotating a large dataset is exceptionally time-consuming and expensive. Therefore, we aim to develop novel techniques to achieve high detection rates in small LWIR datasets using unsupervised techniques. Our ultimate aim is to achieve multi-modal object detection in non-corresponding RGB and LWIR imagery. First, a supervised scenario with only a small low-resolution LWIR dataset available for training is considered. We propose to exploit another larger high-resolution LWIR dataset using transfer learning and fine-tuning techniques usually only used within the RGB domain. This work’s originality lies in our approach focusing on transfer learning across LWIR datasets taken at different locations, angles, resolutions, and target classes at varying distances from the camera. Also, we show that pre-training detection algorithms with the ImageNet RGB dataset significantly improve detection rates in the LWIR imagery by up to ∼ 18%. These results lead us to go a stage further and prove transfer learning between the RGB and LWIR modality is possible. We show that an RGB trained detection algorithm produces some LWIR invariant features. Subsequently, an unsupervised adaptation to this detection network is proposed to increase detection in LWIR imagery. For this adaptation, the distance between the source (RGB) and target (LWIR) distributions is minimised during training to create modality invariant features. This technique can also be applied to non-corresponding imagery, i.e. imagery tha…
author: Abbott, Rachael
language: eng
keywords: Multi-modal imagery, LWIR-RGB imagery, object detection, unsupervised adaptation, modality adaptation
date published: 2021-7
date modified: 2026-07-24T03:55:51Z
provider: Queen's University Belfast
publisher: Networked Digital Library of Theses and Dissertations
type: WebPage
source organization: Queen's University Belfast