Deep image representation learning for knowledge discovery from earth observation data archives

Advances in remote sensing (RS) technology have increased the availability of images regularly acquired by satelliteborne and airborne sensors, while free data policies support researchers to have access to massive Earth observation data archives. To automatically extract knowledge from these archives on a large-scale, deep learning (DL) based RS image representation learning (IRL) has attracted great attention. However, existing methods have limitations on: i) accurate characterization of high-level semantic content and spectral information present in RS images; ii) modelling RS image similarities by exploiting multi-label training images; iii) time efficient and scalable information extraction; iv) effective IRL under noisy training labels; and v) joint use of multiple learning tasks for describing the complex content of RS images. This thesis aims to develop advanced DL-based IRL methods to tackle these limitations, while a particular attention is devoted to image scene classification and content-based image retrieval (CBIR) problems due to their importance for large-scale knowledge discovery. In detail, we propose five DL-based IRL methods throughout the thesis. First, a multi-label classification approach is introduced to accurately describe complex spatial and spectral content of high-spatial resolution RS images, where several spectral bands are associated with varying spatial resolutions. Second, we propose an image triplet sampling method for IRL through the characterization of RS image similarities, which forms the foundation for CBIR. Among multi-label training images, this method selects a small set of the most representative and informative image triplets that lead to a decrease in computational complexity and an increase in learning speed without a significant loss in performance. Third, an approach devoted to simultaneous RS image compression and indexing is introduced for scalable CBIR. This approach characterizes hash codes of RS images on learning based compression domain; and thus prevent the requirement of decoding images prior to CBIR that can save a significant amount of time. Fourth, we propose an approach for IRL when training data includes noisy labels. By integrating generative reasoning into discriminative reasoning, our approach models the complementary characteristics of discriminative and generative reasoning, and thus prevents the interference of noisy labels during training. Fifth, a multitask learning approach is introduced to achieve IRL when multiple learning tasks are jointly utilized. Due to its loss functions and sequential optimization algorithm, this approach preserves the plasticity for each task and the stability in between learning consecutive tasks. For benchmarking the proposed methods, we introduce a large-scale multi-modal multi-label benchmark RS image archive (denoted as BigEarthNet). It includes 590,326 pairs of Sentinel-1 and Sentinel-2 image patches acquired over 10 European countries. We make BigEarthNet, its pre-trained DL models and the codes of all the methods publicly available as open source contributions of the thesis.

Author dc:creator

• Sümbül, Gencer

Advisor dc:contributor.advisor

• Demir, Begüm

Licence dc:rights.uri

• http://rightsstatements.org/vocab/InC/1.0/

Language dc:language.iso

en

Identifier URI

https://doi.org/10.14279/depositonce-18481

OAI identifier oai:identifier

oai:depositonce.tu-berlin.de:11303/19682