Polar Region Sea Ice Classification With Multimodal Learning

The rapid changes in the polar regions due to climate change have significant global implications, necessitating accurate monitoring and classification of sea ice. This thesis presents a multimodal learning approach for classifying polar sea ice thickness using both satellite imagery and altimetry data. The objective is to distinguish between thick ice, thin ice, and open water, which are crucial for understanding ice dynamics and predicting future changes. Our methodology leverages deep learning models to extract meaningful features from satellite images and integrates them with complementary information from altimetry measurements, thereby enhancing classification accuracy. The proposed system employs a multimodal deep learning technique from a convolutional neural networks (CNNs) for polar image data and a RNN architecture for text data fusion, allowing the model to learn from multiple data modalities effectively. The experimental results demonstrate that the multimodal approach significantly outperforms single-modality models in terms of classification precision and accuracy. This work uses three main techniques to perform the fusion of the two modalities: LSTM model and U-Net model using Additive Fusion (AF), Multiplicative Fusion (MLF), and Gated Fusion (GF). Among the three techniques, GF outperforms the other two in terms of accuracy, precision, F1-score, and Recall.

Author dc:creator

• Lechamo, Nathan

Contributors dc:contributor

• Prasad, Sushil
• Tabar, Maryam
• Wang, Wei

• Gated Fusion
• Late Fusion
• Multimodal
• Sea Ice Classification
• Multimodal learning
• Multiplicative Fusion

Identifier

9798346848912

OAI identifier oai:identifier

oai:tdl-ir.tdl.org:20.500.12588/6957