Evaluating the capabilities of modern machine learning techniques for operational over-ocean cloud masking with satellite imagers

Cloud detection is a fundamental first step in retrieving the geophysical properties of the Earth, determining which pixels contain clouds to generate a cloud mask. Analysis of passive imager missions contributing to the Global Energy and Water Cycle Experiment Cloud Assessments reveals slight differences in cloud detection algorithms and large uncertainties in resulting cloud fractions. These inaccuracies can propagate errors into the remote sensing of various geophysical properties such as sea surface temperature, aerosol concentrations, and cloud optical and microphysical properties. Current operational techniques for cloud detection rely primarily on spectral thresholds to distinguish between cloudy and clear sky pixels. However, expert analysis often relies on human vision and cognition to leverage textural information when manually labeling cloud masks. This vital textural information is not yet integrated into operational techniques, which could enhance detection capabilities. Advancements in machine learning have developed methods to extract, learn, and detect objects from textural characteristics, which could be used for cloud detection. However, in reviewing the literature experimenting with machine learning for cloud masking, it is unclear how to operationalize these advancements. Thus, this study helps clarify this by reevaluating which machine learning techniques are the most performative and reviewing the operational characteristics of each of these models. For example, we evaluate how easily these machine learning techniques can be explained and if they can produce purpose driven cloud masks. Our analysis compared nine supervised machine learning models to determine whether textural-based approaches outperform traditional spectral-based methods. To evaluate these models, high-quality training and testing datasets are derived from Terra Moderate Resolution Imaging Spectroradiometer (MODIS) observations and quality controlling MODIS’ cloud mask. We found that a simple convolutional neural network (CNN), a model relying on textural information, outperformed others with an accuracy of ~96%, surpassing the best spectral-based model by 4%. To ensure these models create cloud masks that serve differing purposes of satellite missions, two models were retrained using a more clear sky conservative cloud mask from the Terra Multi-angle Imaging Spectroradiometer (MISR). The retrained CNN continued to excel, demonstrating its adaptability with an accuracy of ~91%. Although these convolutional models show promising results, their complexity poses challenges regarding explainability. We discuss potential analyses and procedures to help physically explain these models' decision-making processes for operational use. Despite the high performance of these models, the need for a high-quality global training dataset is a highly limiting factor when operationalizing. Creating this global dataset manually would require much effort and may be impractical. To mitigate this, we suggest using existing operational cloud masks as training data, although this introduces a risk of propagating uncertainties into machine learning models. We investigate the use of bi-tempered logistic loss to manage this uncertainty. Though this technique is imperfect for operational use, it provides a foundation for future research to refine the methodology to effectively account for uncertainty in the provided training cloud mask.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Atmospheric Sciences

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2024

Author dc:creator

• Nied, Joseph David

Contributors dc:contributor

• Di Girolamo, Larry

• Machine Learning
• Deep Learning
• Convolutional
• Neural Networks
• Terra
• Modis
• Misr
• Computer Vision
• Remote Sensing
• Cloud Masking

Statement dc:rights

• Copyright 2024 Joseph David Nied

Language dc:language

eng, en

Handle dc:identifier

https://hdl.handle.net/2142/127334

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/127334