Heterogeneous machine learning: characterization, generation and comprehension

In the era of big data, the increasing complexity of real-world applications has brought data and model heterogeneity to the forefront of machine learning research. Domains such as sustainability, finance, and agriculture rely on the integration of diverse data sources, including climate time series (e.g., MERRA2, ERA5) and satellite imagery (e.g., Landsat8, SAR), to support tasks ranging from weather forecasting and extreme event analysis to broader climate impact assessments. These tasks highlight the inherent data heterogeneity across modalities and objectives, while simultaneously motivating the need for model heterogeneity to address varying learning paradigms. The recent surge in foundation models, such as large language models, vision transformers, and graph neural networks, has further accentuated the challenge of unifying diverse model architectures, thereby giving rise to a new research paradigm: heterogeneous machine learning (HML). HML aims to improve generalizability, adaptability, and trustworthiness by leveraging the richness of data and model diversity. Despite its promise, heterogeneous machine learning poses significant challenges, including characterizing complex heterogeneity involving multiple modalities (view heterogeneity) and task objectives (label heterogeneity), ensuring trustworthy machine learning that adheres to principles of privacy, fairness, robustness, and interpretability, handling incomplete information, a common issue in large-scale heterogeneous datasets where missing views or partial labels are prevalent and establishing strong theoretical foundations to guide the design, analysis, and interpretation of heterogeneous models. Addressing these challenges requires a systematic and interdisciplinary approach. In this thesis, my research tackles these challenges through three interconnected pillars, i.e., Heterogeneous Machine Learning Characterization, Heterogeneous Machine Learning Generation, and Heterogeneous Machine Learning Comprehension. First, in Heterogeneous Machine Learning Characterization, I focus on understanding and modeling the structure of heterogeneous data by capturing inter-modality correlations and task relatedness. I also integrate principles of trustworthy AI to ensure fairness, interpretability, and robustness throughout the modeling process. Second, in Heterogeneous Machine Learning Generation, I develop generative models to address data incompleteness by learning plausible patterns and restoring missing modalities or labels. These models support applications such as data augmentation, anomaly detection, and secure learning in domains where data quality is often compromised. Finally, in Heterogeneous Machine Learning Comprehension, I aim to bridge the gap between empirical success and theoretical understanding. This includes deriving fairness bounds, leveraging information-theoretic frameworks for contrastive learning, and using graph theory and influence functions to analyze model behavior and detect anomalies.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois Urbana-Champaign

Year dc:date

2025

Author dc:creator

• Zheng, Lecheng

Contributors dc:contributor

• He, JIngrui
• Sun, Jimeng
• Wang, Yuxiong
• Birge, John

• Heterogeneous Learning
• Multi-view Learning
• Multi-label Learning
• Contrastive Learning
• Self-supervised Learning

Statement dc:rights

• Copyright 2025 Lecheng Zheng

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/129820

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F129820