Trustworthy transfer learning

Deep transfer learning aims to improve the generalization performance of a learning algorithm on the target domain, by discovering and transferring latent knowledge from relevant source domains. This paradigm is motivated by the human intelligence that people have an inherent ability to apply knowledge already learned from one task to another related one. Despite the exploding interest in deep transfer learning techniques, several fundamental problems in understanding deep transfer learning remain unsolved. This thesis investigates two fundamental research questions for understanding deep transfer learning. (Q1) Knowledge Transferability: How is the source knowledge transferred to the target domain with performance guarantees? The key challenge behind knowledge transfer- ability is the data heterogeneity between source and target domains. Notably, samples within a domain can be either independent and identically distributed (IID) or non-independent and identically distributed (non-IID) in real scenarios. This leads to various underlying factors contributing to data heterogeneity across domains, such as feature distribution shifts in cross-domain image recognition, and graph structure shifts in cross-network mining. (Q2) Knowledge Trustworthiness: Can the transferred source knowledge be reliable and trustworthy for learning the prediction function on the target domain? Specifically, multiple trustworthy concerns exist in the transfer learning process, such as adversarial robustness, privacy, fairness, transparency, etc. Having both research questions in mind, we formulate the problem of trustworthy transfer learning. The goal is to learn a reliable and trustworthy prediction function on the target domain using relevant knowledge from source domains. In this thesis, we systematically develop a suite of theoretical tools and algorithms, in order to understand deep transfer learning from the perspectives of knowledge transferability and trustworthiness. Concisely, we answer the first question (Q1) by theoretically deriving the generalization performance of deep transfer learning and empirically validating its efficacy across various IID and non-IID learning tasks. Furthermore, to answer the second question (Q2), this thesis delves into various trustworthy properties of deep transfer learning, including demonstrating the adversarial robustness of deep transfer learning techniques, developing privacy-preserving transfer learning approaches, and enhancing performance fairness among participant domains. Finally, we present several promising research directions, including (i) unifying knowledge transferability across different data modalities; (ii) uncovering the correlations between data distribution shifts and model transparency, and (iii) unveiling the trade-off between knowledge transferability and trustworthiness.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2024

Author dc:creator

• Wu, Jun

Contributors dc:contributor

• He, Jingrui
• Han, Jiawei
• Zhao, Han
• Wang, Haixun

• Transfer Learning
• Distribution Shifts
• Trustworthiness

Statement dc:rights

• Copyright 2024 Jun Wu

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/125540

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F125540