Federated Learning With Generalization To New Domains

Federated Learning (FL) is an area of research that focuses on training machine learning models in a decentralized fashion without having the need to store all data on one central server. In this thesis, we address the challenges of data heterogeneity and label scarcity in FL by proposing two novel approaches for federated domain generalization in both unsupervised and supervised settings. First, to tackle federated domain generalization in an unsupervised setting, we introduce Federated Unsupervised Domain Generalization using Global and Local Alignment of Gradients. We establish a connection between domain shifts and gradient alignment in unsupervised federated learning, demonstrating that aligning gradients at both the client and server levels facilitates the generalization of the model to new, unseen domains. FedGaLA performs gradient alignment locally to encourage clients to learn domain-invariant features, and globally at the server to obtain a more generalized aggregated model. Extensive experiments on four multi-domain datasets—PACS, OfficeHome, DomainNet, and TerraInc—show that FedGaLA outperforms comparable baselines. Ablation and sensitivity studies highlight the impact of different components and hyper-parameters in our approach. Second, to address data heterogeneity in a supervised federated learning framework, we propose Federated Domain Generalization with Label Smoothing and Balanced Decentralized Training (FedSB). FedSB utilizes label smoothing at the client level to prevent overfitting to domain-specific features, thereby enhancing generalization capabilities across diverse domains when aggregating local models into a global model. Additionally, FedSB incorporates a decentralized budgeting mechanism that balances training among clients, improving the performance of the aggregated global model. Experiments on four commonly used multi-domain datasets—PACS, VLCS, OfficeHome, and TerraInc—demonstrate that FedSB outperforms competing methods, achieving state-of-the-art results on three out of four datasets. Collectively, these contributions address critical challenges in FL by enhancing model generalization across diverse and unseen domains in both unsupervised and supervised settings. The effectiveness of FedGaLA and FedSB in addressing data heterogeneity is evidenced by their superior performance in extensive empirical evaluations.

Department dc:contributor.department

Electrical and Computer Engineering

Year dc:date.issued

2024

Author dc:creator

• Soltany, Milad

Advisors dc:contributor.supervisor

• Greenspan, Michael
• Etemad, Ali

• Federated Learning
• Computer Vision
• Deep Learning
• Domain Generalization
• Federated Domain Generalization
• Self Supervised Learning

Statement dc:rights

• Attribution 4.0 International

Licence dc:rights.uri

• http://creativecommons.org/licenses/by/4.0/

Language dc:language.iso

eng

Handle dc:identifier.uri

https://hdl.handle.net/1974/34196

OAI identifier oai:identifier

oai:queensu.scholaris.ca:1974/34196