Efficient cross-silo federated learning using a computing power-aware scheduler

Cross-silo federated learning offers a promising solution to collaboratively train robust and generalized machine learning models in domains such as healthcare, finance, and scientific projects lacking a centralized data repository, without transferring datasets and compromising the privacy of sensitive local data. Because of the disparity of computing resources among different federated learning clients (i.e., device heterogeneity), the efficiency of the prevalent synchronous federated learning algorithms is hindered by wasting time in waiting for the slow straggler clients. On the other hand, asynchronous federated learning algorithms face challenges with convergence rate and final model accuracy when dealing with non-identically and independently distributed (non-IID) heterogeneous datasets due to the negative impact of outdated local models and client drifting issues. To tackle these challenges in cross-silo federated learning with heterogeneous clients and data, we introduce FedCompass, an innovative semi-asynchronous federated learning algorithm featuring a server-side computing power-aware scheduler that adaptively assigns varying amounts of training tasks to different clients using the estimation of the computing power of individual clients. FedCompass ensures that multiple locally trained models are received for aggregation almost simultaneously, thereby reducing local model staleness and improving global model performance. Meanwhile, the overall training process remains asynchronous, eliminating delays caused by slow straggler clients. Through experiments with diverse non-IID federated datasets, we demonstrate that FedCompass can achieve faster convergence and higher accuracy compared to other asynchronous algorithms while maintaining superior efficiency over synchronous algorithms in scenarios involving heterogeneous clients. The code for FedCompass is open-sourced at https://github.com/APPFL/FedCompass.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2024

Author dc:creator

• Li, Zilinghan

Contributors dc:contributor

• Kindratenko, Volodymyr

• Federated Learning
• Device Heterogeneity
• Cross-silo Federated Learning

Statement dc:rights

• Copyright 2024 Zilinghan Li

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/124159

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F124159