Machine learning systems in constrained environments

Machine learning (ML) training and inference systems encounter constraints in current computation environments due to increased ML model sizes, the fast-growing popularity of ML/AI, etc. In this thesis, we show how machine learning training and inference systems can be executed successfully and efficiently in constrained computation environments, such as limited-memory GPUs, on-premises clusters, and serverless environments, by using a novel combination of algorithms, optimizations, and well-reasoned system designs. Concretely, we propose (i) a system that enables large ML model training over multiple memory-constrained GPU devices via algorithms and system designs that achieve fast placements with a quality comparable to expert-designed placements, (ii) a system that enables efficient resource sharing among ML inference jobs in fixed-size on-premises clusters by making close-to-optimal autoscaling decisions quickly via several relaxation methods in optimization and prediction, and (iii) a system that enables cost-efficient distributed GNN training on constrained serverless execution environments by auto-tuning configuration via analytic model-based offline optimization and gray-box heuristic-based online optimization.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2024

Author dc:creator

• Jeon, Beomyeol

Contributors dc:contributor

• Gupta, Indranil
• Caesar, Matthew
• Park, Yongjoo
• Wang, Chen

• Machine Learning Systems
• Machine Learning
• Constraints
• Placements
• Autoscaling
• Graph Neural Networks
• Serverless Computing
• Optimizations
• Algorithms

Statement dc:rights

• Copyright 2024 Beomyeol Jeon

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/125692

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F125692