Malleable parallel computing with Ray: a runtime framework for dynamic resource management in iterative solvers

With the increasing demand for scalable parallel computing, frameworks such as MPI have been widely used for scientific research. However, emerging distributed computing frameworks such as Ray provide new opportunities for more flexible and dynamic resource management. This thesis presents an MPI-like framework implemented in Ray which provides one way to migrate the MPI applications to the Ray platform. Our framework features a dynamic process allocation mechanism that empowers process migration by allowing the application’s processes to expand or shrink. This thesis also implements the V-cycle multigrid method, a widely used solver for large-scale linear systems, to evaluate the performance and scalability of our framework. Our experiments demonstrate the trade-offs between static and dynamic resource allocation, highlighting the impact on computational efficiency. We show that while MPI provides strong performance guarantees under fixed workloads, Ray’s flexibility in rank adaptation can significantly improve resource utilization in dynamic environments. Our results provide valuable insights into the feasibility of using Ray as an alternative to MPI for large-scale scientific computing, particularly in scenarios where dynamic load balancing and resource elasticity are crucial.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Electrical & Computer Engr

Grantor

University of Illinois Urbana-Champaign

Year dc:date

2025

Author dc:creator

• Yuan, Yue

Contributors dc:contributor

• Kindratenko, Volodymyr

• Malleable MPI
• Ray
• Cloud

Statement dc:rights

• Copyright 2025 Yue Yuan

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/129273

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F129273