Learning based hierarchical load balancer

Computer architects are finding newer ways to develop faster systems despite the restrictions with respect to the speed of each individual processing units. Often this involves complex parallel and heterogeneous design of these machines. To fully exploit these machines it is becoming necessary for parallel applications running on these machines to improve their software design and also develop better tools to use along with them. One of the major challenges in this design is the problem of load balancing. Load imbalance can lead to wastage of hardware resources and thereby slow the overall running time of the application. The nature of load balancing that we use and the properties of load balancer often is determined by the application and hardware we run the application on. We can use computation-aware, communication-aware or topology-aware load balancers. Amongst them there are a host of available static or dynamic load balancing options that an application can choose from. Often though this needs extensive domain knowledge and multiple runs of the application to tune the load balancing frequency and the list of load balancers that we will use for our final model. This requires both human time and resource utilization just to find the best possible configurations for one particular application. Inspite of doing all these, we still might not always find the best possible configurations. This makes the challenge of choosing a load balancer complex even with the diverse options available to us. Even with finely tuned load balancing configurations, we may have situations where we do not need a global load balancing step. A global load balancing step can have an overhead that affects application performance if this is done when not needed. We can instead solve the problems by having multiple within node load balancing steps, which has a smaller overhead. This creates another challenge with respect to scaling load balancers to large clusters. In the following dissertation we will focus on solving these issues with load balancers while also proposing models that run better than load balancers proposed after multiple iterations of fine-tuning. Here, we propose a load balancing algorithm that uses runtime system collected statistics to represent the state of an application and then uses learning models to predict a good choice of load balancer to use based on the collected statistics. We also propose the load balancer within a hierarchical framework that allows us to scale the load balancer better across multiple nodes. The hierarchical framework also allows us to avoid the overhead of global load balancing, when within node load balancing can solve the problems with an application.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2022

Author dc:creator

• Ghosh, Pathikrit

Contributors dc:contributor

• Kale, Laxmikant

• High Performance Computing
• Load Balancer
• Machine Learning

Statement dc:rights

• Copyright 2022 Pathikrit Ghosh

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/115706

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F115706