Stateless node rebalancing and stateful building blocks for limitless autoscaling

Management of compute resources for cloud-native microservices relies heavily on autoscalers. Most autoscalers are built atop the fundamental mechanism of adjusting CPU limits---restricting the amount of CPU resources a service is allowed to use, and then they innovate scaling policies within those constraints. However, we show that the presence of CPU limits causes resource wastage and complicates autoscaler design. Thus, we advocate for the removal of CPU limits for allocating resources. Our design of Yet Another AutoScaler (YAAS) shows how this "limitless" design pathway opens up new challenges and opportunities. The development of YAAS was a collaborative effort, and this thesis focuses on addressing one specific challenge - the increase of CPU utilization beyond the allocation resulting in issues like node congestion and imbalances. To counter this, YAAS intelligently combines node scaling and rebalancing policies to meet SLOs (latency thresholds) with minimal disruptions. Experiments show that YAAS reduces CPU allocations by an average of 28% against limit-less baselines while satisfying SLOs. YAAS is limited to autoscaling stateless services. Although autoscalers for stateful services exist, most are paid solutions that are tailored to specific stateful services and do not provide users with full control. In this context, we present an in-depth study on scaling Redis and Kafka and identify the common building blocks for scaling stateful services.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois Urbana-Champaign

Year dc:date

2025

Author dc:creator

• Varadharajan, Thrivikraman

Contributors dc:contributor

• Gupta, Indranil

• autoscaling microservices
• cpu limits
• node rebalancing
• stateful autoscaling

Statement dc:rights

• Copyright 2025 Thrivikraman Varadharajan

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/129548

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F129548