{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/130086"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/130086","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Chasing the “tail at scale”: toward cloud-native architectures","abstract":"Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2027-08-01","abstract_html":"Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2027-08-01","abstract_has_math":false,"creators":["Stojkovic, Jovan"],"institution":"University of Illinois Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Torrellas, Josep","Xu, Tianyin","Marinov, Darko","Huang, Jian","Franke, Hubertus","Delimitrou, Christina","Skarlatos, Dimitrios","Hughes, Christopher J."],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025-07-08","date_published":"2025-07-08","updated_at":"2026-07-22T22:25:06Z","subjects":["Cloud Computing","Serverless Computing","Microservices","Computer Architecture"],"languages":["en","eng"],"rights":["Copyright 2025 Jovan Stojkovic"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/130086","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Torrellas, Josep","Xu, Tianyin","Marinov, Darko","Huang, Jian","Franke, Hubertus","Delimitrou, Christina","Skarlatos, Dimitrios","Hughes, Christopher J."]},{"key":"dc:creator","label":"Author","values":["Stojkovic, Jovan"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2025-07-08","2025-08"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Cloud Computing","Serverless Computing","Microservices","Computer Architecture"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2025 Jovan Stojkovic"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/130086"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2027-08-01","The student, Jovan Stojkovic, accepted the attached license on 2025-07-07 at 20:51.","The student, Jovan Stojkovic, submitted this Dissertation for approval on 2025-07-07 at 21:00.","This Dissertation was approved for publication on 2025-07-08 at 13:29.","DSpace SAF Submission Ingestion Package generated from Vireo submission #22435 on 2025-10-25 at 15:30:51","Cloud computing is undergoing a radical transformation with the emergence of lightweight cloud-native computing paradigms, such as microservices and serverless computing. Users build their applications by combining services, benefiting from a simplified programming model and fine-grained billing. At the same time, providers consolidate many services into a smaller number of servers, improving the utilization of their infrastructure. However, the detailed characterization of cloud-native environments presented in this thesis shows that these workloads differ significantly from traditional monolithic applications. They execute services that run for short times, exhibit bursty invocation patterns, and have frequent I/O operations that cause context switches. In addition to their core logic, services also execute many auxiliary operations known as datacenter tax, such as data serialization and encryption. Finally, services have stringent tail latency bounds, requiring the slowest requests to complete within a strict deadline. These characteristics result in significant inefficiencies in performance, energy, and resource utilization when cloud-native workloads run on conventional servers with conventional software stacks, negating the paradigm’s potential benefits. The goal of this thesis is to design hardware platforms and software stacks that enable the execution of cloud-native workloads with orders of magnitude better efficiency. The first part of the thesis designs a new hardware stack for cloud-native services. It introduces µManycore, a CPU architecture that minimizes the tail latency of cloud-native services. The thesis then extends the architecture with HardHarvest to boost utilization via hardware-based core harvesting, and refines the microarchitecture with Mosaic for better performance under frequent context switches. Finally, this thesis integrates on-package accelerators into the architecture and proposes AccelFlow, a framework that enables fine-grained, low-overhead orchestration of accelerators to reduce the datacenter tax in cloud-native environments. To maximize the efficiency of the proposed hardware architecture, the second part of the thesis builds a full software stack that is tightly co-designed with the hardware. It begins with MXFaaS, a mechanism that improves resource utilization by efficiently multiplexing resources during bursts of same-function invocations. Then, it integrates the novel Concord distributed caching system for FaaS environments, and uses SpecFaaS to accelerate end-toend application workflows through speculative service execution. Finally, this thesis improves the energy efficiency of cloud-native environments with two frameworks: EcoFaaS, which uses fine-grained scheduling and dynamic frequency scaling, and SmartOClock, which underprovisions resources and selectively overclocks cores during load spikes."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Chasing the “tail at scale”: toward cloud-native architectures"]}]}],"canonical_facts":{"dc:contributor":["Torrellas, Josep","Xu, Tianyin","Marinov, Darko","Huang, Jian","Franke, Hubertus","Delimitrou, Christina","Skarlatos, Dimitrios","Hughes, Christopher J."],"dc:creator":["Stojkovic, Jovan"],"dc:date":["2025-07-08","2025-08"],"dc:description":["Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2027-08-01","The student, Jovan Stojkovic, accepted the attached license on 2025-07-07 at 20:51.","The student, Jovan Stojkovic, submitted this Dissertation for approval on 2025-07-07 at 21:00.","This Dissertation was approved for publication on 2025-07-08 at 13:29.","DSpace SAF Submission Ingestion Package generated from Vireo submission #22435 on 2025-10-25 at 15:30:51","Cloud computing is undergoing a radical transformation with the emergence of lightweight cloud-native computing paradigms, such as microservices and serverless computing. Users build their applications by combining services, benefiting from a simplified programming model and fine-grained billing. At the same time, providers consolidate many services into a smaller number of servers, improving the utilization of their infrastructure. However, the detailed characterization of cloud-native environments presented in this thesis shows that these workloads differ significantly from traditional monolithic applications. They execute services that run for short times, exhibit bursty invocation patterns, and have frequent I/O operations that cause context switches. In addition to their core logic, services also execute many auxiliary operations known as datacenter tax, such as data serialization and encryption. Finally, services have stringent tail latency bounds, requiring the slowest requests to complete within a strict deadline. These characteristics result in significant inefficiencies in performance, energy, and resource utilization when cloud-native workloads run on conventional servers with conventional software stacks, negating the paradigm’s potential benefits. The goal of this thesis is to design hardware platforms and software stacks that enable the execution of cloud-native workloads with orders of magnitude better efficiency. The first part of the thesis designs a new hardware stack for cloud-native services. It introduces µManycore, a CPU architecture that minimizes the tail latency of cloud-native services. The thesis then extends the architecture with HardHarvest to boost utilization via hardware-based core harvesting, and refines the microarchitecture with Mosaic for better performance under frequent context switches. Finally, this thesis integrates on-package accelerators into the architecture and proposes AccelFlow, a framework that enables fine-grained, low-overhead orchestration of accelerators to reduce the datacenter tax in cloud-native environments. To maximize the efficiency of the proposed hardware architecture, the second part of the thesis builds a full software stack that is tightly co-designed with the hardware. It begins with MXFaaS, a mechanism that improves resource utilization by efficiently multiplexing resources during bursts of same-function invocations. Then, it integrates the novel Concord distributed caching system for FaaS environments, and uses SpecFaaS to accelerate end-toend application workflows through speculative service execution. Finally, this thesis improves the energy efficiency of cloud-native environments with two frameworks: EcoFaaS, which uses fine-grained scheduling and dynamic frequency scaling, and SmartOClock, which underprovisions resources and selectively overclocks cores during load spikes."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/130086"],"dc:language":["en","eng"],"dc:rights":["Copyright 2025 Jovan Stojkovic"],"dc:subject":["Cloud Computing","Serverless Computing","Microservices","Computer Architecture"],"dc:title":["Chasing the “tail at scale”: toward cloud-native architectures"],"dc:type":["text"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:06Z"}