Mélange: Multi-tenant scheduling with adaptive eviction for graph processing clusters

Multi-tenancy is an important approach to resource consolidation in cluster management. In this thesis we design and evaluate Mélange, an efficient multi-tenant scheduler targeted towards graph processing jobs. Mélange supports job priorities and eviction, while attempting to avoid starvation. We propose novel ways of exploiting domain-specific knowledge to achieve better scheduling decisions for graph processing jobs. We evaluate static eviction policies and design Mélange to adapt to the cluster and job state at run time to reduce overhead costs during eviction. We have developed Mélange as a cross-layer scheduler built over Apache Giraph and YARN, and show experimental results with synthetic as well as production workloads.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2018

Author dc:creator

• Mehar, Jayasi

Contributors dc:contributor

• Gupta, Indranil

• graph processing
• multi-tenancy
• eviction
• scheduling

Statement dc:rights

• Copyright 2018 Jayasi Mehar

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/101212

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/101212