Performance models for parallel applications under failures

Due to the growing size of compute clusters, large scale parallel applications increasingly have to deal with hardware malfunctions and other failure scenarios during execution. The overall goal of this research is to get good performance of parallel applications despite failures. This dissertation introduces two mathematical models to improve resilience of parallel applications on two different frameworks. The first one is a mathematical model to minimize job completion time for inter-dependent parallel processes running in a volunteer environment by finding the optimal checkpoint interval. Validation is performed with a sample real world application running on a pool of distributed volunteer nodes. The results shows that the predicted checkpoint interval gives performance closed to optimal checkpoint interval determined empirically after extensive experimentation. The second part of the dissertation evaluates the performance of Hadoop MapReduce applications, with different execution parameters and under different failure scenarios. The dissertation introduces performance models for Hadoop MapReduce applications considering node and process failures. Having a performance model allows to determine optimal settings for some of the parameters, such as split size. Validation of the model is done by running two MapReduce applications with different parameter settings. The results show that different applications require different settings for the same MapReduce parameters and the proposed model can predict the performance very well.

Name thesis:degree_name

Doctor of Philosophy

Level thesis:degree_level

Doctoral

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Houston

Year dc:date.issued

2017

Author dc:creator

• Rahman, Mohammad Tanvir 1983-

Advisor dc:contributor.advisor

• Gabriel, Edgar

Committee members dc:contributor.committeemember

• Subhlok, Jaspal
• Pandurangan, Gopal
• Cheung, Margaret S.

• Performance models
• Parallel Execution
• Fault tolerance
• Volunteer computing
• Checkpointing
• Replication
• Host Selection
• MapReduce
• Hadoop

Statement dc:rights

• The author of this work is the copyright owner. UH Libraries and the Texas Digital Library have their permission to store and provide access to this work. UH Libraries has secured permission to reproduce any and all previously published materials contained in the work. Further transmission, reproduction, or presentation of this work is prohibited except with permission of the author(s).

Language dc:language.iso

eng

Handle dc:identifier.uri

https://hdl.handle.net/10657/5687

OAI identifier oai:identifier

oai:uh-ir.tdl.org:10657/5687