On Communication-Computation Overlap in High-Performance Computing

The number of compute nodes and cores per node have increased many fold on high end computer systems over the last decade. For a parallel application to scale to tens or even hundreds of thousands of processes, all non-computing related operations have to be kept at an absolute minimum, including communication operations. Non-blocking-collective operations extend the concept of collective operations by offering the additional benefit of being able to overlap communication and computation. However, it has been demonstrated that collective operations have to be carefully tuned for a given platform and application scenario to maximize their performance. Also, using non-blocking-collective operations in real-world applications is non-trivial. Application codes often have to be restructured significantly in order to maximize the communication-computation overlap. The goal of this dissertation is to optimize non-blocking collective-communication operations and facilitate their utilization at the end-user level. This is achieved by an automatic run-time tuning of non-blocking collective-communication operations, which allows the communication library to maximize communication-computation overlap, and choose the best performing implementation for a non-blocking-collective operation on a case by case basis. Specifically, an approach to maximize the communication-computation overlap for hybrid OpenMP/MPI applications is developed. It leverages automatic parallelization by extending existing concepts to utilize non-blocking-collective operations. It also integrates the run-time auto-tuning techniques of non-blocking-collective operations, optimizing both, the algorithms used for the non-blocking-collective operations as well as location and frequency of accompanying progress-function calls.

Name thesis:degree_name

Doctor of Philosophy

Level thesis:degree_level

Doctoral

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Houston

Year dc:date.issued

2016

Author dc:creator

• Barigou, Youcef 1987-

Advisor dc:contributor.advisor

• Gabriel, Edgar

Committee members dc:contributor.committeemember

• Subhlok, Jaspal
• Shah, Shishir Kirit
• Grabow, Lars C.

• High performance computing
• Message Passing Interface
• MPI
• Collective Operations
• Communication-computation Overlap
• Optimization
• Distributed memory
• Automatic Parallelization

Statement dc:rights

• The author of this work is the copyright owner. UH Libraries and the Texas Digital Library have their permission to store and provide access to this work. UH Libraries has secured permission to reproduce any and all previously published materials contained in the work. Further transmission, reproduction, or presentation of this work is prohibited except with permission of the author(s).

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/10657/3274

OAI identifier oai:identifier

oai:uh-ir.tdl.org:10657/3274