Efficient Resource Utilization for Parallel I /O in Cluster Environments

In this thesis work, performance factors for parallel I/O on clusters are examined and several algorithms are designed to support parallel I/O efficiently for scientific applications running on commodity clusters, making better utilization of system resources. Specifically, our algorithms are designed to (i) minimize data transfer over the network during I/O if network bandwidth is limited, (ii) reduce message passing latency during I/O of finely-distributed data, (iii) place I/O servers on the appropriate processors in heterogeneous environments, and (iv) balance I/O workload dynamically when necessary. These algorithms have been implemented in the Panda parallel I/O library and tested on several actual and simulated cluster environments. Performance results show that our algorithms improve overall parallel I/O performance significantly with only an insignificant amount of overhead. These algorithms can also be used in other parallel I/O runtime libraries or job schedulers for cluster systems.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2015

Author dc:creator

• Cho, Yong Eun

Contributors dc:contributor

• Winslett, Marianne

• Computer Science

Language dc:language

eng

Identifier

(MiAaPQ)AAI9952989

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/81953