Auto-tuned optimized parallel I/O for GIScience and spatial applications

Reading and writing big data is increasingly becoming a major bottleneck of using high-performance computing systems as we are heading towards the Exascale era. An unprecedented amount of data is being produced everyday by different sources. On the other hand, the computation power of HPC systems is getting scaled to hundreds of thousands cores. However, for an application to be able to utilize this much data and computation power, using I/O effectively is a must. One of the fields dealing with huge amount of data is geographic information science. In this thesis, we have implemented a parallel I/O library specialized for spatial data analysis in GIScience, capable of treating different I/O patterns such as Row-Wise, Column-Wise and Block-Wise I/O. We then establish an auto-tuning framework for finding optimal parallel I/O configurations. This auto-tuning framework is based on genetic algorithm and works on a range of configurations from the parallel file system all the way up to spatial data-analysis applications. The results and findings of a set of I/O intensive experiments executed on large HPC systems are also presented to demonstrate the effectiveness of the framework.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2013

Author dc:creator

• Behzad, Babak

Contributors dc:contributor

• Snir, Marc
• Wang, Shaowen

• Parallel I/O
• GIScience
• High Performance Computing (HPC)
• Spatial Applications
• Auto-Tuning

Statement dc:rights

• Copyright 2013 Babak Behzad

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/44414

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/44414