Terapixel image processing and simulation with distributed halide

Many image processing and simulation tasks are naturally expressed as a pipeline of small computational kernels known as stencils. Halide is a popular domain-specific language and compiler designed to implement stencil algorithms. Halide uses simple language constructs to express what to compute and a separate scheduling co-language for expressing how to perform the computation. This approach has demonstrated performance comparable to or better than hand-optimized code. Until now, Halide has been restricted to parallel shared memory execution, limiting its performance and applicability to tomorrow's terapixel image processing tasks. In this thesis we present an extension to Halide to support distributed-memory parallel execution of stencil pipelines. These extensions compose with the existing scheduling constructs in Halide, allowing expression of complex computation and communication strategies. Existing Halide applications can be distributed with minimal changes, allowing programmers to explore the tradeoff between recomputation and communication with little effort. Approximately 10 new of lines code are needed even for a 200 line, 99 stage application. On nine image processing benchmarks, my extensions give up to a 1.4 speedup on the same number of cores over regular multithreaded execution by mitigating the effects of non-uniform memory access. The image processing benchmarks achieve up to 18 speedup on a 16 node testing machine and up to 57 speedup on 64 nodes of the NERSC Cori supercomputer. A 3D heat finite-difference simulation benchmark achieves linear scaling from 64 to 512 Cori nodes on a 10, 0003, or 1 terapixel, input. We also demonstrate scalability results for two of the image processing benchmarks on 1 terapixel inputs, and make the argument that supporting such large scale is essential for tomorrow's image processing and simulation needs.

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2016

Author dc:creator

• Denniston, Tyler

Advisor dc:contributor.advisor

• Saman Amarasinghe.

• Electrical Engineering and Computer Science.

Statement dc:rights

• M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/1721.1/103678

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/103678