Computing with Chunks

Modern computing substrates like general-purpose GPUs, massively multi-core processors, and cloud computing clusters offer practically unlimited resources at the cost of requiring programmers to manage those resources for correctness, efficiency, and performance. Instead of using generic abstractions to write their programs, programmers of modern computing substrates are forced to structure their programs around available hardware. This thesis argues for a new generic machine abstraction, the Chunk Model, that explicitly exposes program and machine structure, making it easier to program modern computing substrates. In the Chunk Model, fixed-sized chunks replace the flat virtual memories of traditional computing models. Chunks may link to other chunks, preserving the structure of and important relationships within data and programs as an explicit graph of chunks. Since chunks are limited in size, large data structures must be divided into many chunks, exposing the structure of programs and data structures to run-time systems. Those run-time systems, in turn, may optimize run-time execution, both for ease of programming and performance, based on the exposed structure. This thesis describes a full computing stack that implements the Chunk Model. At the bottom layer is a distributed chunk memory that exploits locality of hardware components while still providing programmer-friendly consistency semantics and distributed garbage collection. On top of the distributed chunk memory, we build a virtual machine that stores all run-time state in chunks, enabling computation to be distributed through the distributed chunk memory system. All of these features are aimed at making it easier to program modern computing substrates. This thesis evaluates the Chunk Model through example applications in cloud computing, scientific computing, and shared client/server computing.

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2013

Author dc:creator

• Paluska, Justin Mazzola, 1981-

Advisor dc:contributor.advisor

• Steve Ward.

• Electrical Engineering and Computer Science.

Statement dc:rights

• M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/1721.1/82188

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/82188