Evaluating a Cluster of Low-Power ARM64 Single-Board Computers with MapReduce

<p>With the meteoric rise of enormous data collection in science, industry, and the cloud, methods for processing massive datasets have become more crucial than ever. MapReduce is a restricted programing model for expressing parallel computations as simple serial functions, and an execution framework for distributing those computations over large datasets residing on clusters of commodity hardware. MapReduce abstracts away the challenging low-level synchronization and scalability details which parallel and distributed computing often necessitate, reducing the concept burden on programmers and scientists who require data processing at-scale. Typically, MapReduce clusters are implemented using inexpensive commodity hardware, emphasizing quantity over quality due to the fault-tolerant nature of the MapReduce execution framework. The nascent explosion of inexpensive single-board computers designed around multi-core 64bit ARM processors, such as the RasberryPi 3, Pine64, and Odroid C2, has opened new avenues for inexpensive and low-power cluster computing. In this thesis, we implement a novel cluster around low-power ARM64 single-board computers and the Disco Python MapReduce execution framework. We use MapReduce to empirically evaluate our cluster by solving the Word Count and Inverted Link Index problems for the Wikipedia article dataset. We benchmark our MapReduce solutions against local solutions of the same algorithms for a conventional low-power x86 platform. We show our cluster out-performs the conventional platform for larger benchmarks, thus demonstrating low-power single-board computers as a viable avenue for data-intensive cluster computing.</p>

Name thesis:degree_name

Master of Science (MS) in Computer Science

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Year

2018

Author dc:creator

• McDermott, Daniel

• Databases and Information Systems
• Numerical Analysis and Scientific Computing
• Theory and Algorithms

Statement dc:rights

• Access is available to all users

Repository record dc:identifier

https://dc.ewu.edu/theses/474

OAI identifier oai:identifier

oai:dc.ewu.edu:theses-1474