Scaling simple, compact and extended compact genetic algorithms using MapReduce

Data-intensive computing has emerged as a key player for processing large volumes of data exploiting massive parallelism. Data-intensive computing frameworks have shown that terabytes and petabytes of data can be routinely processed. However, there has been little effort to explore how data-intensive computing can help scale evolutionary computation. We present a detailed step-by-step description of how three different evolutionary computation algorithms, having different execution profiles, can be translated into the MapReduce paradigm. Results show that (1) Hadoop is an excellent choice to push evolutionary computation boundaries on very large problems, and (2) that transparent linear speedups are possible without changing the underlying data-intensive flow thanks to its inherent parallel processing.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2010

Author dc:creator

• Verma, Abhishek

Contributors dc:contributor

• Campbell, Roy H.

• MapReduce
• Genetic Algorithms
• Distributed Systems

Statement dc:rights

• Copyright 2010 Abhishek Verma

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/16856

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/16856