The application of the Hadoop software framework in Bioinformatics programs

<p>The project described in this dissertation proposal attempted to improve the efficiency and scalability performance as well as the usability and user experience of three Bioinformatics applications - DNA/peptide sequence similarity comparison, digital DNA library subtraction, and DNA/peptide sequence de-duplication - by 1) adopting the Hadoop MapReduce algorithms and distributed file system and 2) implementing the fully automated Hadoop programs into a user friendly graphical user interface (GUI). In addition, the researcher was also interested in investigating the advantages and limitations of applying the Hadoop software framework as a general methodology in parallelizing Bioinformatics programs.</p> <p>After considering the original calculation algorithms in the serial version of the programs, the available computational resources, the nature of the MapReduce framework, and the optimization of performance, a processing pipeline with one pre-processing step, three mappers, two reducers and one post-processing step was developed. Then a GUI interface that enabled users to specify input/output files and program parameters was created. Also implanted into the GUI were user friendly features such as organized instruction, detailed log files, multi-user accessibility, and so on.</p> <p>The new and fully automated Hadoop Bioinformatics toolkit showed execution efficiency comparable with their MPI counterparts with median to large scale data, and better efficiency than MPI when ultra-large dataset was provided. In addition, good scalability was observed with testing dataset up to 20 Gb.</p>

Name thesis:degree_name

Doctor of Philosophy (PhD)

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Technology

Year

2016

Author dc:creator

• Wang, Dan

Contributors dc:contributor

• John Springer
• Kari L. Clase
• Michael A. Kane
• Dawn D. Laux
• Eric Matson

• Biological sciences
• Applied sciences
• Bioinformatics
• Graphical user interface
• Hadoop
• Parallelization

Repository record dc:identifier

https://docs.lib.purdue.edu/open_access_dissertations/725

OAI identifier oai:identifier

oai:docs.lib.purdue.edu:open_access_dissertations-1889