New techniques for improving biological data quality through information integration

As databases become more pervasive through the biological sciences, various data quality concerns are emerging. Biological databases tend to develop data quality issues regarding data legacy, data uniformity and data duplication. Due to the nature of this data, each of these problems is non-trivial and can cause many problems for the database. For biological data to be corrected and standardized, methods and frameworks must be developed to handle both structural and traditional data. The BIG-AJAX framework has been developed for solving these problems through both data cleaning and data integration. This framework exploits declarative data cleaning and exploratory data mining to help improve biological data quality. Within the framework, the problems and difficulties for cleaning biological data are discussed, specifically concerning phylogenetic data. Subsequently, it is demonstrated how BIG-AJAX can be used to improve the quality of the data in phylogeny. Moreover, within the cleaning architecture, data integration plays a key role in improving data quality. Data integration, due to the distributed and heterogeneous nature of the data sets becomes an enabling technology for improving data quality. From these concepts, two tools have been built, BIG-AJAX for TreeBASE and BIG-AJAX for Lineage Paths. Each system approaches the data quality problem using the conceptual BIG-AJAX framework while implementing different methods working within the framework to improve the data quality.

Name thesis:degree_name

Doctor of Philosophy in Computing Sciences - (Ph.D.)

Discipline thesis:degree_discipline

Computer Science

Year

2004

Author dc:creator

• Herbert, Katherine Grace

Contributors dc:contributor

• Jason T. L. Wang
• Narain Gehani
• James A. McHugh

• Data integration
• Data cleaning
• Bioinformatics
• Data mining
• Phylogeny
• Databases
• Computer Sciences

Repository record dc:identifier

https://digitalcommons.njit.edu/dissertations/631

OAI identifier oai:identifier

oai:digitalcommons.njit.edu:dissertations-1686