Systematic replications and statistical reproducibility of educational research

Science is at a critical juncture: the findings of many studies are unable to be replicated and reproduced, while scientific output is growing exponentially and becoming easily accessible. The inability of findings to replicate has been particularly prominent in the field of psychology, where it has been estimated that less than half of findings are able to be replicated. A similar conclusion has been drawn about educational research. At the same time, thousands of papers are published making it increasingly difficult for researchers to know whether or not findings have replicated. This thesis addresses the replicability and reproducibility of educational research and proposes tools that could help researchers sift through large amounts of scholarly output. The first paper of this thesis differentiates between the ideas of replicability and reproducibility, and describes how educational researchers can design systematic replications and report the details needed to reproduce statistical analyses. The second paper examines the use of different text classifiers to extract details about the findings and contextual factors of published articles, where this information can be used by researchers to determine whether two papers are systematic replications of one another. The third paper develops text classifiers to identify the details needed to reproduce the statistical analyses in published papers. These three papers demonstrate there are many components needed to replicate and reproduce educational studies, and these details are sometimes easily identified by text classifiers.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Educational Psychology

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2019

Author dc:creator

• Crues, Robert Wesley

Contributors dc:contributor

• Anderson, Carolyn J.
• Perry, Michelle
• Paquette, Luc
• Zhai, ChengXiang

• systematic replication, statistical reproducibility, text mining

Statement dc:rights

• Copyright 2019 Robert Crues

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/104986

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/104986