Failure to Reject the p-value is Not the Same as Accepting it: The Development, Validation, and Administration of the KPVMI Instrument

The purpose of this study was to investigate on a national scale the baseline level of p-value fluency of future researchers (i.e., doctoral students). To that end, two research questions were investigated. The first research question, Can a sufficiently reliable and valid measure of p-value misinterpretations (in a research context) be constructed?, was addressed via the development and validation of the Keller P-value Misinterpretation Inventory instrument (KPVMI). An iterative process of expert review, pilot testing, and field testing resulted in an adequately reliable measure (Alpha = .8030) of p-value fluency as assessed across 18 misinterpretations and 2 process levels as well as an independently validated sub-measure of p-value fluency in context as assessed across 18 misinterpretations (Alpha = .8298). The second research question, What do the results of the KPVMI administration tell us about the current level of p-value fluency among doctoral students nationally?, was addressed via analysis of a subset of the field test data (n = 147) with respect to performance on the subset of items considered sufficiently validated as developed in Phases I-III (KPVMI-1). The median score was 10/18 items answered correctly indicating that future researchers on the aggregate struggle to properly interpret and report p-values in context; furthermore, there was insufficient evidence to indicate training and experience are positively correlated with performance. These results aligned with the extant literature regarding the p-value misinterpretations of practicing researchers.

Name thesis:degree_name

Doctor of Philosophy

Level thesis:degree_level

doctoral

Discipline thesis:degree_discipline

Curriculum and Instruction

Department dc:contributor.department

Education, Vocational-Technical

Grantor dc:publisher

Virginia Tech

Year dc:date.issued

2019

Author dc:creator

• Keller, Rachel Elizabeth

Chairs dc:contributor.committeechair

• Skaggs, Gary E.
• Ulrich, Catherine L.

Committee members dc:contributor.committeemember

• Johnson, Estrella
• Driscoll, Anne R.

• p-values
• research methods
• statistical knowledge

Statement dc:rights

• In Copyright

Licence dc:rights.uri

• http://rightsstatements.org/vocab/InC/1.0/

Dc Identifier Other

vt_gsexam:19245

OAI identifier oai:identifier

oai:vtechworks.lib.vt.edu:10919/100743