Development of a Method to Measure Clinical Reasoning in Pediatric Residents: The Pediatric Script Concordance Test

Context: National medical examinations boards have requested that methods be developed to assess the clinical reasoning competency of medical trainees. The Script Concordance Test (SCT) is an emerging method of assessment that holds promise for the evaluation of clinical reasoning skills. Objectives: This study had 3 objectives: 1) to examine the validity of SCT scores in accurately discriminating clinical reasoning ability between junior (JR) and senior (SR) pediatric residents and experienced pediatricians serving on the panel of experts (POE), 2) to determine if higher reliability of the SCT method could be achieved by applying specific strategies to the design and construction, and 3) to explore participants’ acceptability of the SCT method. Methods: A 90-minute SCT containing 24 cases and 137 questions was administered to 91 residents from four pediatric residency training centers. Twenty-one pediatricians served on the POE. All participants completed a post-test survey designed to explore their impressions and attitudes regarding the SCT assessment method. Results: Overall, there was a difference in performance across the three levels of experience, F = 22.84 (df = 2); p < 0.001. The POE scored higher than both the SR (mean difference 9.15; p < 0.001) and the JR (mean difference = 14.90; p < 0.001) and the SR scored higher than the JR (mean difference = 5.76; p < 0.002). The reliability of the SCT scores (Cronbach’s α) was 0.85. Sub-test data, based exclusively on the new evolving type cases demonstrated similar results. Participants expressed keen interest and engagement in the SCT format of assessment. Conclusions: The findings of this study contribute to a growing body of literature suggesting that SCT holds promise as a valid, reliable and acceptable method to assess the core competency of clinical reasoning in medicine. The SCT method is based on two core principles we believe are central to the assessment of clinical reasoning: acknowledging uncertainty and the possibility of more than one “right answer.” We propose the SCT may be effectively integrated into formative residency assessment and with increasing exposure, experience and refinement may soon be ready to pilot within summative assessments in pediatric medical education.

Name thesis:degree_name

Doctor of Philosophy (PhD)

Discipline thesis:degree_discipline

Medical Science

Grantor dc:publisher.institution

Graduate Studies

Year dc:date.issued

2015

Author dc:creator

• Cooke, Suzette

Advisor dc:contributor.advisor

• Beran, Tanya

• Medicine and Surgery

Statement dc:rights

• University of Calgary graduate students retain copyright ownership and moral rights for their thesis. You may use this material in any way that is permitted by the Copyright Act or through licensing that has been assigned to the document. For uses that are not allowable under copyright legislation or licensing, you are required to seek permission.

Language dc:language.iso

eng

DOI

http://dx.doi.org/10.11575/PRISM/25660

OAI identifier oai:identifier

oai:ucalgary.scholaris.ca:11023/2576