The Effect of Test Administration Modes in the Assessment of the Next Generation Science Standards

The Performance Expectations described in the Next Generation Science Standards (NGSS) pose significant challenges for assessment design. Multiple models of assessments, including paper-and-pencil and hands-on based performance assessments, can be utilized to assess student competency while applying scientific and engineering practices, in the context of disciplinary core ideas and crosscutting concepts. However, the unique features of different assessments may lead to test mode effects. Accordingly, there is an urgent need for empirical studies on the comparability and score equivalency of these different test modes. This study aims to develop a measurement instrument involving two test administration modes (i.e., a paper-and-pencil test and a hands-on based test), for measuring students' proficiencies in science, as expected in the NGSS (Achieve, 2013). Additionally, this study aims to validate the instrument measures, and to investigate the test administration mode effects, as well as interaction effects between trait and mode, gender and trait/mode . The instrument was developed through pilot-testing, followed by field-testing. The field testing took place in Jiangsu, China, and involved 2604 ninth graders from four Jiangsu-area junior high schools. Each participant completed both the paper-based test and the hands-on based test. The Many-Facet partial credit Rasch model was applied to the field test data, in order to establish evidence of the validity and reliability of the instrument measures, and to investigate the test administration mode effects and interaction effects. The results indicated that measures from this instrument are reasonably valid and reliable, and thus are appropriate for assessing students' competencies in science, reflecting the integration of the three dimensions envisioned by the National Research Council Framework and the NGSS. The findings also indicated that there are significant test administration mode effects and interaction effects between mode and trait, gender and mode/trait. Students' preference for a certain test mode, and their prior experience in taking hands-on based tests, can also significantly impact their hands-on based test performance.

Grantor dc:publisher

State University of New York at Buffalo

Year dc:date.issued

2018

Author dc:creator

• Chi, Shaohui; 0000-0002-1261-7951

Contributors dc:contributor

• Liu, Xiufeng
• Learning and Instruction

• science education

Statement dc:rights

• Users of works found in University at Buffalo Institutional Repository (UBIR) are responsible for identifying and contacting the copyright owner for permission to reuse. University at Buffalo Libraries do not manage rights for copyright-protected works and cannot assist with permissions.
• Copyright retained by author.

Language dc:language

eng

Handle dc:identifier

http://hdl.handle.net/10477/78052

OAI record oai:identifier

https://ubir.buffalo.edu/oai/request?verb=GetRecord&metadataPrefix=qdc&identifier=oai%3Aubir.buffalo.edu%3A10477%2F78052