Estimating the reliability and validity of concept mapping as a tool to assess prior knowledge

An initial activity in the instructional design process is an assessment of learner characteristics such as prior knowledge. Although this step is important for the instructional design process, Reiss and Morley (1993) found that only about half of the practicing instructional technologists they surveyed always performed some kind of learner analysis. The reasons for not conducting a learner analysis included lack of time and lack of knowledge about how to do such an analysis. Concept mapping has been proposed as a convenient tool for assessing learners’ prior knowledge (Jonassen, Beissner, & Yacci, 1993; Novak & Gowin, 1993). It has been said that a concept mapping procedure can be quickly developed and then conveniently administered to learners (Jonassen et al., 1993; Novak & Gowin, 1993). However, there is no evidence supporting the reliability or validity of concept mapping as a way to measure learners’ prior knowledge. The purpose of this research was to investigate the reliability and validity of concept mapping as a tool to assess learners’ prior knowledge. Students in a graduate-level statistics class and an undergraduate level engineering class were administered concept mapping and multiple-choice tests before and after instruction. The interrator reliability, internal consistency, content validity, face validity, concurrent validity, and construct validity of the concept mapping procedure were then examined. Overall, the data do not support the reliability and validity of the concept mapping procedure used in this study. However, the reliability and validity levels achieved by the concept mapping test were comparable to the multiple-choice test developed by the classroom instructor. It may be that concept mapping is no less reliable and valid than other forms of classroom assessments. Nonetheless, the concept mapping procedure used in this study can not be recommend as a practical measure of students’ prior knowledge. In addition to the poor reliability and validity, the concept mapping procedure is difficult to learn and the scoring procedure is difficult to use. In the end, the user will have to weigh the results of this study and judge the adequacy of concept mapping for its intended purpose.

Name thesis:degree_name

Ph. D.

Level thesis:degree_level

doctoral

Discipline thesis:degree_discipline

Instructional Technology and Design

Department dc:contributor.department

Instructional Technology and Design

Grantor dc:publisher

Virginia Tech

Year dc:date.issued

1994

Author dc:creator

• Coleman, Susan Lee

Chair dc:contributor.committeechair

• Sherman, Thomas M.

Committee members dc:contributor.committeemember

• Burton, John K.
• Koball, Elizabeth H.
• Murawski, Marcia N.
• Moore, David M.

Statement dc:rights

• In Copyright

Licence dc:rights.uri

• http://rightsstatements.org/vocab/InC/1.0/

Language dc:language.iso

en

Dc Identifier Other

etd-06062008-164956

OAI identifier oai:identifier

oai:vtechworks.lib.vt.edu:10919/38314