Validity and Validation in Psychology

How well does a psychological test measure its target? This is the nagging question which motivates my thesis. If we define well-being in a certain way how do we know our test scores track this psychological attribute, and not something else? Within psychology the twin terms which are most often associated with measurement are ‘validity’ on the one hand and ‘validation’ on the other. Validity is roughly understood as good measurement, while validation denotes the processes or practices needed to secure this property. These twin concerns will be the anchor points of the thesis. My focus is on validity and validation and my aim is to improve them in order that we might better measure the attributes of psychology. I argue that at several important points psychologists fail to connect the test to the psychological attribute when conducting validation, and therefore even with validity supposedly secured, it is unclear what relationship has been established between test scores and the attribute. We first see this in Chapter 2, where I argue that psychologists often cash out validity in an operationalist manner, where it is the performance of one or more methods which defines validity, and nothing more. Simply performing validation methods without also paying close attention to the test’s relationship to the attribute means we risk marking a test valid without having a clear idea of the ways in which it tracks the attribute, if at all. In order to combat this problem I propose that any validity account should clearly tell us what relationship between attribute and test we should aim to secure. I propose my own definition too, and say that a test is valid to the degree in which test scores change with changes in the target attribute (test responsiveness) and change only with changes in this attribute (test discriminantness). This will also be my working definition of validity for the remainder of the thesis. In Chapters 3, 4, and 5 we dive into the weeds of validation methodology. I look at some of the tools and practices used to secure validity, and point to areas where the relationship between attribute and test has been neglected or confused. In Chapters 3 and 4 the focus is on the popular validation method of factor analysis. Firstly, in Chapter 3, I look at the importance of test items in test construction. When using factor analysis for this task we will at various points have to make decisions on whether to retain or delete items from a test, which will in turn affect its validity. I will argue that current advice on when to remove items risks creating tests which lack responsivity to the target attribute. In Chapter 4 I turn to the question of measurement equivalence across groups, namely the question of whether we can be justified in thinking that different studies measured the same attribute. One often recommended technique is to establish what is called ‘factorial invariance’, this being when the same factor model holds across different groups. I will argue that this recommendation is flawed, as factorial invariance re-examines the relationship between factor and items, rather than that between attribute and items. When measuring attributes therefore, factorial invariance is thus neither sufficient nor necessary to justify group comparison claims. Finally in Chapter 5 I turn to content validity, which focuses on whether test items are sufficiently relevant and representative of the intended attribute, as judged by a group of subject-matter experts. After proposing a way to clear up ambiguity in its definition, I will argue that evidence from content validity should be included throughout the test construction process, particularly when we use factor analysis. As it stands, current guidance fails to recommend this.

Name dc:type.qualificationname

Doctor of Philosophy (PhD)

Level dc:type.qualificationlevel

Doctoral

Grantor dc:publisher.institution

University of Cambridge

Year dc:date.issued

2024

Author dc:creator

• Holdsworth, Oliver

Advisor dc:contributor.advisor

• Alexandrova, Anna

• Validity
• Validation
• Psychology
• Measurement
• Factor Analysis
• Content Validity
• Factorial Invariance
• Philosophy of Psychology

Licence

• https://www.repository.cam.ac.uk/bitstreams/a67daf59-54ab-4177-97d2-8548d954200f/download
• http://purl.org/NET/rdflicense/allrightsreserved

Language dc:language

eng

DOI dc:identifier.doi

https://doi.org/10.17863/CAM.123421

OAI identifier oai:identifier

oai:www.repository.cam.ac.uk:1810/392833