The making of impact: Causal validity and epistemic justice in international development impact evaluation

This dissertation investigates the manifestation and negotiation of causal validity and epistemic justice in international development impact evaluation (IDIE), with a specific focus on agricultural interventions. Drawing on Ernest House's (1980) assertion that evaluations should be not only true but also just, and Donald Campbell's (1986) reconceptualization of internal validity as "local molar causal validity," the study examines how evaluators balance methodological rigor with inclusive knowledge production. Through narrative-reflective interviews with nine evaluators representing diverse methodological perspectives, the research employs Eyal's (2013) "expertise as network" as well as research on evaluation's concepts of validity as a conceptual framework, exploring how causation and justice are part of an interconnected network of agents, concepts, devices, and institutional arrangements affect impact determination. Findings reveal multiple causality conceptions in practice, with evaluators acknowledging the counterfactual as necessary but insufficient for robust causal claims, aligning with Campbell's nuanced view of validity. Analysis utilizing Byskov's (2021) conditions for epistemic injustice and House and Howe's (2000) Deliberative Democratic Evaluation framework demonstrates that evaluators employ various strategies to promote epistemic justice while negotiating practical constraints. The study presents a four-quadrant matrix delineating elements that: (1) inhibit epistemic justice while promoting causal validity; (2) inhibit causal validity while promoting epistemic justice; (3) inhibit both; and (4) promote both. This framework illuminates potential areas of methodological and disciplinary convergence between evaluation approaches, highlighting contextual sensitivity, inclusive research practices, and methodological flexibility as practices that simultaneously advance both the causal validity championed by Campbell and the justice-oriented validity advocated by House. The findings have significant implications for evaluation practice, especially amid the structural transformations occurring within international development organizations.

Name thesis:degree_name

Doctor of Philosophy

Level thesis:degree_level

doctoral

Discipline thesis:degree_discipline

Agricultural and Extension Education

Department dc:contributor.department

Agricultural, Leadership, and Community Education

Grantor dc:publisher

Virginia Tech

Year dc:date.issued

2025

Author dc:creator

• Travis, Elli Madeleine

Chair dc:contributor.committeechair

• Archibald, Thomas Greig

Committee members dc:contributor.committeemember

• Mark, Melvin Michael
• Breslau, Daniel
• Niewolny, Kimberly Lee

• impact
• international development
• justice
• causality
• methodological pluralism

Statement dc:rights

• Creative Commons Attribution 4.0 International

Licence dc:rights.uri

• http://creativecommons.org/licenses/by/4.0/

Language dc:language.iso

en

Dc Identifier Other

vt_gsexam:43097

OAI identifier oai:identifier

oai:vtechworks.lib.vt.edu:10919/135531