A Culturally Responsive Evaluation Lens to Logic Model Design

<p>Culturally responsive evaluation (CRE) is an approach that centers all evaluation processes around the culture of a program’s secondary stakeholders. Specifically, this entails ensuring shared meanings in a group through communication. However, minimal connections have been made between CRE and logic model designs. Logic models commonly used by evaluators are data visualization and communication tools designed to aid in effectively communicating a program’s theory. Nevertheless, little is understood about the role culture plays in this process. This multiphase mixed methods study explored the integration of CRE to logic model designs using individualism and collectivism (IC) as a construct for culture and as a basis for tailoring the designs. Specifically, Phase 1 utilized an exploratory sequential design to develop the Logic Model Knowledge Test (LMKT) based upon expert evaluation insight and Amazon Mechanical Turk (MTurk). This addressed how learning from a logic model can be assessed. The LMKT was designed to assess the understanding of logic model components and the content of the program theory. It yielded good validity and reliability and was used in the final phase of the study. Phase 2 used a quantitative two-group comparison design to answer the question, which measures of IC would be more likely to help inform how people interpret logic models differently? Of three existing measures, Wagner (1995) was identified as a highly reliable measure that better distinguished IC-oriented people. Accordingly, it was used in the final phase of the study. Lastly, Phase 3’s mixed methods experimental design utilizing the LMKT and Wagner’s (1995) IC measure was implemented to help answer six questions regarding whether culturally tailored language, visualizations, or both impacted participants’ visual efficiency (VE) and perceived credibility of the program (CP). The findings suggest collectivistic MTurk participants had higher VE across seven culturally tailored logic model conditions. Collectivistic-oriented participants could better quickly and accurately answer the LMKT questions with less mental effort across all logic model conditions than individualistic-oriented participants. Yet, these conditions did not impact IC MTurk participants’ CP. Participants rated the program theory as credible across the logic model conditions regardless of their IC orientation. The findings provided partial empirical support for integrating CRE in logic model designs. Moreover, they added to the validity of the IC construct and Wagner’s (1995) measure. The findings also had valuable implications for evaluation practice through measures that could be used to better understand stakeholder cultural values aligned with communication styles and their knowledge of logic models. Finally, these same tools could be used for future research on evaluation concerning CRE and logic model designs.</p>

Name thesis:degree_name

Psychology, PhD

Level thesis:degree_level

Open Access Dissertation

Discipline thesis:degree_discipline

School of Social Science, Politics, and Evaluation

Year dc:date.available

2023

Author dc:creator

• Knight, Ciara Cascharelle

Contributors dc:contributor

• Wanda Casillas
• Stewart Donaldson

• Culturally responsive evaluation
• Data visualization
• Logic models
• Communication tools
• Logic Model Knowledge Test
• Biological Psychology

Repository record dc:identifier

https://scholarship.claremont.edu/cgu_etd/542

OAI identifier oai:identifier

oai:scholarship.claremont.edu:cgu_etd-1564