Data visualization (data viz) is a valuable tool within evaluation for its ability to aid cognitive efficiency over text-based presentation of data (Card et. al., 1999; Evergreen, 2017, 2018; Few, 2012; Nussbaumer Knaflic, 2015; Tufte 2001). This exploratory multi-phase mixed methods research study considers purposes for using data viz in evaluation that can be achieved with that increased efficiency through the research question: “What conceptualizations of data viz use do program evaluators have beyond increased efficiency?”
In Phase I, secondary analysis of existing interview data with experts in both data viz and evaluation was used to better understand conceptualizations of data viz and their prevalence in evaluation. Support within evaluation was established for a Utilization-Focused Evaluation Framework (U-FE) (Patton & Campbell-Patton, 2022) conceptualization of data viz; an explain-explore model that considers data visualization from the perspective of who is having the experience along a continuum between explain and explore (Evergreen & Metzner, 2013; Kirk, 2019), and a model on a continuum from data to insight familiar to the fields of computer science and cognitive science that converts data into information, knowledge, understanding, sense-making, and/or insight (Chen, 2009).
A model of using data viz for stakeholder or audience engagement and to extend evaluation use emerged and was further described using follow-up interviews in Phase II. This audience engagement model resembles the information processing model of memory (Huang et al., 2009) from an evaluation perspective. In this model, attracting and holding audience attention is intended to lead to connection (interaction) and memory (learning), which in turn leads to evaluation use. The model also considers evaluation specific conceptualizations of the role(s) that brand identity, data viz design principles, artifacts, capacity building, professionalism, credibility, satisfaction, and confidence may play in audience engagement and evaluation use.
In Phase III, three of these conceptual frameworks (explain-explore, data to insight, and audience engagement) were presented as brief explainer videos to a sample of 131 evaluators who are members of the American Evaluation Association to determine their familiarity with and perceived usefulness of the models. Findings suggest that there is significant conceptual overlap between the models. All three models are complementary, appropriate in evaluation, add value to the efficiency rationale of data viz, make sense to evaluators, and are considered useful in evaluation. Each has the potential to benefit evaluators as they consider why they should use data visualization in their work and evaluators provided many examples of using each model in their work.
This research supported that program evaluators usually (38.2%) or always (42.7%) use data viz in their evaluation work, accept the efficiency rationale, and are interested in other reasons for using data viz beyond efficiency. While data viz is a time-consuming skill, providing evaluators with conceptualizations of data viz beyond efficiency may make them more willing to expend the time and effort needed to apply data viz to their evaluation work. Participant interest in both the content and the medium (e.g., brief explainer videos) suggests that there is interest, need, and desire for more professional development in data visualization and associated skills. Beyond skills workshops and “how to” guides, findings suggest a desire for more learning opportunities about abstract concepts, which offers new opportunities for teaching experiences and professional development opportunities within the profession.
Overall, findings suggest that the explain-explore model is a simple framework that an evaluator can use to consider the purpose of a particular visual before beginning to design and the data to insight model is a linear description of how to get the most information and insight out of a particular data viz. The audience engagement model is a holistic approach to thinking through the relationships in the evaluation to support evaluation use. While there is no clear hierarchy of models suggested in this study, comments supported that the audience engagement model is the most specific to evaluation – to the extent that it might not even be data viz specific.
","abstract_html":"<p>Data visualization (data viz) is a valuable tool within evaluation for its ability to aid cognitive efficiency over text-based presentation of data (Card et. al., 1999; Evergreen, 2017, 2018; Few, 2012; Nussbaumer Knaflic, 2015; Tufte 2001). This exploratory multi-phase mixed methods research study considers purposes for using data viz in evaluation that can be achieved with that increased efficiency through the research question: “What conceptualizations of data viz use do program evaluators have beyond increased efficiency?”</p> <p>In Phase I, secondary analysis of existing interview data with experts in both data viz and evaluation was used to better understand conceptualizations of data viz and their prevalence in evaluation. Support within evaluation was established for a <em>Utilization-Focused Evaluation Framework</em> (<em>U-FE</em>) (Patton & Campbell-Patton, 2022) conceptualization of data viz; an explain-explore model that considers data visualization from the perspective of who is having the experience along a continuum between explain and explore (Evergreen & Metzner, 2013; Kirk, 2019), and a model on a continuum from data to insight familiar to the fields of computer science and cognitive science that converts data into <em>information, knowledge, understanding, sense-making,</em> and/or<em> insight</em> (Chen, 2009).</p> <p>A model of using data viz for stakeholder or audience engagement and to extend evaluation use emerged and was further described using follow-up interviews in Phase II. This <em>audience engagement</em> model resembles the information processing model of memory (Huang et al., 2009) from an evaluation perspective. In this model, attracting and holding audience attention is intended to lead to connection (interaction) and memory (learning), which in turn leads to evaluation use. The model also considers evaluation specific conceptualizations of the role(s) that brand identity, data viz design principles, artifacts, capacity building, professionalism, credibility, satisfaction, and confidence may play in audience engagement and evaluation use.</p> <p>In Phase III, three of these conceptual frameworks (<em>explain-explore</em>, <em>data to insight,</em> and<em> audience engagement</em>) were presented as brief explainer videos to a sample of 131 evaluators who are members of the American Evaluation Association to determine their familiarity with and perceived usefulness of the models. Findings suggest that there is significant conceptual overlap between the models. All three models are complementary, appropriate in evaluation, add value to the <em>efficiency rationale</em> of data viz, make sense to evaluators, and are considered useful in evaluation. Each has the potential to benefit evaluators as they consider <em>why they</em> should <em>use </em>data visualization in their work and evaluators provided many examples of using each model in their work.</p> <p>This research supported that program evaluators <em>usually </em>(38.2%) or <em>always</em> (42.7%) use data viz in their evaluation work, accept the <em>efficiency </em>rationale, and are interested in other reasons for using data viz beyond <em>efficiency</em>. While data viz is a time-consuming skill, providing evaluators with conceptualizations of data viz beyond efficiency may make them more willing to expend the time and effort needed to apply data viz to their evaluation work. Participant interest in both the content and the medium (e.g., brief explainer videos) suggests that there is interest, need, and desire for more professional development in data visualization and associated skills. Beyond skills workshops and “how to” guides, findings suggest a desire for more learning opportunities about abstract concepts, which offers new opportunities for teaching experiences and professional development opportunities within the profession.</p> <p>Overall, findings suggest that the <em>explain-explore </em>model is a simple framework that an evaluator can use to consider the purpose of a particular visual before beginning to design and the <em>data to insight </em>model is a linear description of how to get the most information and insight out of a particular data viz. The <em>audience engagement </em>model is a holistic approach to thinking through the relationships in the evaluation to support evaluation use. While there is no clear hierarchy of models suggested in this study, comments supported that the <em>audience engagement </em>model is the most specific to evaluation – to the extent that it might not even be data viz specific.</p>","abstract_has_math":false,"creators":["Douville, Sarah"],"institution":null,"degree_name":"Psychology, PhD","degree_level":"Open Access Dissertation","degree_discipline":null,"degree_department":null,"school":null,"contributors":["Stewart Donaldson","David Fetterman","Thomas Archi"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024-01-01T08:00:00Z","date_published":"2024-01-01T08:00:00Z","updated_at":"2026-07-24T01:40:49Z","subjects":["conceptual model","data visualization","evaluation","evaluation use","Educational Assessment, Evaluation, and Research","Psychology"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://scholarship.claremont.edu/cgu_etd/865","outbound_label":"Repository record","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Stewart Donaldson","David Fetterman","Thomas Archi"]},{"key":"dc:creator","label":"Author","values":["Douville, Sarah"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.available","label":"Dc Date Available","values":["2024-11-14T08:00:00Z"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Open Access Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Psychology, PhD"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["conceptual model","data visualization","evaluation","evaluation use","Educational Assessment, Evaluation, and Research","Psychology"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://scholarship.claremont.edu/cgu_etd/865"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Data visualization (data viz) is a valuable tool within evaluation for its ability to aid cognitive efficiency over text-based presentation of data (Card et. al., 1999; Evergreen, 2017, 2018; Few, 2012; Nussbaumer Knaflic, 2015; Tufte 2001). This exploratory multi-phase mixed methods research study considers purposes for using data viz in evaluation that can be achieved with that increased efficiency through the research question: “What conceptualizations of data viz use do program evaluators have beyond increased efficiency?”
In Phase I, secondary analysis of existing interview data with experts in both data viz and evaluation was used to better understand conceptualizations of data viz and their prevalence in evaluation. Support within evaluation was established for a Utilization-Focused Evaluation Framework (U-FE) (Patton & Campbell-Patton, 2022) conceptualization of data viz; an explain-explore model that considers data visualization from the perspective of who is having the experience along a continuum between explain and explore (Evergreen & Metzner, 2013; Kirk, 2019), and a model on a continuum from data to insight familiar to the fields of computer science and cognitive science that converts data into information, knowledge, understanding, sense-making, and/or insight (Chen, 2009).
A model of using data viz for stakeholder or audience engagement and to extend evaluation use emerged and was further described using follow-up interviews in Phase II. This audience engagement model resembles the information processing model of memory (Huang et al., 2009) from an evaluation perspective. In this model, attracting and holding audience attention is intended to lead to connection (interaction) and memory (learning), which in turn leads to evaluation use. The model also considers evaluation specific conceptualizations of the role(s) that brand identity, data viz design principles, artifacts, capacity building, professionalism, credibility, satisfaction, and confidence may play in audience engagement and evaluation use.
In Phase III, three of these conceptual frameworks (explain-explore, data to insight, and audience engagement) were presented as brief explainer videos to a sample of 131 evaluators who are members of the American Evaluation Association to determine their familiarity with and perceived usefulness of the models. Findings suggest that there is significant conceptual overlap between the models. All three models are complementary, appropriate in evaluation, add value to the efficiency rationale of data viz, make sense to evaluators, and are considered useful in evaluation. Each has the potential to benefit evaluators as they consider why they should use data visualization in their work and evaluators provided many examples of using each model in their work.
This research supported that program evaluators usually (38.2%) or always (42.7%) use data viz in their evaluation work, accept the efficiency rationale, and are interested in other reasons for using data viz beyond efficiency. While data viz is a time-consuming skill, providing evaluators with conceptualizations of data viz beyond efficiency may make them more willing to expend the time and effort needed to apply data viz to their evaluation work. Participant interest in both the content and the medium (e.g., brief explainer videos) suggests that there is interest, need, and desire for more professional development in data visualization and associated skills. Beyond skills workshops and “how to” guides, findings suggest a desire for more learning opportunities about abstract concepts, which offers new opportunities for teaching experiences and professional development opportunities within the profession.
Overall, findings suggest that the explain-explore model is a simple framework that an evaluator can use to consider the purpose of a particular visual before beginning to design and the data to insight model is a linear description of how to get the most information and insight out of a particular data viz. The audience engagement model is a holistic approach to thinking through the relationships in the evaluation to support evaluation use. While there is no clear hierarchy of models suggested in this study, comments supported that the audience engagement model is the most specific to evaluation – to the extent that it might not even be data viz specific.
"]},{"key":"dc:title","label":"Title","values":["Conceptualizations of Data Visualization Use Beyond Efficiency in Evaluation"]}]}],"canonical_facts":{"dc:contributor":["Stewart Donaldson","David Fetterman","Thomas Archi"],"dc:creator":["Douville, Sarah"],"dc:date.available":["2024-11-14T08:00:00Z"],"dc:description.abstract":["Data visualization (data viz) is a valuable tool within evaluation for its ability to aid cognitive efficiency over text-based presentation of data (Card et. al., 1999; Evergreen, 2017, 2018; Few, 2012; Nussbaumer Knaflic, 2015; Tufte 2001). This exploratory multi-phase mixed methods research study considers purposes for using data viz in evaluation that can be achieved with that increased efficiency through the research question: “What conceptualizations of data viz use do program evaluators have beyond increased efficiency?”
In Phase I, secondary analysis of existing interview data with experts in both data viz and evaluation was used to better understand conceptualizations of data viz and their prevalence in evaluation. Support within evaluation was established for a Utilization-Focused Evaluation Framework (U-FE) (Patton & Campbell-Patton, 2022) conceptualization of data viz; an explain-explore model that considers data visualization from the perspective of who is having the experience along a continuum between explain and explore (Evergreen & Metzner, 2013; Kirk, 2019), and a model on a continuum from data to insight familiar to the fields of computer science and cognitive science that converts data into information, knowledge, understanding, sense-making, and/or insight (Chen, 2009).
A model of using data viz for stakeholder or audience engagement and to extend evaluation use emerged and was further described using follow-up interviews in Phase II. This audience engagement model resembles the information processing model of memory (Huang et al., 2009) from an evaluation perspective. In this model, attracting and holding audience attention is intended to lead to connection (interaction) and memory (learning), which in turn leads to evaluation use. The model also considers evaluation specific conceptualizations of the role(s) that brand identity, data viz design principles, artifacts, capacity building, professionalism, credibility, satisfaction, and confidence may play in audience engagement and evaluation use.
In Phase III, three of these conceptual frameworks (explain-explore, data to insight, and audience engagement) were presented as brief explainer videos to a sample of 131 evaluators who are members of the American Evaluation Association to determine their familiarity with and perceived usefulness of the models. Findings suggest that there is significant conceptual overlap between the models. All three models are complementary, appropriate in evaluation, add value to the efficiency rationale of data viz, make sense to evaluators, and are considered useful in evaluation. Each has the potential to benefit evaluators as they consider why they should use data visualization in their work and evaluators provided many examples of using each model in their work.
This research supported that program evaluators usually (38.2%) or always (42.7%) use data viz in their evaluation work, accept the efficiency rationale, and are interested in other reasons for using data viz beyond efficiency. While data viz is a time-consuming skill, providing evaluators with conceptualizations of data viz beyond efficiency may make them more willing to expend the time and effort needed to apply data viz to their evaluation work. Participant interest in both the content and the medium (e.g., brief explainer videos) suggests that there is interest, need, and desire for more professional development in data visualization and associated skills. Beyond skills workshops and “how to” guides, findings suggest a desire for more learning opportunities about abstract concepts, which offers new opportunities for teaching experiences and professional development opportunities within the profession.
Overall, findings suggest that the explain-explore model is a simple framework that an evaluator can use to consider the purpose of a particular visual before beginning to design and the data to insight model is a linear description of how to get the most information and insight out of a particular data viz. The audience engagement model is a holistic approach to thinking through the relationships in the evaluation to support evaluation use. While there is no clear hierarchy of models suggested in this study, comments supported that the audience engagement model is the most specific to evaluation – to the extent that it might not even be data viz specific.
"],"dc:identifier":["https://scholarship.claremont.edu/cgu_etd/865"],"dc:subject":["conceptual model","data visualization","evaluation","evaluation use","Educational Assessment, Evaluation, and Research","Psychology"],"dc:title":["Conceptualizations of Data Visualization Use Beyond Efficiency in Evaluation"],"thesis:degree_level":["Open Access Dissertation"],"thesis:degree_name":["Psychology, PhD"]},"updated_at":"2026-07-24T01:40:49Z"}