Supporting Human Performance with Post-hoc Explanations in Automated Decision Assistance

One contemporary challenge in automation design is determining the type of information that automated decision aids should provide to users to promote appropriate reliance behaviors. Post-hoc explanations have emerged as a strategy to support appropriate reliance on automated decision aids based on machine learning. However, existing methods to generate post-hoc explanations often fail to demonstrate systematic effectiveness in aiding human performance. In addition, past studies offer limited guidance on what explanations are suitable for specific applications.This dissertation presents two controlled experiments on the effects of model-agnostic explanations on human performance in an industrial application where effective and efficient detection of system failure is critical to ensure operational continuity. The experiments tested different combinations of example-based normative, contrastive, and counterfactual explanations. Results from the first experiment suggested that normative explanations reduced decision time and workload, and the addition of contrastive explanations to normative explanations also supported effective reliance. Further analysis revealed a lack of significant performance differences between participants with lower and higher data literacy. The second experiment explored the suitability of including counterfactuals in post-hoc explanations as compared to normative or generic contrastive explanations. The conditions included one baseline with no explanations, one with normative plus contrastive explanations, one with normative plus counterfactual explanations, and one with all three types of explanations. The results suggested that the condition with all three explanations led to a reduction in false alarm rate, time, and workload compared to the baseline. This dissertation expands the literature on human-subjects evaluations in explainable decision aids by providing empirical evidence on the influence of specific combinations of model-agnostic, example-based explanations on human performance. The findings can inform the design of explanation interfaces that support effective and efficient detection of failures in safety-critical systems.

Department dc:contributor.department

Mechanical and Industrial Engineering

Year dc:date.issued

2024

Author dc:creator

• Gentile, Davide

Advisors dc:contributor.advisor

• Jamieson, Greg A
• Donmez, Birsen

• Automation reliance behavior
• Contrastive explanations
• Counterfactual explanations
• Explainable AI
• Human factors
• Normative explanations

Handle dc:identifier.uri

http://hdl.handle.net/1807/140857

OAI identifier oai:identifier

oai:utoronto.scholaris.ca:1807/140857