Response Criterion Placement Modulates the Effects of Graded Alerting Systems on Human Performance and Learning in a Target Detection Task

Human operators can perform better with the use of an automated diagnostic aid than without the use of an aid in a signal detection task. This experiment aimed to determine whether any differences existed among graded aids—automated diagnostic aids that use a scale of confidence levels reflecting a spectrum of probabilistic information or uncertainty when making a judgment—that enabled better human detection performance, and either binary or graded aid produced better learning. Participants performed a visual search framed as a medical decision making task. Stimuli were arrays of random polygons (“cells”) generated by distorting a prototype shape. The target was a shape more strongly distorted than the accompanying distracters. A target was present on half of the trials. Each participant performed the task with the assistance of either a binary aid, one of three graded aids, or no aid. The aids’ sensitivities were the same (d′ = 2); the difference between the aids lay in the placement of their decision criteria, which determines a tradeoff between the aid’s predictive value and the frequency with which it makes a diagnosis. The graded aid with 90% reliability provided a judgment on the greatest number of trials, the graded aid with 94% reliability gave a judgment on fewer trials, and the third graded aid with 96% reliability gave a judgment on the least number of trials. The binary aid with 84% reliability gave a judgment on each trial. All aids improved human detection performance, though the graded aids trended towards improving performance more than the binary aid. The binary and graded aids did not produce significantly better or worse learning than did unaided performance. The binary and graded aids did not significantly help learning, but they certainly did not worsen human detection performance when compared to the unaided condition. These results imply that the decision boundaries of a graded alert might be fixed to encourage appropriate reliance on the aid and improve human detection performance, and indicate employing either a graded or binary automated aid may be beneficial to learning in a detection task.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Human Factors

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2011

Author dc:creator

• Swanson, Leah A.

Contributors dc:contributor

• McCarley, Jason S.

• Automated alerting systems
• Automation reliance
• Graded alerts
• Automation in a signal detection task

Statement dc:rights

• Copyright 2010 Leah Ann Swanson

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/18377

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/18377