Evaluating Fairness of Artificial Intelligence Models for Radiology Image Classification

With the increasing prevalence of AI-assisted decision-making in the healthcare domain, evaluating fairness of machine learning models is more central than ever. Measuring the fairness of medical decision-support systems has enormous impacts on patients of different backgrounds and can influence how clinicians make decisions. In this study, we conduct a fairness analysis on the top 8-10 performing machine learning and artificial intelligence models from the Radiological Society of North America cervical spine fracture detection challenge and abdominal trauma detection challenge. Seven metrics are used for a more comprehensive assessment on fairness. Our findings indicate that cervical spine fracture detection models exhibit overall fairness, while abdominal trauma detection models demonstrate some unfairness in specific injury regions, possibly due to limited sample size. We also explore the performance of top models from the intracranial hemorrhage detection challenge across clinician-labeled "easy," "medium," and "hard" cases, revealing a lower accuracy rate on hard cases. This study underscores the need for additional model testing and comprehensive data representation to ensure fairness before real-world deployment in healthcare systems.

Name thesis:degree_name

Master

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2024

Author dc:creator

• Sandadi, Varsha

Advisor dc:contributor.advisor

• Ghassemi, Marzyeh

Statement dc:rights

• In Copyright - Educational Use Permitted
• Copyright retained by author(s)

Licence dc:rights.uri

• https://rightsstatements.org/page/InC-EDU/1.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/156974

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/156974