Evaluation of Biases That Challenge the Implementation and Use of Machine Learning Clinical Decision Support Tools

Clinical machine learning (ML) tools hold promise to improve patient care, though their effective implementation requires that several barriers be overcome. This thesis includes three projects designed to further understand these barriers and explore potential solutions. The first study investigated the presence and causes of sociodemographic bias in clinical ML algorithms. It found that algorithmic bias evaluations are infrequently performed, that algorithmic bias is often present, and that addressing algorithmic bias requires an approach tailored to the specific ML model and population. The second study evaluated for algorithmic bias across the development and implementation of a clinical ML tool that has been deployed into practice. This study is among the first such evaluations and identified an absence of algorithmic bias in model performance, clinical processes of care and patient outcomes for the CHARTwatch ML early warning system. This study also identified an opportunity for ML tools to be used to address pre-existing healthcare inequities, as well as outlining a roadmap for future ML implementation bias assessments. The third study evaluated how the clinical use of an ML algorithm can affect its performance over time through contamination bias. This study identified how key model parameters influence the magnitude of model performance change and explored methods to prevent model deterioration. These studies together provide strategies to identify and overcome key challenges in implementing clinical ML tools.

Department dc:contributor.department

Dalla Lana School of Public Health

Year dc:date.issued

2025

Author dc:creator

• Colacci, Michael

Advisor dc:contributor.advisor

• Mamdani, Muhammad

• Algorithmic Bias
• Bias
• Clinical Decision Support
• Clinical Medicine
• Internal Medicine
• Machine Learning

Statement dc:rights

• Attribution 4.0 International

Licence dc:rights.uri

• http://creativecommons.org/licenses/by/4.0/

Handle dc:identifier.uri

https://hdl.handle.net/1807/145041

OAI identifier oai:identifier

oai:utoronto.scholaris.ca:1807/145041