Uncertainty quantification in machine learning with Bayesian models

Uncertainty quantification plays a vital role to the adoption of modern machine learning methods in real-world applications by improving the trustworthiness and reliability of complex models. In the following chapters, we develop methods in uncertainty quantification that address several major areas of current research. The first two chapters focus on developing novel recalibration methods that can be applied to pre-trained models to improve their probabilistic predictions. In the classification setting, we extend the standard temperature scaling method by identifying one of its main characteristics of always increasing the uncertainty of the prediction and developing a method that enforces this property. In the regression setting, we introduce an optimization framework for optimization for which we can recover the well-known quantile recalibration method, and we use this framework to propose a novel method. In the third chapter, we propose a novel epistemic uncertainty quantification method and show that it faithfully targets the formal definition of epistemic uncertainty in terms of accuracy gain. All of our methods are designed with Bayesian methods in mind; the methods of Chapters 3 and 4 are specifically used with Bayesian models, and the method of Chapter 2 can be extended to Bayesian models.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Statistics

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2024

Author dc:creator

• Qian, Christopher

Contributors dc:contributor

• Liang, Feng
• Li, Bo
• Simpson, Douglas
• Adams, Jason

• Recalibration
• Epistemic Uncertainty
• Dropout

Statement dc:rights

• Copyright 2024 Christopher Qian

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/125761

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F125761