Eliciting Expert Uncertainty from Decision Making

Eliciting expert uncertainty is a complex task with many caveats. Its usefulness in high-level decision-making and Bayesian inference makes it a necessary task that must be completed accurately. This thesis explores uncertainty elicitation in terms of prior elicitation for Bayesian inference and the wider knowledge elicitation field. It describes methods currently available for prior elicitation and proposes a new typology, highlighting lesser-known methods. Based on the limitations of current methods, a new method for prior elicitation is introduced. This new method allows an analyst to model expert decision-making data to elicit a probability distribution that reflects expert uncertainty. An example of parole board decision-making is used to elicit a prior distribution of a prisoner re-offending upon release from prison. This example shows analysts how to elicit distributions for tabular data; however, more complex data types, such as images and reports, are often used for decision-making. To elicit distributions capturing expert uncertainty from more complex data, this thesis introduces a deep learning approach and uses an example of eliciting cancer risk, in histopathology, to illustrate this approach for the wider knowledge elicitation field.

Grantor dc:publisher.institution

The University of Waikato

Year dc:date.issued

2024

Author dc:creator

• Falconer, Julia Ruth

Advisors dc:contributor.advisor

• Joshi, Chaitanya
• Frank, Eibe
• Polaschek, Devon L. L.

• bayesian
• prior
• distribution
• uncertainty
• thesis with publication

Statement dc:rights

• All items in Research Commons are provided for private study and research purposes and are protected by copyright with all rights reserved unless otherwise indicated.

Licence

• https://researchcommons.waikato.ac.nz/bitstreams/393b7a9c-a040-4176-9016-b02f5535a431/download