Abstract
dc:description.abstractProbability predictions are essential to inform decision making across many fields. Ideally, probability predictions are (i) well calibrated, (ii) accurate, and (iii) bold, i.e., spread out enough to be informative for decision making. However, there is a fundamental tension between calibration and boldness, since calibration metrics can be high when predictions are overly cautious, i.e., non-bold. The purpose of this work is to develop a Bayesian model selection-based approach to assess calibration, and a strategy for boldness-recalibration that enables practitioners to responsibly embolden predictions subject to their required level of calibration. Specifically, we allow the user to pre-specify their desired posterior probability of calibration, then maximally embolden predictions subject to this constraint. We demonstrate the method with a case study on hockey home team win probabilities and then verify the performance of our procedures via simulation. Additionally, we introduce BRcal, an R package implementing Boldness-Recalibration and supporting methodology. We reformulate boldness-recalibration as a nonlinear optimization of boldness with a nonlinear constraint on calibration, and describe how this is implemented in BRcal. The BRcal package is demonstrated using a case study on foreclosure prediction. Lastly, we extend the methods to account for underlying spatial association in the data and demonstrate via a case study on moose presence.
Degree
thesis:*- Name thesis:degree_name
- Doctor of Philosophy
- Level thesis:degree_level
- doctoral
- Discipline thesis:degree_discipline
- Statistics
- Department dc:contributor.department
- Statistics
- Grantor dc:publisher
- Virginia Tech
- Year dc:date.issued
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Guthrie, Adeline Pearl
- Chair dc:contributor.committeechair
-
- Franck, Christopher Thomas
- Committee members dc:contributor.committeemember
-
- Lattimer, Alan Martin
- Xing, Xin
- Van Mullekom, Jennifer Huffman
Subjects
dc:subject × 5Rights
dc:rights- Statement dc:rights
-
- In Copyright
- Licence dc:rights.uri
- Language dc:language.iso
- en
Identifiers
dc:identifier.*- Dc Identifier Other
- vt_gsexam:43561
- OAI identifier oai:identifier
- oai:vtechworks.lib.vt.edu:10919/133153