Data Augmentation and Conformal Prediction

Conformal prediction is a popular line of research in uncertainty quantification. Conformal predictors output sets of predictions accompanied by a guarantee that the set contains the true label. Conformal prediction is particularly promising because it makes no distributional assumptions and requires only a black-box classifier to produce sets with this type of guarantee. Unfortunately, existing conformal predictions can produce uninformatively large prediction sets for certain examples, which limits their applications to real-world contexts. In this thesis, we explore the impact of data augmentation, a popular computer vision technique, on the performance of conformal predictors. In particular, we present multiple ways of combining data augmentation with conformal prediction by introducing five methods of test-time-augmentation-enhanced conformal prediction (TTA-CP). We find that certain TTA-CP methods can improve upon the size and stability of prediction sets created by traditional conformal prediction. Using ImageNet and Fitzpatrick 17k, two datasets differing in size, complexity, and balance, we reveal dataset-dependent decisions that are key to improving performance in conformal prediction.

Name thesis:degree_name

Master

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2023

Author dc:creator

• Lu, Helen

Advisors dc:contributor.advisor

• Guttag, John
• Shanmugam, Divya

Statement dc:rights

• In Copyright - Educational Use Permitted
• Copyright retained by author(s)

Licence dc:rights.uri

• https://rightsstatements.org/page/InC-EDU/1.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/151275

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/151275