Efficient Uncertainty Quantification of Large Language Models

Large Language Models (LLMs) have demonstrated remarkable success in many applications; however, their reliability remains a critical concern, especially in high-stakes domains such as healthcare, finance, and law. Uncertainty Quantification (UQ) is essential for assessing LLM outputs and ensuring trust. However, existing UQ methods for LLMs face challenges: high computational costs, difficulties in handling unstructured outputs, and limited generalizability. This thesis addresses these challenges by proposing a systematic investigation into robust and efficient UQ methodologies tailored for LLMs. Specifically, this work focuses on: (1) analyzing probing methods to determine how hidden layers encode information relevant to uncertainty and accuracy, (2) developing novel UQ metrics that strongly correlate with actual model performance, and (3) designing computationally efficient pipelines to make UQ practical for real-world applications. By bridging these gaps, this research aims to establish UQ as a reliable tool for evaluating and improving the trustworthiness of LLM outputs, facilitating their safe and effective deployment in critical domains.

Name thesis:degree_name

Master

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2025

Author dc:creator

• Li, Angela

Advisor dc:contributor.advisor

• Ghassemi, Marzyeh

Statement dc:rights

• In Copyright - Educational Use Permitted
• Copyright retained by author(s)

Licence dc:rights.uri

• https://rightsstatements.org/page/InC-EDU/1.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/164851

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/164851