Differential privacy in the era of generative AI: promises and challenges

Large language models (LLMs) are seeing rapid development and widespread deployment. As these models become increasingly capable and are deployed across diverse domains involving sensitive data, privacy concerns have intensified. Their inadvertently memorizing and leaking private information creates significant privacy risks when they are fine-tuned with user data or deployed as interactive agents. This thesis addresses the critical privacy challenges emerging in the era of generative AI, with a particular focus on protecting training data privacy in LLMs across various learning paradigms and application scenarios, as well as understanding what protection we actually offer. As a central tool, we leverage and scrutinize differential privacy (DP). Concretely, we develop a novel DP framework for language model alignment through preference tuning (RLHF), formalize new privacy definitions for multi-user training data scenarios, and critically examine DP-SGD—the workhorse algorithm for DP LLM training—and reveal an alarming variance in its empirical privacy protection. Together, these contributions advance both the practical applications and fundamental understanding of differential privacy in LLMs, providing researchers and practitioners with new tools and insights to navigate the landscape of privacy in generative AI.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois Urbana-Champaign

Year dc:date

2025

Author dc:creator

• Wu, Fan

Contributors dc:contributor

• Forsyth, David A.
• Chandrasekaran, Varun
• Forsyth, David A
• Wang, Gang
• Peng, Hao
• Kohno, Tadayoshi

• Differential Privacy
• Machine Learning
• Generative Ai
• Large Language Models

Statement dc:rights

• Copyright 2025 Fan Wu

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/129900

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F129900