Guarding truthfulness: Detecting and correcting false information

The proliferation of false information in today’s digital age is becoming a global epidemic, leading to societal instability and discord. Large language models, such as ChatGPT, have unintentionally escalated the problem by occasionally generating misleading content, promoting harmful views, and eroding faith in public conversation. Based on these realities, this thesis sets out to establish effective strategies to counter and curtail the spread of false information. We approach the mitigation of false news from two distinct angles: detection and correction. Robustly detecting false information is a stepping stone towards combating false information and preventing the further spread of inaccurate content. We have contributed to the field by proposing two approaches that address two important challenges: the gap between machine-generated fake news and human-written fake news, and the scarcity of fact-checking annotated data in low-resource languages. First, we generate more realistic fake news by explicitly incorporating propaganda techniques and adopting self-critical sequence training to instruct the generator not to produce factual content. We show that detectors trained on our generated data are more effective in identifying human-written fake news. Second, for the challenge of data scarcity, we leverage cross-lingual retrieval techniques to identify relevant passages across languages by learning to retrieve from different languages via pseudo-feedback, thereby enabling cross-lingual fact-checking. Our method outperforms existing models and alleviates the data scarcity issues. Upon identifying false information, the intuitive next step is to automatically correct the information based on the supporting or refuting evidence retrieved. To this end, we devised an interpretable factual error correction framework, which remediates factual errors in a zero-shot manner. This framework, inspired by how humans correct errors, asks questions about the input claim, looks for answers in the evidence, and reconstructs a factual statement. By breaking this task into several sub-tasks that have much richer resources, we eliminate the need for training data in factual error correction. More importantly, the decomposability of our framework offers natural interpretability, as the questions and answers generated explicitly illuminate which parts of the original claim were incorrect and why. We demonstrate that our approach outperforms fully-supervised approaches that have been fine-tuned. Additionally, we extended the scope of factual error correction to visual data, focusing on chart understanding. Recognizing the inherent complexity of interpreting graphical information, we proposed a methodology for converting charts into tables for enhanced error correction. Furthermore, we introduced the Multi-Agent Debate Refinement (MADR) framework to tackle unfaithfulness in explanations generated by large language models for fact-checking. This iterative, multi-agent strategy significantly enhanced the faithfulness and reliability of these explanations.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2024

Author dc:creator

• Huang, Kung-Hsiang

Contributors dc:contributor

• Ji, Heng
• Zhai, ChengXiang
• McKeown, Kathleen
• Peng, Hao
• Zhao, Han
• Joty, Shafiq

• Fact-checking
• Factual Inconsistency
• Fake News Detection
• Factual Error Correction

Statement dc:rights

• Copyright 2024 Kung-Hsiang Huang

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/125546

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F125546