Evidence-based verification and correction of textual claims

This thesis considers the task of fact-checking: predicting the veracity of claims made in written or spoken language using evidence. However, in previous task formulations, modelling assumptions ignore the requirement for systems to retrieve the necessary evidence. To better model how human fact-checkers operate, who first find evidence before labelling a claim's veracity, the methodology proposed in this thesis requires automated systems to retrieve evidence from a corpus to justify the veracity predictions made when modelling this task. The primary contribution of this thesis is the development and release of FEVER, a large-scale collection of human-written claims annotated with evidence from Wikipedia. Analysis of systems trained on this data highlights challenges in resolving ambiguity and context, as well as being resilient to imperfect evidence retrieval. To understand the limitations of models trained on datasets such as FEVER, contemporary fact verification systems are further evaluated using adversarial attacks -- instances constructed specifically to identify weaknesses and blind spots. However, as automated means for generating adversarial instances induce their own errors, this thesis proposes considering instances' correctness, allowing fairer comparison. The thesis subsequently considers how biases captured in these models can be mitigated with fine-tuning regularised with elastic weight consolidation. Finally, the thesis presents a new extension to the verification task: factual error correction. Rather than predicting the claim's veracity, systems must also generate a correction for the claim so that it is better supported by evidence, acting as another means to communicate the claim's veracity to an end-user. In contrast to previous work on explainable fact-checking, the method proposed in this chapter does not require additional data for supervision.

Name dc:type.qualificationname

Doctor of Philosophy (PhD)

Level dc:type.qualificationlevel

Doctoral

Grantor dc:publisher.institution

University of Cambridge

Year dc:date.issued

2021

Author dc:creator

• Thorne, James

Advisor dc:contributor.advisor

• Vlachos, Andreas

• natural language processing
• machine learning
• fact verification
• evidence
• fact-checking
• fact checking
• nlp

Licence

• https://www.rioxx.net/licenses/all-rights-reserved/

Language dc:language

eng

Author Identifier

0000-0003-2123-5071

OAI identifier oai:identifier

oai:www.repository.cam.ac.uk:1810/333449