Tailoring large language models for zero-shot relation extraction

Relation extraction (RE) aims to identify semantic relationships between entities within text. Despite considerable advancements, existing models predominantly require extensive annotated training data, which is both costly and labor-intensive to obtain. Moreover, these models often struggle to adapt to novel or unseen relations. Few-shot learning, which aims to reduce annotation demands, typically provides incomplete and biased supervision for target relations, resulting in degraded and unstable performance. To accurately and explicitly describe relation semantics while minimizing the need for annotations, we explore the definition only zero-shot RE setting, in which only relation definitions expressed in natural language are used to train an RE model. We introduce REPaL, a framework comprising three stages: (1) We leverage large language models (LLMs) to generate initial seed instances from relation definitions and an unlabeled corpus. (2) We then fine-tune a bidirectional small language model (SLM) using these initial seeds to learn relational patterns specific to the target domain. (3) To expand pattern coverage and mitigate biases introduced by the initial seeds, we incorporate feedback derived from the SLM's predictions on the unlabeled corpus and the historical synthesis data. To accomplish this, we leverage the multi-turn conversational ability of LLMs to generate additional instances in follow-up dialogues informed by both the feedback and synthesis history. Our studies reveal that definition-oriented seed synthesis effectively enhances pattern coverage, whereas indiscriminately increasing the number of seeds leads to performance saturation. Experiments on two datasets demonstrate that REPaL significantly improves cost-effective zero-shot performance by substantial margins.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois Urbana-Champaign

Year dc:date

2025

Author dc:creator

• Zhou, Sizhe

Contributors dc:contributor

• Han, Jiawei

• relation extraction
• large language models
• small language models

Statement dc:rights

• Copyright 2025 Sizhe Zhou

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/129194

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F129194