{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/129194"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/129194","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Tailoring large language models for zero-shot relation extraction","abstract":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-10-19 without embargo terms","abstract_html":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-10-19 without embargo terms","abstract_has_math":false,"creators":["Zhou, Sizhe"],"institution":"University of Illinois Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Han, Jiawei"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025-04-14","date_published":"2025-04-14","updated_at":"2026-07-22T22:25:04Z","subjects":["relation extraction","large language models","small language models"],"languages":["en","eng"],"rights":["Copyright 2025 Sizhe Zhou"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/129194","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Han, Jiawei"]},{"key":"dc:creator","label":"Author","values":["Zhou, Sizhe"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2025-04-14","2025-05"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["relation extraction","large language models","small language models"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2025 Sizhe Zhou"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/129194"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-10-19 without embargo terms","The student, Sizhe Zhou, accepted the attached license on 2025-04-11 at 17:09.","The student, Sizhe Zhou, submitted this Thesis for approval on 2025-04-11 at 18:33.","This Thesis was approved for publication on 2025-04-14 at 13:03.","DSpace SAF Submission Ingestion Package generated from Vireo submission #21749 on 2025-10-19 at 18:09:19","Relation extraction (RE) aims to identify semantic relationships between entities within text. Despite considerable advancements, existing models predominantly require extensive annotated training data, which is both costly and labor-intensive to obtain. Moreover, these models often struggle to adapt to novel or unseen relations. Few-shot learning, which aims to reduce annotation demands, typically provides incomplete and biased supervision for target relations, resulting in degraded and unstable performance. To accurately and explicitly describe relation semantics while minimizing the need for annotations, we explore the definition only zero-shot RE setting, in which only relation definitions expressed in natural language are used to train an RE model. We introduce REPaL, a framework comprising three stages: (1) We leverage large language models (LLMs) to generate initial seed instances from relation definitions and an unlabeled corpus. (2) We then fine-tune a bidirectional small language model (SLM) using these initial seeds to learn relational patterns specific to the target domain. (3) To expand pattern coverage and mitigate biases introduced by the initial seeds, we incorporate feedback derived from the SLM's predictions on the unlabeled corpus and the historical synthesis data. To accomplish this, we leverage the multi-turn conversational ability of LLMs to generate additional instances in follow-up dialogues informed by both the feedback and synthesis history. Our studies reveal that definition-oriented seed synthesis effectively enhances pattern coverage, whereas indiscriminately increasing the number of seeds leads to performance saturation. Experiments on two datasets demonstrate that REPaL significantly improves cost-effective zero-shot performance by substantial margins."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Tailoring large language models for zero-shot relation extraction"]}]}],"canonical_facts":{"dc:contributor":["Han, Jiawei"],"dc:creator":["Zhou, Sizhe"],"dc:date":["2025-04-14","2025-05"],"dc:description":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-10-19 without embargo terms","The student, Sizhe Zhou, accepted the attached license on 2025-04-11 at 17:09.","The student, Sizhe Zhou, submitted this Thesis for approval on 2025-04-11 at 18:33.","This Thesis was approved for publication on 2025-04-14 at 13:03.","DSpace SAF Submission Ingestion Package generated from Vireo submission #21749 on 2025-10-19 at 18:09:19","Relation extraction (RE) aims to identify semantic relationships between entities within text. Despite considerable advancements, existing models predominantly require extensive annotated training data, which is both costly and labor-intensive to obtain. Moreover, these models often struggle to adapt to novel or unseen relations. Few-shot learning, which aims to reduce annotation demands, typically provides incomplete and biased supervision for target relations, resulting in degraded and unstable performance. To accurately and explicitly describe relation semantics while minimizing the need for annotations, we explore the definition only zero-shot RE setting, in which only relation definitions expressed in natural language are used to train an RE model. We introduce REPaL, a framework comprising three stages: (1) We leverage large language models (LLMs) to generate initial seed instances from relation definitions and an unlabeled corpus. (2) We then fine-tune a bidirectional small language model (SLM) using these initial seeds to learn relational patterns specific to the target domain. (3) To expand pattern coverage and mitigate biases introduced by the initial seeds, we incorporate feedback derived from the SLM's predictions on the unlabeled corpus and the historical synthesis data. To accomplish this, we leverage the multi-turn conversational ability of LLMs to generate additional instances in follow-up dialogues informed by both the feedback and synthesis history. Our studies reveal that definition-oriented seed synthesis effectively enhances pattern coverage, whereas indiscriminately increasing the number of seeds leads to performance saturation. Experiments on two datasets demonstrate that REPaL significantly improves cost-effective zero-shot performance by substantial margins."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/129194"],"dc:language":["en","eng"],"dc:rights":["Copyright 2025 Sizhe Zhou"],"dc:subject":["relation extraction","large language models","small language models"],"dc:title":["Tailoring large language models for zero-shot relation extraction"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:04Z"}