{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/121297"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/121297","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Knowledge acquisition for natural language understanding","abstract":"Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2025-08-01","abstract_html":"Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2025-08-01","abstract_has_math":false,"creators":["Lai, Tuan Manh"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Ji, Heng","Zhai, ChengXiang","Han, Jiawei","Bui, Trung H"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2023,"date_issued":"2023-08","date_published":"2023-08","updated_at":"2026-07-22T22:24:57Z","subjects":["Natural Language Processing","Information Extraction","Deep Learning","Large Language Models"],"languages":["en","eng"],"rights":["Copyright 2023 Tuan Lai"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/121297","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Ji, Heng","Zhai, ChengXiang","Han, Jiawei","Bui, Trung H"]},{"key":"dc:creator","label":"Author","values":["Lai, Tuan Manh"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2023-08","2023-05-24"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Natural Language Processing","Information Extraction","Deep Learning","Large Language Models"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2023 Tuan Lai"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/121297"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2025-08-01","The student, Tuan Lai, accepted the attached license on 2023-05-22 at 19:19.","The student, Tuan Lai, submitted this Dissertation for approval on 2023-05-22 at 19:22.","This Dissertation was approved for publication on 2023-05-24 at 14:01.","DSpace SAF Submission Ingestion Package generated from Vireo submission #19386 on 2023-12-04 at 17:17:46","Large neural models pretrained on vast volumes of text have achieved remarkable success in various natural language processing tasks. However, these models may still face challenges in knowledge-intensive tasks due to their training methods, which typically focus on learning directly from raw texts and do not incorporate existing linguistic resources or structured domain knowledge. This thesis aims to develop methods to effectively incorporate external knowledge into existing neural models to enhance their performance. We propose three novel approaches that offer varying levels of explicitness for incorporating external knowledge into neural models, accommodating a wide range of use cases and providing great flexibility. The first approach involves incorporating various types of domain knowledge from multiple sources into language models using lightweight adapter modules. For each knowledge source of interest, we train an adapter module to capture the knowledge in a self-supervised way. The knowledge encoded in the adapters can then be combined for downstream tasks using fusion layers. This approach provides an easy-to-use, implicit way of incorporating external knowledge. The second approach involves utilizing a retrieval system to retrieve relevant passages from a knowledge base, which can then be used to enhance an output generation model. To train the retrieval components, we use a novel method for generating pseudo-labels to avoid the need for collecting costly gold-standard retrieval labels. This approach offers a more explicit way of accessing and using external knowledge than the adapter-based approach and provides greater interpretability. Additionally, new knowledge can typically be added to the knowledge base without updating any parameters of the neural component. The third approach involves using entity linking to extract the exact part of a knowledge graph that is relevant to the task at hand. We then utilize graph neural networks to incorporate the extracted subgraph into the existing neural model. This approach provides an even more explicit way of incorporating external knowledge, allowing for fine-grained control over what knowledge to incorporate and offering even more interpretability. We demonstrate the effectiveness of our proposed methods on various knowledge-intensive natural language processing tasks, including biomedical information extraction and knowledge-grounded dialog. We show that incorporating external knowledge can help overcome the difficulty of learning domain-specific knowledge and enhance the model's efficiency and interpretability. Our methods also allow for natural updates and additions of external knowledge, providing a flexible and scalable way of enhancing large neural language models. Overall, our methods achieve state-of-the-art results on many benchmarks."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Knowledge acquisition for natural language understanding"]}]}],"canonical_facts":{"dc:contributor":["Ji, Heng","Zhai, ChengXiang","Han, Jiawei","Bui, Trung H"],"dc:creator":["Lai, Tuan Manh"],"dc:date":["2023-08","2023-05-24"],"dc:description":["Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2025-08-01","The student, Tuan Lai, accepted the attached license on 2023-05-22 at 19:19.","The student, Tuan Lai, submitted this Dissertation for approval on 2023-05-22 at 19:22.","This Dissertation was approved for publication on 2023-05-24 at 14:01.","DSpace SAF Submission Ingestion Package generated from Vireo submission #19386 on 2023-12-04 at 17:17:46","Large neural models pretrained on vast volumes of text have achieved remarkable success in various natural language processing tasks. However, these models may still face challenges in knowledge-intensive tasks due to their training methods, which typically focus on learning directly from raw texts and do not incorporate existing linguistic resources or structured domain knowledge. This thesis aims to develop methods to effectively incorporate external knowledge into existing neural models to enhance their performance. We propose three novel approaches that offer varying levels of explicitness for incorporating external knowledge into neural models, accommodating a wide range of use cases and providing great flexibility. The first approach involves incorporating various types of domain knowledge from multiple sources into language models using lightweight adapter modules. For each knowledge source of interest, we train an adapter module to capture the knowledge in a self-supervised way. The knowledge encoded in the adapters can then be combined for downstream tasks using fusion layers. This approach provides an easy-to-use, implicit way of incorporating external knowledge. The second approach involves utilizing a retrieval system to retrieve relevant passages from a knowledge base, which can then be used to enhance an output generation model. To train the retrieval components, we use a novel method for generating pseudo-labels to avoid the need for collecting costly gold-standard retrieval labels. This approach offers a more explicit way of accessing and using external knowledge than the adapter-based approach and provides greater interpretability. Additionally, new knowledge can typically be added to the knowledge base without updating any parameters of the neural component. The third approach involves using entity linking to extract the exact part of a knowledge graph that is relevant to the task at hand. We then utilize graph neural networks to incorporate the extracted subgraph into the existing neural model. This approach provides an even more explicit way of incorporating external knowledge, allowing for fine-grained control over what knowledge to incorporate and offering even more interpretability. We demonstrate the effectiveness of our proposed methods on various knowledge-intensive natural language processing tasks, including biomedical information extraction and knowledge-grounded dialog. We show that incorporating external knowledge can help overcome the difficulty of learning domain-specific knowledge and enhance the model's efficiency and interpretability. Our methods also allow for natural updates and additions of external knowledge, providing a flexible and scalable way of enhancing large neural language models. Overall, our methods achieve state-of-the-art results on many benchmarks."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/121297"],"dc:language":["en","eng"],"dc:rights":["Copyright 2023 Tuan Lai"],"dc:subject":["Natural Language Processing","Information Extraction","Deep Learning","Large Language Models"],"dc:title":["Knowledge acquisition for natural language understanding"],"dc:type":["text"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:57Z"}