{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/109521"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/109521","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Multilingual multitask joint neural information extraction","abstract":"In the age of information overload, the ability to automatically extract useful structured information from texts is urgently needed by a wide range of applications, such as information retrieval and question answering. Over the past decades, researchers have proposed various Information Extraction (IE) techniques to discover important knowledge elements (e.g., entities, relations, events) from unstructured documents. However, as these approaches typically rely on specific hand-crafted rules or manually annotated data, it is usually expensive to adapt them for new settings, such as new languages, domains, scenarios, or genres. Therefore, the goal of this thesis is to develop more robust and portable models for information extraction tasks and build a joint neural architecture that performs multiple IE tasks within a single model. We first focus on the generality of IE models. As most existing neural models use word embeddings as input features, they are sensitive to the quality of word representations. We investigate the possible factors that cause performance degradation when applying a name tagger to new data and tackle this issue from two aspects: 1. Robustness. The reliability and the amount of information of each feature is inconsistent among words. We incorporate reliability signals and dynamic feature composition to enable to model to select reliable and effective features. 2. Generality. Overfitting is a major problem leading to the huge performance gap between seen and unseen names. As a solution, we encourage the model to leverage contextual features that are more general. Next, we explore the portability of models for sequence labeling, the underlying problem of many Natural Language Processing (NLP) tasks such as name tagging. Current models cannot be applied to very dissimilar settings (e.g., other languages), whereas annotating new data for all possible settings is infeasible. Hence, we propose to transfer knowledge across different models through multitask learning to reduce the need for data annotation. To maximize the knowledge being transferred, we design a unified and extendable architecture that integrates multiple transfer approaches. After that, we extend this framework to more IE tasks and propose a joint neural architecture, OneIE, that performs multilingual entity, relation, and event extraction simultaneously. In addition to multitask learning, we further incorporate global features to capture the cross-subtask and cross-instance interactions among knowledge elements. Finally, we propose OneIE to perform joint inference without using additional global features.","abstract_html":"In the age of information overload, the ability to automatically extract useful structured information from texts is urgently needed by a wide range of applications, such as information retrieval and question answering. Over the past decades, researchers have proposed various Information Extraction (IE) techniques to discover important knowledge elements (e.g., entities, relations, events) from unstructured documents. However, as these approaches typically rely on specific hand-crafted rules or manually annotated data, it is usually expensive to adapt them for new settings, such as new languages, domains, scenarios, or genres. Therefore, the goal of this thesis is to develop more robust and portable models for information extraction tasks and build a joint neural architecture that performs multiple IE tasks within a single model. We first focus on the generality of IE models. As most existing neural models use word embeddings as input features, they are sensitive to the quality of word representations. We investigate the possible factors that cause performance degradation when applying a name tagger to new data and tackle this issue from two aspects: 1. Robustness. The reliability and the amount of information of each feature is inconsistent among words. We incorporate reliability signals and dynamic feature composition to enable to model to select reliable and effective features. 2. Generality. Overfitting is a major problem leading to the huge performance gap between seen and unseen names. As a solution, we encourage the model to leverage contextual features that are more general. Next, we explore the portability of models for sequence labeling, the underlying problem of many Natural Language Processing (NLP) tasks such as name tagging. Current models cannot be applied to very dissimilar settings (e.g., other languages), whereas annotating new data for all possible settings is infeasible. Hence, we propose to transfer knowledge across different models through multitask learning to reduce the need for data annotation. To maximize the knowledge being transferred, we design a unified and extendable architecture that integrates multiple transfer approaches. After that, we extend this framework to more IE tasks and propose a joint neural architecture, OneIE, that performs multilingual entity, relation, and event extraction simultaneously. In addition to multitask learning, we further incorporate global features to capture the cross-subtask and cross-instance interactions among knowledge elements. Finally, we propose OneIE to perform joint inference without using additional global features.","abstract_has_math":false,"creators":["Lin, Ying"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Ji, Heng","Han, Jiawei","Zhai, ChengXiang","Roth, Dan","Stoyanov, Veselin"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2021,"date_issued":"2021-03-05T21:42:47Z","date_published":"2021-03-05T21:42:47Z","updated_at":"2026-07-22T22:24:50Z","subjects":["Joint Information Extraction","Multitask Learning"],"languages":["en"],"rights":["Copyright 2020 Ying Lin"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/109521","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Ji, Heng","Han, Jiawei","Zhai, ChengXiang","Roth, Dan","Stoyanov, Veselin"]},{"key":"dc:creator","label":"Author","values":["Lin, Ying"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2021-03-05T21:42:47Z","2023-03-05T21:43:00Z","2020-12-02","2020-12"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Joint Information Extraction","Multitask Learning"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2020 Ying Lin"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/109521"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["In the age of information overload, the ability to automatically extract useful structured information from texts is urgently needed by a wide range of applications, such as information retrieval and question answering. Over the past decades, researchers have proposed various Information Extraction (IE) techniques to discover important knowledge elements (e.g., entities, relations, events) from unstructured documents. However, as these approaches typically rely on specific hand-crafted rules or manually annotated data, it is usually expensive to adapt them for new settings, such as new languages, domains, scenarios, or genres. Therefore, the goal of this thesis is to develop more robust and portable models for information extraction tasks and build a joint neural architecture that performs multiple IE tasks within a single model. We first focus on the generality of IE models. As most existing neural models use word embeddings as input features, they are sensitive to the quality of word representations. We investigate the possible factors that cause performance degradation when applying a name tagger to new data and tackle this issue from two aspects: 1. Robustness. The reliability and the amount of information of each feature is inconsistent among words. We incorporate reliability signals and dynamic feature composition to enable to model to select reliable and effective features. 2. Generality. Overfitting is a major problem leading to the huge performance gap between seen and unseen names. As a solution, we encourage the model to leverage contextual features that are more general. Next, we explore the portability of models for sequence labeling, the underlying problem of many Natural Language Processing (NLP) tasks such as name tagging. Current models cannot be applied to very dissimilar settings (e.g., other languages), whereas annotating new data for all possible settings is infeasible. Hence, we propose to transfer knowledge across different models through multitask learning to reduce the need for data annotation. To maximize the knowledge being transferred, we design a unified and extendable architecture that integrates multiple transfer approaches. After that, we extend this framework to more IE tasks and propose a joint neural architecture, OneIE, that performs multilingual entity, relation, and event extraction simultaneously. In addition to multitask learning, we further incorporate global features to capture the cross-subtask and cross-instance interactions among knowledge elements. Finally, we propose OneIE to perform joint inference without using additional global features.","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2022-12-01","The student, Ying Lin, accepted the attached license on 2020-12-02 at 15:20.","The student, Ying Lin, submitted this Dissertation for approval on 2020-12-02 at 15:26.","This Dissertation was approved for publication on 2020-12-02 at 16:00.","DSpace SAF Submission Ingestion Package generated from Vireo submission #16041 on 2021-03-04 at 16:20:23","Made available in DSpace on 2021-03-05T21:42:47Z (GMT). No. of bitstreams: 3 LIN-DISSERTATION-2020.pdf: 5385567 bytes, checksum: 0eab79f17dbbb5269db9dcad6febc205 (MD5) LICENSE.txt: 4205 bytes, checksum: 359ebd4300b1e4f6ff36935649ad65cd (MD5) PROQUEST_LICENSE.txt: 4551 bytes, checksum: ff145c4be86e1ec7e1555127a585b220 (MD5) Previous issue date: 2020-12-02","Embargo set by: Seth Robbins for item 117226 Lift date: 2023-03-05T21:43:00Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","U of I Only"]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Multilingual multitask joint neural information extraction"]}]}],"canonical_facts":{"dc:contributor":["Ji, Heng","Han, Jiawei","Zhai, ChengXiang","Roth, Dan","Stoyanov, Veselin"],"dc:creator":["Lin, Ying"],"dc:date":["2021-03-05T21:42:47Z","2023-03-05T21:43:00Z","2020-12-02","2020-12"],"dc:description":["In the age of information overload, the ability to automatically extract useful structured information from texts is urgently needed by a wide range of applications, such as information retrieval and question answering. Over the past decades, researchers have proposed various Information Extraction (IE) techniques to discover important knowledge elements (e.g., entities, relations, events) from unstructured documents. However, as these approaches typically rely on specific hand-crafted rules or manually annotated data, it is usually expensive to adapt them for new settings, such as new languages, domains, scenarios, or genres. Therefore, the goal of this thesis is to develop more robust and portable models for information extraction tasks and build a joint neural architecture that performs multiple IE tasks within a single model. We first focus on the generality of IE models. As most existing neural models use word embeddings as input features, they are sensitive to the quality of word representations. We investigate the possible factors that cause performance degradation when applying a name tagger to new data and tackle this issue from two aspects: 1. Robustness. The reliability and the amount of information of each feature is inconsistent among words. We incorporate reliability signals and dynamic feature composition to enable to model to select reliable and effective features. 2. Generality. Overfitting is a major problem leading to the huge performance gap between seen and unseen names. As a solution, we encourage the model to leverage contextual features that are more general. Next, we explore the portability of models for sequence labeling, the underlying problem of many Natural Language Processing (NLP) tasks such as name tagging. Current models cannot be applied to very dissimilar settings (e.g., other languages), whereas annotating new data for all possible settings is infeasible. Hence, we propose to transfer knowledge across different models through multitask learning to reduce the need for data annotation. To maximize the knowledge being transferred, we design a unified and extendable architecture that integrates multiple transfer approaches. After that, we extend this framework to more IE tasks and propose a joint neural architecture, OneIE, that performs multilingual entity, relation, and event extraction simultaneously. In addition to multitask learning, we further incorporate global features to capture the cross-subtask and cross-instance interactions among knowledge elements. Finally, we propose OneIE to perform joint inference without using additional global features.","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2022-12-01","The student, Ying Lin, accepted the attached license on 2020-12-02 at 15:20.","The student, Ying Lin, submitted this Dissertation for approval on 2020-12-02 at 15:26.","This Dissertation was approved for publication on 2020-12-02 at 16:00.","DSpace SAF Submission Ingestion Package generated from Vireo submission #16041 on 2021-03-04 at 16:20:23","Made available in DSpace on 2021-03-05T21:42:47Z (GMT). No. of bitstreams: 3 LIN-DISSERTATION-2020.pdf: 5385567 bytes, checksum: 0eab79f17dbbb5269db9dcad6febc205 (MD5) LICENSE.txt: 4205 bytes, checksum: 359ebd4300b1e4f6ff36935649ad65cd (MD5) PROQUEST_LICENSE.txt: 4551 bytes, checksum: ff145c4be86e1ec7e1555127a585b220 (MD5) Previous issue date: 2020-12-02","Embargo set by: Seth Robbins for item 117226 Lift date: 2023-03-05T21:43:00Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","U of I Only"],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/109521"],"dc:language":["en"],"dc:rights":["Copyright 2020 Ying Lin"],"dc:subject":["Joint Information Extraction","Multitask Learning"],"dc:title":["Multilingual multitask joint neural information extraction"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:50Z"}