{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/113346"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/113346","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Relation extraction: exploring syntax parsing and constructing it as attention-like structure","abstract":"Relation extraction has attracted scientists’ attention since early 21st centuries and it has been one of the common natural language processing (NLP) tasks. It is so important since it could extract semantic relationships from corpus. There are several subtasks in relation extraction area, including joint entity and relation recognition, dialog relation extraction and few-shot relation classification. Some literatures have conducted experiments on the combination of syntax parsing, attention mechanism and transformers and they had obtained outstanding improvement in their tasks. Motivated by application of syntax parsing and recent state of arts of syntax-aware BERT [1] related models, we construct a syntax mechanism to convert syntax tree structure to matrixes and apply on the finetuning process of SyntaxBERT [2] and syntax-aware -local-attention attention BERT (SLA) [3] to strengthen their ability to learning entity relations. They would be finetuned on TACRED [4] dataset. They are compared with the finetuning of SpanBERT [5] and SpanBERT+RECENT [6], which has obtained the best result in TACRED. The result of the experiments inspires some interesting thoughts on syntax parsing, attention mechanism and BERT.","abstract_html":"Relation extraction has attracted scientists’ attention since early 21st centuries and it has been one of the common natural language processing (NLP) tasks. It is so important since it could extract semantic relationships from corpus. There are several subtasks in relation extraction area, including joint entity and relation recognition, dialog relation extraction and few-shot relation classification. Some literatures have conducted experiments on the combination of syntax parsing, attention mechanism and transformers and they had obtained outstanding improvement in their tasks. Motivated by application of syntax parsing and recent state of arts of syntax-aware BERT [1] related models, we construct a syntax mechanism to convert syntax tree structure to matrixes and apply on the finetuning process of SyntaxBERT [2] and syntax-aware -local-attention attention BERT (SLA) [3] to strengthen their ability to learning entity relations. They would be finetuned on TACRED [4] dataset. They are compared with the finetuning of SpanBERT [5] and SpanBERT+RECENT [6], which has obtained the best result in TACRED. The result of the experiments inspires some interesting thoughts on syntax parsing, attention mechanism and BERT.","abstract_has_math":false,"creators":["Huang, Rui"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Industrial Engineering","degree_department":null,"school":null,"contributors":["Sun, Ruoyu"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022-01-12T22:56:15Z","date_published":"2022-01-12T22:56:15Z","updated_at":"2026-07-22T22:24:53Z","subjects":["Syntax Parsing","Natural Language Processing","Relation Extraction","BERT"],"languages":["en"],"rights":["Copyright 2021 Rui Huang"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/113346","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Sun, Ruoyu"]},{"key":"dc:creator","label":"Author","values":["Huang, Rui"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2022-01-12T22:56:15Z","2024-01-12T22:56:20Z","2021-07-21","2021-08"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Industrial Engineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Syntax Parsing","Natural Language Processing","Relation Extraction","BERT"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2021 Rui Huang"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/113346"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Relation extraction has attracted scientists’ attention since early 21st centuries and it has been one of the common natural language processing (NLP) tasks. It is so important since it could extract semantic relationships from corpus. There are several subtasks in relation extraction area, including joint entity and relation recognition, dialog relation extraction and few-shot relation classification. Some literatures have conducted experiments on the combination of syntax parsing, attention mechanism and transformers and they had obtained outstanding improvement in their tasks. Motivated by application of syntax parsing and recent state of arts of syntax-aware BERT [1] related models, we construct a syntax mechanism to convert syntax tree structure to matrixes and apply on the finetuning process of SyntaxBERT [2] and syntax-aware -local-attention attention BERT (SLA) [3] to strengthen their ability to learning entity relations. They would be finetuned on TACRED [4] dataset. They are compared with the finetuning of SpanBERT [5] and SpanBERT+RECENT [6], which has obtained the best result in TACRED. The result of the experiments inspires some interesting thoughts on syntax parsing, attention mechanism and BERT.","Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2023-08-01","The student, Rui Huang, accepted the attached license on 2021-07-21 at 11:58.","The student, Rui Huang, submitted this Thesis for approval on 2021-07-21 at 12:38.","This Thesis was approved for publication on 2021-07-21 at 14:50.","DSpace SAF Submission Ingestion Package generated from Vireo submission #17050 on 2022-01-12 at 13:05:30","Made available in DSpace on 2022-01-12T22:56:15Z (GMT). No. of bitstreams: 3 HUANG-THESIS-2021.pdf: 419961 bytes, checksum: 399e77c32d01f5d8f1bc8eea322bbdef (MD5) Rui Huang_671327879_Thesis.docx: 311076 bytes, checksum: f23a25c3d7ef6042fe5c2eaeb05604c2 (MD5) LICENSE.txt: 4206 bytes, checksum: 9438c695c8e6e391e961327d9d986002 (MD5) Previous issue date: 2021-07-21","Embargo set by: Seth Robbins for item 121275 Lift date: 2024-01-12T22:56:20Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Author requested closed access (OA after 2yrs) in Vireo ETD system","Limited"]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Relation extraction: exploring syntax parsing and constructing it as attention-like structure"]}]}],"canonical_facts":{"dc:contributor":["Sun, Ruoyu"],"dc:creator":["Huang, Rui"],"dc:date":["2022-01-12T22:56:15Z","2024-01-12T22:56:20Z","2021-07-21","2021-08"],"dc:description":["Relation extraction has attracted scientists’ attention since early 21st centuries and it has been one of the common natural language processing (NLP) tasks. It is so important since it could extract semantic relationships from corpus. There are several subtasks in relation extraction area, including joint entity and relation recognition, dialog relation extraction and few-shot relation classification. Some literatures have conducted experiments on the combination of syntax parsing, attention mechanism and transformers and they had obtained outstanding improvement in their tasks. Motivated by application of syntax parsing and recent state of arts of syntax-aware BERT [1] related models, we construct a syntax mechanism to convert syntax tree structure to matrixes and apply on the finetuning process of SyntaxBERT [2] and syntax-aware -local-attention attention BERT (SLA) [3] to strengthen their ability to learning entity relations. They would be finetuned on TACRED [4] dataset. They are compared with the finetuning of SpanBERT [5] and SpanBERT+RECENT [6], which has obtained the best result in TACRED. The result of the experiments inspires some interesting thoughts on syntax parsing, attention mechanism and BERT.","Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2023-08-01","The student, Rui Huang, accepted the attached license on 2021-07-21 at 11:58.","The student, Rui Huang, submitted this Thesis for approval on 2021-07-21 at 12:38.","This Thesis was approved for publication on 2021-07-21 at 14:50.","DSpace SAF Submission Ingestion Package generated from Vireo submission #17050 on 2022-01-12 at 13:05:30","Made available in DSpace on 2022-01-12T22:56:15Z (GMT). No. of bitstreams: 3 HUANG-THESIS-2021.pdf: 419961 bytes, checksum: 399e77c32d01f5d8f1bc8eea322bbdef (MD5) Rui Huang_671327879_Thesis.docx: 311076 bytes, checksum: f23a25c3d7ef6042fe5c2eaeb05604c2 (MD5) LICENSE.txt: 4206 bytes, checksum: 9438c695c8e6e391e961327d9d986002 (MD5) Previous issue date: 2021-07-21","Embargo set by: Seth Robbins for item 121275 Lift date: 2024-01-12T22:56:20Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Author requested closed access (OA after 2yrs) in Vireo ETD system","Limited"],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/113346"],"dc:language":["en"],"dc:rights":["Copyright 2021 Rui Huang"],"dc:subject":["Syntax Parsing","Natural Language Processing","Relation Extraction","BERT"],"dc:title":["Relation extraction: exploring syntax parsing and constructing it as attention-like structure"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Industrial Engineering"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:53Z"}