Relation extraction: exploring syntax parsing and constructing it as attention-like structure

Relation extraction has attracted scientists’ attention since early 21st centuries and it has been one of the common natural language processing (NLP) tasks. It is so important since it could extract semantic relationships from corpus. There are several subtasks in relation extraction area, including joint entity and relation recognition, dialog relation extraction and few-shot relation classification. Some literatures have conducted experiments on the combination of syntax parsing, attention mechanism and transformers and they had obtained outstanding improvement in their tasks. Motivated by application of syntax parsing and recent state of arts of syntax-aware BERT [1] related models, we construct a syntax mechanism to convert syntax tree structure to matrixes and apply on the finetuning process of SyntaxBERT [2] and syntax-aware -local-attention attention BERT (SLA) [3] to strengthen their ability to learning entity relations. They would be finetuned on TACRED [4] dataset. They are compared with the finetuning of SpanBERT [5] and SpanBERT+RECENT [6], which has obtained the best result in TACRED. The result of the experiments inspires some interesting thoughts on syntax parsing, attention mechanism and BERT.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Industrial Engineering

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2022

Author dc:creator

• Huang, Rui

Contributors dc:contributor

• Sun, Ruoyu

• Syntax Parsing
• Natural Language Processing
• Relation Extraction
• BERT

Statement dc:rights

• Copyright 2021 Rui Huang

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/113346

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/113346