Generating descriptions for a structured knowledge base

Generating natural language sentences to narrate structured knowledge automatically has been in great demand for human communication and knowledge propagation. We aim to automatically generate natural language descriptions about an input structured knowledge graph (KG). Graph-to-text generation has benefited from pre-trained language models (PLMs) in achieving better performance than structured graph encoders. However, they fail to fully utilize the structure information of the input graph. In this thesis, we first build our generation framework based on a pointer network which can copy facts from the input knowledge base (KB), and add two attention mechanisms: (i) slot-aware attention to capture the association between a slot type and its corresponding slot value; and (ii) a new table position self-attention to capture the inter-dependencies among related slots. We also create a new data set which includes 106,216 pairs of structured KBs and their corresponding natural language descriptions for two distinct entity types. We then aim to further improve the performance of the pre-trained language model by proposing a structured graph-to-text model with a two-step fine-tuning mechanism which first fine-tunes the model on Wikipedia before adapting to the graph-to-text generation. In addition to using the traditional token and position embeddings to encode the knowledge graph (KG), we propose a novel tree-level embedding method to capture the inter-dependency structures of the input graph. Experiments show that our approach significantly outperforms state-of-the-art methods.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2022

Author dc:creator

• Wang, Qingyun

Contributors dc:contributor

• Ji, Heng

• Knowledge Base
• Pre-trained Language Models
• Attention Mechanism
• Position Embeddings
• Table Position Self-Attention
• Two-step Fine-tuning Mechanism

Statement dc:rights

• Copyright 2022 Qingyun Wang

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/115750

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F115750