ABM: attention-based message passing network for knowledge graph completion

Knowledge graphs are ubiquitous and play an important role in many real-world applications, including recommender systems, question-answering, fact-checking, and so on. However, most of the knowledge graphs are incomplete which can hamper their practical usage. Fortunately, knowledge graph completion (KGC) can mitigate this problem by inferring missing edges in the knowledge graph according to the existing information. In this thesis, we propose a novel KGC method named Attention-Based Message passing (ABM) which focuses on predicting the relation between any two entities in a knowledge graph. The proposed ABM consists of three integral parts, including (1) context embedding, (2) structure embedding, and (3) path embedding. In the context embedding, the proposed ABM generalizes the existing message passing neural network to update the node embedding and the edge embedding to assimilate the knowledge of nodes’ neighbors, which captures the relative role information of the edge that we want to predict. In the structure embedding, the proposed method overcomes the shortcomings of the existing Graph Neural Network (GNN) method (i.e., most methods ignore the structural similarity between nodes.) by assigning different attention weights to different nodes while conducting the aggregation. Path embedding generates paths between any two entities and treats these paths as sequences. Then, the sequence can be used as the input of the Transformer to update the embedding of the knowledge graph to gather the global role of the missing edges. By utilizing these three mutually complementary strategies, the proposed ABM is able to capture both local and global information which in turn leads to a superb performance. Experiment results show that ABM outperforms baseline methods on a wide range of datasets.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2022

Author dc:creator

• Xu, Weikai

Contributors dc:contributor

• Tong, Hanghang

• Knowledge Graph Completion
• Attention
• Message Passing

Statement dc:rights

• Copyright 2022 Weikai Xu

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/117646

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F117646