Exploring the power of text-rich graph representation learning

During my doctoral research, I observed that many applications related to graphs cannot be captured by existing simple network models. Lots of real networks exhibit massive text information on various type of objects, also known as text-rich or text-attributed networks. Traditional graph representation learning (e.g. network embedding) largely overlook the complex textual information within nodes and edges. Consequently, many recent graph mining algorithms employ supervised representation learning using neural networks on graph-structured data, specifically graph neural networks (GNNs). In this dissertation, I am motivated to bridge the gap between the frontier machine learning techniques and real-world problems on graph structured data (e.g. web-scale retrieval and classification). Two primary obstacles have hindered previous work on text-rich graphs. First, they assume sufficient and well-posed task-specific annotations. Second, an abundant computation budget is required for graph neural network training and inference, which is unrealistic considering large language model with billions parameters. From a practitioner's perspective, my research follows label efficient and parameter efficient principles to design graph representation learning algorithms for effective and efficient modeling of text-rich graphs. The first part of my dissertation work focuses on label-efficient representation learning, which reduces the need for extensive annotation across various tasks and graphs. Then I will introduce my recent efforts on seamless integration of language models and graph neural networks without excessive amount of training parameters. In contrast to existing work that develops powerful architectures for specific applications, my thesis overcomes the barriers to achieving flexibility and efficiency in general graph neural networks. Therefore, the proposed models can not only achieve superior performance in selected applications, but also enhance the capacity of any existing graph mining tasks. Together with all these efforts, the developed algorithms improve the adaptation of existing graph neural networks on more sophisticated text-rich networks and seek a more powerful representation learning paradigm in this area.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2023

Author dc:creator

• Zhu, Qi

Contributors dc:contributor

• Han, Jiawei
• Tong, Hanghang
• Sundaram, Hari
• Perozzi, Bryan

• Graph Neural Networks
• Text-rich Graph
• Representation Learning
• Language Model
• Generalization

Statement dc:rights

• Copyright 2023 Qi Zhu

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/122007

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F122007