Adversarial graph contrastive learning with information regularization

Contrastive learning is an effective unsupervised method in graph representation learning, and the key component of contrastive learning lies in the construction of positive and negative samples. Previous methods usually utilize the proximity of nodes in the graph as the principle to decide the positive and negative samples. Recently, the data augmentation based contrastive learning method has advanced to show great power in the visual domain. This method forces the encoder to maximize the mutual information between the positive pairs generated through the data augmentation of the same instance. It achieves comparable performance to the supervised learning on various downstream tasks. Some works also extended this method from images to graphs. However, most prior works are directly adapted from the models designed for images. Unlike the data augmentation on images, the data augmentation on graphs is far less intuitive and much harder to provide high-quality contrastive samples, which are the key to the performance of contrastive learning models. Although some works also try to design an adaptive data augmentation strategy based on human knowledge, they can only take effect on some specific datasets and fail to achieve a consistent improvement. This leaves much space for improvement over the existing graph contrastive learning frameworks. In this thesis, by introducing an adversarial graph view for data augmentation, we propose a simple but effective method, Adversarial Graph Contrastive Learning (ARIEL), to extract informative contrastive samples within a reasonable constraint. We make use of the Projected Gradient Descent attack to generate a new augmentation view by adding perturbations on the original graph, which maximizes the contrastive loss against a standard augmentation view, subject to the constraints on perturbations. Due to the instability brought by the adversarial view, we also develop a technique called information regularization to penalize the high similarity granted to a less similar pair. By using the subgraph sampling technique, we successfully scale ARIEL to large graphs with affordable computational cost. ARIEL consistently outperforms the current graph contrastive learning methods in the node classification and graph classification tasks over various real-world datasets. We further demonstrate that ARIEL could behave better on the attacked graphs than previous graph contrastive learning algorithms, which indicates ARIEL is more robust to adversarial attacks.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2022

Author dc:creator

• Feng, Shengyu

Contributors dc:contributor

• Tong, Hanghang

• graph representation learning
• contrastive learning
• adversarial training
• mutual information

Statement dc:rights

• Copyright 2022 Shengyu Feng

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/115773

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F115773