Graph Contrastive Multi-view Learning: A Pre-training Framework for Graph Classification
Authors: Adjeisah, M., Zhu, X., Xu, H. and Ayall, T.A.
Journal: Knowledge-Based Systems
Volume: 299
ISSN: 0950-7051
DOI: 10.1016/j.knosys.2024.112112
Abstract:Recent advancements in node and graph classification tasks can be attributed to the implementation of contrastive learning and similarity search. Despite considerable progress, these approaches present challenges. The integration of similarity search introduces an additional layer of complexity to the model. At the same time, applying contrastive learning to non-transferable domains or out-of-domain datasets results in less competitive outcomes. In this work, we propose maintaining domain specificity for these tasks, which has demonstrated the potential to improve performance by eliminating the need for additional similarity searches. We adopt a fraction of domain-specific datasets for pre-training purposes, generating augmented pairs that retain structural similarity to the original graph, thereby broadening the number of views. This strategy involves a comprehensive exploration of optimal augmentations to devise multi-view embeddings. An evaluation protocol, which focuses on error minimization, accuracy enhancement, and overfitting prevention, guides this process to learn inherent, transferable structural representations that span diverse datasets. We combine pre-trained embeddings and the source graph as a beneficial input, leveraging local and global graph information to enrich downstream tasks. Furthermore, to maximize the utility of negative samples in contrastive learning, we extend the training mechanism during the pre-training stage. Our method consistently outperforms comparative baseline approaches in comprehensive experiments conducted on benchmark graph datasets of varying sizes and characteristics, establishing new state-of-the-art results.
Source: Scopus
Graph Contrastive Multi-view Learning: A Pre-Training Framework for Graph Classification
Authors: Adjeisah, M., Zhu, X., Xu, H. and Ayall, T.A.
Journal: Knowledge-Based Systems
Publisher: Elsevier
ISSN: 0950-7051
DOI: 10.1016/j.knosys.2024.112112
Abstract:Recent developments in node and graph classification tasks result from graph contrastive learning and similarity search. For performance reasons, the similarity search in contrastive learning, usually implemented in non-transferable to out-of-domain data, mostly yields low, competitive results in various tasks. Hypothetically, keeping the task domain-specific alleviates the extra similarity search and has shown promise for improving task performance. Motivated by this, a fraction of domain-specific datasets are used for pre-training, employing augmented pairs with structural similarity to the original graph, thereby increasing the number of views. The approach entails a collective exploration of optimal augmentations for constructing multi-view embeddings, with an evaluation protocol like minimizing errors, accuracy, and overfitting as guidance to learn intrinsic and transferable structural representations across diverse datasets. The pre-trained embeddings and the source graph are merged as input, serving as a valuable strategy for leveraging local and global graph information to enhance downstream tasks. In addition, we prolong the training mechanism during pre-training for more negative samples for each sample, considering their effectiveness in contrastive learning. Extensive experiments on benchmark graph datasets of different sizes with distinct characteristics show that our approach outperforms the compared baselines with new state-of-the-art results, making better generalizable features and avoiding over-smoothing.
Source: Manual