Global Communication or Graph-Specific Memory?

Abstract

Scalable Graph Transformers are commonly trained and evaluated on static large graphs in a transductive setup. Many scalable Graph Transformer components can be formulated as a constant-size shared memory, similar to virtual nodes, providing compressed information about the whole graph. The counterpart of these models in language models and other domains is justified as the input changes, and this mechanism learns to compress some useful information about the input. In transductive learning on a single fixed graph, however, any shared memory can be seen as a constant at test time. This raises the question of what exactly this shared memory does in this static setup. We give preliminary evidence that optimizing a shared memory directly performs similarly to global communication methods, and so normal local message-passing models can embed similar information in their weights. Thus, these settings may be a poor fit for evaluating global communication in graph neural networks.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Machine Learning

• arXiv preprint · 5 October 2026

  Two-Sample Testing via Path-based Inference

  Eshant English, Wei-Cheng Lai, Yanfeng Yang and 3 others
• arXiv preprint · 6 October 2026

  Is $\sqrt{d}$ Separation Necessary for Gradient EM to Learn Gaussian Mixtures in High Dimensions?

  Yiran Zhang, Mo Zhou, Weihang Xu and 2 others
• arXiv preprint · 5 October 2026

  From Pixels, Without Pre-training: Joint Generative and Self-Supervised Representation Learning in One Model

  Vicente Balmaseda, Ching-Long Lin and Tianbao Yang
• arXiv preprint · 6 October 2026

  Learning a Mixture of GFlowNets

  Tiago da Silva, Amauri H. Souza and Salem Lahlou
• arXiv preprint · 5 October 2026

  Retrieval-Based In-Context Learning: A Domain Adaptation Framework

  Yilun Zhu, Naihao Deng, Yingcong Li and 2 others

All new papers in Machine Learning