arXiv preprint ·
Pre-training with Graph Transformers
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
This article investigates pre-training strategies for graph transformers in the biochemistry domain. By conducting comprehensive experiments, the study reveals that supervised pre-training using computed properties as labels provides the highest performance gain on downstream tasks. The results also highlight the importance of constraining model capacity to mitigate overfitting in graph transformers.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Machine Learning
An Efficient and Modular Framework for Targeted Harm Mitigation in LLMS
Partition-Aware Scheduling for Mobile Heterogeneous Inference Co-Execution
EI-DDLGN: Efficient Encrypted Inference with Deep Differentiable Logic Gate Networks under TFHE
Graph-Transformer Fraud Detection with Self-Supervised Pretraining and Conformal Risk Control
FlowTSFM: Turning Encoder Depth into Quantile Transport
Get the few papers worth your time in Machine Learning every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.