OpenAlex ·
Learning Task-Specific Antibody Representations via Function-Aware Masking
- Published
- Source
- OpenAlex
- DOI
- 10.48550/arxiv.2609.00518
- Original
- Read it at the publisher
- Topics
Abstract
Antibody-specific language models pretrained via masked language modeling (MLM) learn representations that are critical for downstream sequence design and property prediction tasks. Yet, the corruption process itself is rarely leveraged as a source of inductive bias during pretraining. While preferentially masking complementarity-determining regions (CDRs) improves binding-related predictions, antibodies possess diverse biological priors over a variety of functions. Herein, we introduce function-aware masking, a family of pretraining algorithms that align mask placement with specific functional priors (e.g., from IMGT annotations or structure predictions) to shape the learned representation space. We show that these specialist masking strategies significantly improve performance on their respective objectives, yielding up to a 14% gain on structure-related tasks and up to a 5.9x improvement on CDR-related tasks. To further improve performance across multiple functional axes, we develop hybrid masking strategies that integrate multiple priors, balancing reconstruction over binding, structural, and biophysical objectives. Our results demonstrate that informed mask placement provides a parameter-free mechanism for imposing functional inductive biases in antibody language model training.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Radiology & Imaging
Conditional Flow Matching for Cross-Field MRI Harmonisation
The 2024 MRSI Data Processing and Quantification Challenge Synthetic Dataset
Prior-Guided Implicit Neural Representations for Single-Subject Diffusion MRI Super-Resolution
A multicenter benchmark and clinically structured metric for coronary CTA report generation
RadMatch: Auditable Radiology Report Evaluation via Finding-Level Matching
Get the few papers worth your time in Radiology & Imaging every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.