arXiv preprint ·
Reading Why: Born-Measured Attention with Exact Word Evidence on Light and Qubits
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
When a language model answers, we can usually only estimate why. We build a transformer in which every step is a Born measurement - attention $|\langle q|k\rangle|^4$ normalised over each row, a measured product in place of the feed-forward nonlinearity, and a Born readout. With its denominators held at their values for the input, each class probability is a polynomial in the word amplitudes, so its integrated-gradient evidence is computed by a finite quadrature with no quadrature error: the words' contributions sum to the class probability above its no-information share, to $10^{-11}$ or better. On e-SNLI natural language inference, with about 98k parameters outside the embedding, the model reaches $76.98 \pm 0.17$% test accuracy, level with a softmax transformer of the same size ($76.62 \pm 0.36$%). Deleting the word its evidence ranks first changes the answer in 45% of test pairs, out of the 53% any single deletion can change, against 32% for the softmax model explained the same way and 40% by its attention x gradient. Attention can look as much like the words annotators highlighted as the evidence does, yet accounts for the answer less well: deleting the word it ranks first changes the Born Transformer's answer in 32% of pairs. The same weights compile exactly to optoelectronic optics and to qubits, and a coherent attention layer becomes one circuit equal to the model (852 CNOTs on 23 qubits). On IBM's ibm_kingston, reading only the attention overlaps by Hadamard tests, the device gave the model's answers, also when a word was deleted. With calibration circuits as deep as the overlaps, each word's evidence with the device's attention row sums followed the exact evidence (correlations 0.997 and 0.932 on two pairs chosen in advance); where a pair's evidence is a small total of large opposite terms the earlier runs say it does not, and we say why.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Quantum Physics
Thermally enhanced magnon-phonon resonance in van der Waals antiferromagnet FePS3
Time-resolved infrared spectroscopy with entangled photons and visible-light detection
Asymptotic electromagnetic transition conditions for thin layers: the arithmetic-mean field and susceptibility hybridization
Optical angular momentum and pseudospin-orbit coupling in photonic graphene
Reference-Beam-Assisted Perturbation-Resistant Imaging Through A Single Multimode Fiber
Get the few papers worth your time in Quantum Physics every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.