arXiv preprint ·
Tampered ML: Evaluating LLM Judgment of Machine Learning Research
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
In brief
Can AI Spot a Flawed Experiment?
Tampered ML introduces a benchmark of 1,858 pairs of experiment proposals based on published machine-learning papers. In each pair, one proposal is minimally edited to add a targeted flaw, letting researchers test whether models pick the more methodologically sound version across both presentation orders.
What they found: Frontier models performed above chance, but the strongest named result in the abstract was Opus 5’s 77.9% accuracy. The evaluation used Tampered ML’s 1,858 paired proposals, with human review estimating that 86% of items were valid (95% CI: 81–91%). Models were sensitive to presentation order and often preferred flawed designs that looked more specific, transparent, or conventional.
AI-written summary. Check anything you rely on against the original paper.
Abstract
One approach to accelerating progress on AI alignment is to automate parts of alignment research itself. Doing so requires systems that can perform reliably on hard-to-supervise research tasks, including identifying flaws in research proposals and evaluating whether a proposed experiment supports its intended claim. To evaluate this capability, we introduce Tampered ML, a benchmark of 1,858 item-pairs, each consisting of an experiment proposal reconstructed from a published machine learning paper and a minimally edited counterfactual that introduces a targeted methodological flaw. Models must identify the more methodologically sound proposal in both presentation orders. A human validation study on a stratified sample estimates that 86% of items are valid (95% CI: 81-91%). Frontier models perform above chance but remain far from reliable (with Opus 5 achieving 77.9% accuracy). They are highly sensitive to the order of presentation and favor flawed designs that appear more specific, transparent, or conventional.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Computation & Language / NLP
Jaxnasium: Embracing PyTrees For Any-Agent, Cross-Suite Reinforcement Learning
When More Metadata Misleads: Probing Field-Level Grounding in LLM-Generated Museum Descriptions
Ignore the Writing: A Proposal for Peer Review in the Age of AI
Major Revision, Minor Accountability: LLMs as Reviewers of Published Articles in Top-Tier Journals
Integrated data, metadata, and paradata management system for 3D Digital Cultural Heritage objects: Workflow automation, federated authentication, and publication
Get the few papers worth your time in Computation & Language / NLP every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.