arXiv preprint ·
GradeTrap: Authority Cues in Images Shift VLM Judgments Despite Explicit Instructions to Ignore Them
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
In brief
Navigating Authority in AI Judgments
This paper introduces GradeTrap, a novel evaluation framework that tests how different social cues influence VLM judgments. It reveals that even when instructed to disregard authority, models still exhibit bias towards official answers over student responses.
What they found: The study found that VLMs are significantly influenced by authoritative cues, with a 19.5% selection rate for official answers compared to a mere 5.4% for generic controls. The teacher review effect was also noted at 6.9%, while peer reviews had no measurable impact. These results highlight the challenges of ensuring independent judgment in AI models, as they often default to authority despite explicit instructions to ignore it.
AI-written summary. Check anything you rely on against the original paper.
Abstract
As vision-language models (VLMs) become increasingly capable and are deployed in consequential real-world settings, they must evaluate evidence independently rather than defer uncritically to human authority. We introduce GradeTrap, a controlled evaluation that places two social cues in direct conflict: a student answer, which should attract sycophantic agreement, and a conflicting answer attributed to a peer, teacher, or official answer key, which should attract authority-based deference. Models produce free-form answers while being explicitly instructed to solve independently and ignore all student answers, feedback, and grading marks. We test the models on 60 synthetic real-world trade-off scenarios. Five neutral trials establish a stable model-relative preference, followed by three repetitions of six experimental cues including controls. On the 45-item common intersection across Gemini 3.5 Flash-Lite, GPT-5.6 Luna, and Claude Haiku 4.5, a generic second-answer control yields 5.4% conflicting-answer selection. Relative to that control, pooled within-item changes show no reliable peer-review effect, a 6.9-point teacher-review effect, and a 19.5-point official-key effect. In contrast, a displayed conflicting student answer alone compared to a displayed student reference answer alone only raises selection from 2.2% to 5.2%. Official-key provenance therefore redirects judgements more than a student answer or the generic second-answer control, despite an explicit ignore instruction and an opposing student answer given along with the official key. Effects vary in magnitude across the three models.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Computer Vision
Agentic BAIM-LLM Evaluation (ABLE): Benchmarking LLM Use of Protein Design Tools
Reliability, validity, and diagnostic evidence for multi-model LLM short-answer scoring
Intent Drift at SME Scale: Deployment Practice, Not Model Capability, Determines Agentic Compliance
A Translational Note on AI Safety Evaluation
Ordinary, Reasonable Chatbots: Do AI Models Track Human Legal Judgments?
Get the few papers worth your time in Computer Vision every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.