arXiv preprint ·
Do LLMs Really Lack Difference Awareness? Uncovering Heuristic Substitution in Group-Differentiating Fairness
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
In this paper, we investigate the issue of difference awareness in LLMs, defined as a model's ability to recognize and acknowledge meaningful differences between social groups in fairness decision-making. Previous studies have consistently shown that models perform poorly on such tasks, a finding commonly attributed to a lack of difference awareness. We propose an alternative account, which we term Fairness Heuristic Substitution: difference awareness is not necessarily absent, but the complex demand of simultaneously identifying relevant group differences and determining their implications for fairness decisions may lead models to substitute the simpler, more accessible equal-treatment heuristic, i.e., formal equality. Based on this hypothesis, we introduce TDD, a Two-stage Distinguish-Decide framework that explicitly decomposes fairness reasoning into first distinguishing contextually relevant group differences and then making the corresponding decision, thereby discouraging reliance on formal-equality shortcuts. (i) Across four benchmarks and ten models ranging from billions to trillions of parameters, TDD significantly outperforms one-stage baselines, even one strengthened through simulated test-set leakage. (ii) Combining TDD with neuron-level contrastive probing provides mechanistic evidence consistent with the Fairness Heuristic Substitution hypothesis by revealing two signatures of the formal-equality: a compact, locally coordinated representational structure and a pronounced activation bias under default reasoning. (iii) TDD's stage-wise decomposition uncovers overlooked failure modes in fairness reasoning, including heuristic displacement and misaligned stage dynamics.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Social & Information Networks
Scam Town: Can Scam Education Spread Like a Vaccine?
Optimizing Marketing Budgets for Social Influence: A Generalized Fractional Incentives Framework
Self-organizing reinforcement learning: a learner-free swarm approaches the explore-exploit optimum
Where Does Rhetorical Distortion Live? A Multi-Platform, Multi-Language Measurement of Influencer Content
Beyond a Single Sentiment Score: Topical and Affective Dimensions of Emoji Meaning on Twitter
Get the few papers worth your time in Social & Information Networks every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.