arXiv preprint ·
A Battery for AI Propensity Measurement: Rubrics, Synthetic Item Banks, and Annotations
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
We release a battery of 28 propensities for AI research, with expert-written rubrics, synthetic datasets, and annotated demand intervals, built with the method of Romero-Alvarado et al. (2026). We selected a wide collection of generic propensities that may influence the behavior of AI systems and humans. They are inspired by psychometric, psychological, and behavioral research, so that experimental results can be understood in the context of familiar constructs previously used also for human behavior research. The rubrics are useful for annotating existing benchmarks whose items may be relevant or not for one or more propensities. For easier profiling, however, the synthetic datasets are defined in such a way that they should be sensitive to one propensity only, allowing for a quick prototyping of propensity profiles using the battery of datasets. We also outline directions for future propensity research.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Artificial Intelligence
The Policies of Programming Language Ecosystems: A "Qualitative" Analysis
Safe by Benchmark, Unsafe in Practice: Lessons from 1331 Child-Related AI Incidents
Learning by Collaborating in the Age of AI: Research Mentorship for Polymathic, Workflow-Centered Science
Arya Forge: Auditing Validation and Release Approval for Financial Instruction Data
High Scores, Hidden Failures: Evaluating Codebook-Based LLM Content Analysis
Get the few papers worth your time in Artificial Intelligence every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.