arXiv preprint ·
Ask More, Hear Less: How a Review Prompt Changes What Online Ratings Show
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Platforms ask customers for online reviews to reduce self-selection, but theory conflicts on whether review solicitation makes online ratings more or less informative. Selection theory implies that asking helps, voice theory that it dilutes the news, and research on rating inflation that compression destroys information. We reconcile these views with signal detection theory, which separates what a rating knows (its sensitivity) from what it shows (its criterion). We test the framework on Steam, a leading PC video game store, whose client began in October 2019 to prompt players for a review after about three hours of play. Using a difference-in-discontinuities design at that inferred threshold in 21.6 million Steam reviews of 315 games, we find that review counts just above the threshold become 4.25 times those just below, and the jump in the positive share grows by 6.6 percentage points. Agreement with critics changes little near launch and erodes later. The rating's movement changes far more: pass-through from reviews written in the first two hours falls from 0.73 to 0.18 just above the threshold, against 0.81 to 0.64 just below, and a fixed rating tier responds to 12% of downturns in those reviews above the threshold, down from 42%, against 41% and 47% below it. What the stream knows changes less. At equal review counts its sensitivity falls more above the threshold, because added reviews respond about half as much and scores near the ceiling are noisier per review, but at natural counts added volume offsets this loss, leaving a change of -0.03 relative to the stream below (95% CI -0.13 to 0.06). Failures that developers acknowledge, which run through complaints, still move the score after launch. Review prompts therefore mainly change how a rating must be read, and platforms that solicit reviews should recalibrate their rating labels.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Human-Computer Interaction
Decision Models as Scalable Economic Surrogates that Capture Human Variation
How Green Is Your Manifesto? Scoring Environmental Positions in the 2024 EU Elections with an LLM Panel
Does Frontier AI Strengthen Cyber Defense? Evidence from Project Glasswing
Designing Behavioral Profiles for Language Model Simulation of Out-of-Distribution Human Behavior
You Know What You Will Like: People Predict Their Own Ratings Better Than Recommender Systems
Get the few papers worth your time in Human-Computer Interaction every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.