arXiv preprint ·
Where Language-Model Vocabulary Rose on Stack Exchange
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Stack Overflow banned answers generated by ChatGPT five days after the model's release, and many other Stack Exchange sites announced similar rules. I measure the vocabulary of the answers posted afterwards, 3.8 million answers on 170 sites from 2019 to 2024. A fixed list of 56 words that language models overuse rose about 14 percent above its pre-2023 trend, while the highest of 1,000 frequency-matched lists of ordinary words rose about 7 percent. In a screen of 15 site traits measured before the release, the strongest predictor of where the rise happened was the share of answers that received a body edit within 30 days of posting. On the least edited third of sites the list rose about 29 percent relative to matched controls, and on the most edited third it did not change. The gradient is also present among answers that were never edited. It follows the share of answers edited by their own authors, and the share edited by others adds little once that share is held fixed. Use of the model is unobserved, so these results cannot separate lower use on sites with a high self-edit share from any effect of editing on what the vocabulary records. Any vocabulary-based estimate for the network as a whole therefore pools sites where the marker vocabulary rose sharply with sites where it did not change.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Computation & Language / NLP
Calibrate, Don't Retrain: Decomposing the Value of Annotator Disagreement in Cyberbullying Severity
HyBridge: Bridge-Guided Self-Supervised Hyperbolic Hypergraph Representation Learning
Regime-Uncertain Data Transactions: A Game-Theoretic Framework for Self-Adapting and Self-Diagnosing Multi-ML-Agent Systems
Tracking What Works: Social Engineering Lures as Signatures for Victim-Centric Infostealer Campaign Discovery
Designing Institutions for Autonomous AI
Get the few papers worth your time in Computation & Language / NLP every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.