arXiv preprint ·
Understanding ChatGPT Citations: Evidence from Production Web Search Across Model Generations
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Generative engines such as ChatGPT Web retrieve webpages to answer user queries, but only a subset are shown as citations. Understanding this retrieval-to-citation process is important for generative engine optimization (GEO), which aims to improve content visibility in generative-engine responses. However, prior studies mainly analyze final responses, leaving the internal execution process and its variation across model settings less understood. To this end, we conduct an observational study of four ChatGPT Web settings from ChatGPT 5.2 to 5.5, covering Instant and Thinking modes at the URL, hostname, and entity levels. We examine how ChatGPT rewrites queries, retrieves candidate webpages, and selects references. The Instant settings remain mostly single-round, while GPT-5.4 Thinking frequently performs multiple retrieval rounds. Across repeated runs, candidates retrieved more often are also more likely to be cited. We then evaluate Google Search and the OpenAI Responses API as external proxies. Responses API candidates provide broader coverage, while its citations more closely match sources cited by ChatGPT Web. Both proxies match ChatGPT Web better at the hostname and entity levels than at the exact-URL level. Even when combined, they recover only about one quarter of cited URLs. We further test whether these external signals remain predictive when combined. Within the external candidate set, a supervised model combining them reaches AUCs above 0.74 for GPT-5.3 and GPT-5.5 at all three levels. These findings suggest that GEO should distinguish retrieval visibility from citation visibility, account for rewritten queries and run-to-run variation, and consider hostname- and entity-level presence in addition to exact URLs. Differences across settings also suggest that GEO strategies should be re-evaluated when the deployed setting or serving environment changes.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Information Retrieval
Retrieving Complementary Evidence for Multi-Hop RAG with Semantic Gain Graphs
OT-FlowRec: Optimal Transport-Guided Multi-Bridge Flow for Multimodal Sequential Recommendation
UniSGR-V2: Stepwise Target-Side Feature Anticipation and Crossing for Generative Recommendation
Toward Explainability in Multimodal Recommendation
Recall Begets Recall: Recurrent Memory Ranking for Personalized LLMs
Get the few papers worth your time in Information Retrieval every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.