arXiv preprint ·
Where LLMs Fail with Visualization DSLs
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
As LLMs take up the role of authoring charts using visualization domain-specific languages (DSLs), the human constraints that shaped those languages may no longer apply, as what is easy for a person is not necessarily easy for a model. To understand how LLMs might work better with DSLs, we explore where and how they fail with current DSL designs. We evaluate 10 JSON-style visualization DSLs with 41 tasks across 3 LLMs, then assess the generated specifications with JSON and rendering checks, and qualitative coding of failed cases. Analyzing how this specification generation process fails, we identify four recurring failure patterns, link each to specific DSL features, and discuss design considerations for future DSL designs.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Human-Computer Interaction
ABDA-NL: A Natural-Language Scenario Explorer for Argument-Based Reasoning
Asterism: Exploring and Synthesizing Scattered Observations into Literature-Grounded Hypotheses and Theories
Cybernetic and Epistemic: A Missing Vocabulary for Trustworthy Agentic Delegation
DataWeave: Deploying Human-LLM Analytics for Exploratory Structured Data Analysis
Scaling Peer Assessments: An Integrity Report from a Large Engineering Internship
Get the few papers worth your time in Human-Computer Interaction every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.