arXiv preprint ·
Agentic Evolution of SQL Test Generators
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Automated testing has uncovered many logic bugs in Database Management Systems (DBMSs). Existing tools are typically limited to a few SQL dialects and a subset of these dialects' features. Coding agents based on Large Language Models (LLMs) could encode the semantics of additional SQL features into reusable test generators, requiring no additional LLM calls during testing and thus enabling high efficiency and low cost. However, a key challenge is validating the added capabilities in the presence of potential hallucinations. In this paper, we propose Gungnir, an approach for evolving DBMS test generators using coding agents. Our core mechanism is witness replay: the coding agent provides a generation witness recording the database setup and semantic choices needed to generate a target statement. A validator replays these choices through the modified generator and checks that it reproduces the target statement. Historical witnesses serve as regression tests for subsequent modifications. We evaluated Gungnir on SQLite, DuckDB, and PostgreSQL and found 44 bugs confirmed by developers, of which 34 have been fixed. Beyond DBMSs, witness replay could generalize to other systems to guide agents in extending test generators.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Software Engineering
Co-Modifying Application Code and Deployment Infrastructure: A Causal Study of AI Agent Pull Requests
HoldSpec: A Machine-Checked Model and Conformance Suite for the Payment Authorization Hold Lifecycle
Evaluation of Software Quality Through Refactoring Commit-Impact Analysis
Specification-Based Testing of Large Language Models
Vulnerable Models in the AI Supply Chain: An Empirical Study of Adoption, Propagation, and Usage Patterns
Get the few papers worth your time in Software Engineering every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.