arXiv preprint ·
Do Modifications Made by Coding Agents Make Subsequent Software Maintenance Harder?
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Coding agents increasingly maintain repositories over time, where each change becomes the starting point for subsequent tasks. Yet existing benchmarks either start tasks from developer-written code or cannot separate later-task difficulty from the effects of an agent's earlier changes. We ask whether agent-written code makes subsequent maintenance harder and whether the cost stems from inherited test failures or the code itself. We introduce SWE-Maintain, comprising 100 three-event maintenance episodes from 50 real-world Python repositories, with hidden tests certified across each sequence. Each later task is evaluated twice with the same agent and tests: from the developers' preceding patches (Developer-History) and from the agent's own changes (Agent-History). Across five coding systems, two backbone models, and 1,996 paired later events, Agent-History reduces later-task resolution by 5.6 percentage points on average. When success additionally requires all earlier tests to pass, the gap reaches 14.0 points. Two-thirds of Agent-History starting repositories already fail earlier tests, and only 1.9% of events starting from those states end with all accumulated tests passing. When the inherited code passes earlier tests and the next task is not already solved, the disadvantage disappears (+2.7 points; 95% CI [-0.5, +6.1]). These findings suggest that the main downstream cost of agent-written changes lies in inherited test failures rather than a measurable maintenance disadvantage in code that preserves earlier tested behavior. Evaluations should therefore include agent-produced history, and coding agents should preserve earlier behavior across changes.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Software Engineering
HoldSpec: A Machine-Checked Model and Conformance Suite for the Payment Authorization Hold Lifecycle
Verifying Autoware Controller Node by Combining Model Verification and Runtime Monitoring
Specification-Based Testing of Large Language Models
PassSum: Leveraging Paths of Abstract Syntax Trees and Self-Supervision for Code Summarization
Taylor-Approximation-Based Repair of High Floating-Point Errors
Get the few papers worth your time in Software Engineering every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.