arXiv preprint ·
From Candidate Admission to Task-Level Return: Auditing LLM-Generated Code under Execution Contracts
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Admission rules retain generated programs, whereas selection policies return one candidate per task. We audit how these decisions interact with execution contracts in two separately frozen, disjoint HumanEvalPlus cohorts, each comprising 32 tasks, four samples per task per profile and two served profiles. Under the inline execution contract, a structural-and-self-test screen admits 4-9 expanded-suite nonpasses per 80-107 admissions and excludes 14-43 suite passes per 128 planned attempts. A restricted module-supply condition adds 20 and 14 passing admissions for one profile in the respective cohorts, plus one follow-up nonpass, and reduces the excluded suite passes to 14-23. All fixed first-admitted outcomes and abstentions remain unchanged. In the original pool, the added passes enter already all-passing admitted pools; in the follow-up, four of 24 common sample orders lose one passed task. Selected-subset diagnostics separately address registration-sensitive completion and reference-relative assertion disagreement, accounting for input domains and comparator tolerances. In these high-pass pools, candidate retention and task-level return diverge and are reported separately. Results are conditional on the saved pools, contracts and pinned suites; downstream training effects are unmeasured.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Software Engineering
Safety Analysis of CPSs Over-the-air Updates: Test Case Prioritization and Selection
Theory of Developer Momentum: The Developer's Experience of Progress and the Craft of Maintaining Motion
Effective Construction of Feature Database of Fingerprinting Functions for Code-Clone SCA
Over-Defense in Coding Agents Begins at Silent Forks
Backend-Frontend Converter: Component-Based Conversion of .NET Computational Models to TypeScript
Get the few papers worth your time in Software Engineering every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.