arXiv preprint ·
When Is RTL Better? Auditing Evaluation Stability with Tool-Grounded Requirement Profiles
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
RTL-generation studies increasingly report functional correctness and post-synthesis quality, yet the evaluator itself is rarely treated as an experimental variable. A candidate can appear better when a security rule, protocol obligation, target context, feasibility gate, uncertainty policy, or provenance constraint is omitted. We introduce tool-grounded requirement profiles: machine-readable contracts binding each requirement's meaning to its executable evidence and comparison rule, together with cross-requirement gates, uncertainty handling, and provenance. We audit evaluator stability by freezing candidate pairs and perturbing only their evaluation profile. In a preregistered controlled pilot with six designs, twelve pairs, three requirement families, six evaluator conditions, and 72 decisions, a generic cell-count evaluator reports unsupported improvement for 6/12 pairs (50%; Wilson 95% interval 25.4 to 74.6%; design-cluster bootstrap interval 25 to 75%). The matched evaluator detects all twelve injected regressions, whereas syntax, generic synthesis, context mismatch, and gate ablation each lose the required distinction for 12/12 pairs. In a separately frozen external validation of all 290 public RTLLM v1.1 text-only candidates, 81/215 evaluable comparisons (37.7%; task-cluster 95% interval 24.8 to 51.4%) yield unsupported scalar improvements, including 72 functional-gate reversals. Separately, 26 OpenROAD-flow-scripts runs show why timing context and activity qualification must be part of the contract. The study does not rank generators; it establishes evaluator configuration as a measurable, auditable source of conclusion instability.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Hardware Architecture
SafeDSE: Joint Exploration of Safety Architecture Patterns, Mapping, and Scheduling for Dataflow Applications With Safety Arguments (Extended Version)
HILO: Enabling Low-power, Dual-Band Communication using Tunnel Diode Oscillators
FORGE: Procedural Weight Generation for Sparse MoE Inference
Where the SRAM Goes: Per-Symbol Attribution of Static Memory in FreeRTOS-C and Embassy-Rust IoT Firmware on a 64 KB RISC-V Microcontroller
LocalTrain: Integer-Only Local Learning for Energy-Efficient CNN Training on Edge FPGAs
Get the few papers worth your time in Hardware Architecture every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.