Audit report
A local execution-backed audit on 10 certified logic-defect cases, contrasting the verified reference patch with a generic adversarial baseline.
validated_case_rateGenerated 2026-06-18 22:40
reference-patch
trust not recorded
Detected 100%, Applied 100%, Tests 100%, Repaired 100%.
100% repaired
shallow-patch
trust not recorded
Detected 100%, Applied 100%, Tests 0%, Repaired 0%.
0% repaired
Stops at: tests failed
| Reviewer | Model | Trust | Pack snapshot | Run | Detection F-beta | Validated rate | Deterministic pass rate | Patch apply rate | Test pass rate | Structural pass rate | False positives / case |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Reference Patch | - | Trust not recorded | not recorded | 2026-06-18_18-35-35 | 1.000 | 1.000 | 100.0% | 100.0% | 100.0% | - | 0.000 |
| Shallow Patch | - | Trust not recorded | not recorded | 2026-06-18_18-35-38 | 1.000 | 0.000 | 0.0% | 100.0% | 0.0% | - | 0.000 |
The report file holds rates only, so each stage shows its rate in tenths on a 0% to 100% axis. Repaired is the share of execution-backed cases (those with runnable tests). Detected is detection recall. Controls are deterministic scripts built into the benchmark, not models or products.
arena validate benchmark_sets/audit_v2 arena run benchmark_sets/audit_v2 --reviewer reference-patch --mode full --allow-local-execution arena run benchmark_sets/audit_v2 --reviewer shallow-patch --mode full --allow-local-execution arena leaderboard runs/ --metric validated_case_rate --beta 1.0 --include-unverified arena audit-report runs/ --output docs/reports/audit_v2-results.md