Date: 2026-09-12
Initiative: eve-sota-gap-closure-2026-09-01
Task leg: 10.4-current-release-leg
Outcome#
The exact seven release-blocked cases were rerun at k=10 through the production
assistant eval route under the authoritative V1.0 current expectation scope.
The registered OpenRouter binding used deepseek/deepseek-v4-flash-0731; Baidu
served all reported turns. The run finished 55/70, with zero provider retries
and zero observed forbidden-tool failure runs. It billed $0.0084, had a
4.9-second median turn latency, and reported 643,584 cache-read tokens over
680,407 prompt tokens on 68/70 runs.
| Case | Passes |
|---|---|
veritas-tool-selection |
9/10 |
family-md-empty-saved-articles |
2/10 |
family-md-course-continue |
10/10 |
family-md-metis-search |
6/10 |
family-md-metis-recommend |
10/10 |
family-md-empty-course-search |
8/10 |
family-md-empty-recommend |
10/10 |
This is an observational advisory result, not a promotion. Failed runs remain failures. The prior advisory reason was not edited, full-deck/family/builder floors were not evaluated, and no floor moved. The run used a seeded, run-unique PostgreSQL database and retained proof of exact teardown without retaining its identity.
Retention and controls#
The receipt retains the exact case counts, release/mode, provider/model/serving
provenance, spend, cache use, latency, retry count, source hashes, and database
lifecycle. The log retains aggregate grader diagnostics but no prompts, model
response excerpts, credential, database URL, or run-unique database identity.
The provider key remained in process memory only and the external secrets file
remained mode 0600.
The verifier recomputes receipt arithmetic, compares every per-case result and run header against the sanitized log, checks source hashes and the shipping release constant, scans the log for forbidden retained data, and verifies the advisory/no-floor/future-release boundaries. Eleven semantic mutations are retained as observed-red controls.
Remaining release gate#
V1.2 has not opened. Accordingly no restored-semantics provider run was made and no annotation was promoted or retired. Task 10.4, Phase 10, and G8 remain open until the product release changes to V1.2, the exact restored semantics are rerun at k≥10, their required floors pass, and the owners record the explicit annotation decision.