# Review of the V1–V9 Autonomous Content SOTA Audit (2026-07-15)

**Review date:** 2026-07-15

**Subject:** `V1_V9_AUTONOMOUS_CONTENT_SOTA_AUDIT_2026-07-15.md` (1,857 lines,
committed across `5f1397ad54`, `a1eaae8d97`, `9302fa49b3`)

**Reconciliation status:** this review is retained as adversarial provenance,
not as a parallel current truth source. Accepted and corrected findings are
folded into `V1_V9_AUTONOMOUS_CONTENT_SOTA_AUDIT_2026-07-15.md`; the generated
disposition ledger is
`evidence/v1-v9/AUD-008/reconciliation-status.generated.md`.

**Review method:** independent re-verification against source. Every check in
this review was executed sequentially in this session on this host (macOS
checkout, branch `oshun-mobile-parity-2026-07-04`): source files were read
directly, cited constants re-derived, the V8 docs-drift verifier re-run, two of
the report's test-count claims reproduced by running the suites, and
production-consumer searches repeated from scratch. External references were
checked against known paper/spec identifiers.

## 1. Overall verdict on the report

The report is **accurate, unusually honest, and largely reproducible**. Its
central thesis — strong components and local verification, not yet a unified
production-observed autonomous content operating system, with governance scored
at the invoked seam — survives adversarial re-checking. Of the ~30 load-bearing
claims re-verified, all but one were exactly correct, including every numeric
constant sampled across V1, V2, V4, V8, and V9. The single factual error found
(V3 gate count) was inherited from an upstream audit document, which itself
validates the report's own warning about prose truth-source drift.

The material contributions of this review are: one inherited factual error, one
class of evidence-staleness the report applies to others but not to its own
inputs, and six additions. Subsequent source review qualified two additions:
**no in-repository operator UI consumer of the V1 autonomy review API was
found**, although an external private client cannot be excluded, and the live
plane persists **partial planner token usage** but lacks complete measured cost
accounting. Those corrected boundaries sharpen, rather than contradict, the
report's P0 register.

## 2. Claims re-verified exactly (confirmatory pass)

Every item below was checked against primary source in this session, not against
the report or the July per-version audits.

| #   | Report claim                                                                                                                                         | Verified evidence                                                                                                                                                                                                                                                                               |
| --- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 1   | Four V1 launch categories                                                                                                                            | `apps/oshun/bff/src/agentic/autonomy-route.ts:69-95` — exactly `veritas.story-explainer`, `nyx.sky-event-brief`, `nisaba.study-passage`, `tara.seasonal-ritual`                                                                                                                                 |
| 2   | `authorizeOperatorAction` / `retryWithChanges` / `operatorStep` have no app consumer                                                                 | Only match outside the package is `apps/bellona/remote-host/src/macos-host-support.ts:422` — an unrelated `operatorSteps: readonly string[]` field (manual setup steps), a false positive                                                                                                       |
| 3   | `@yemaya/human-override` has no app/service import                                                                                                   | Zero matches across `apps/` and `services/`                                                                                                                                                                                                                                                     |
| 4   | Runs-route checks a pre-call budget but never consumes it, never appends tool/cost ledger entries                                                    | Full chain read: `runs-route.ts` → `orchestrator.ts` → `dispatcher.ts` → `executor.ts`. `admitToolCall` returns `run: input.run` unmutated; `consumeBudget` is never called by orchestrator, dispatcher, or route; the same `BudgetAdmissionInput[]` (same meter) is re-checked per step        |
| 5   | Runs-route does not invoke grant resolver / egress guard / secret scope / isolation                                                                  | Route imports only `runGuardedToolPlan` + `emptyCostLedger` from the studio package                                                                                                                                                                                                             |
| 6   | Hathor `_safetyConfig` accepted but unused; `confidenceScore` constant `0.85`                                                                        | `libs/hathor/llm-npc/src/dialogue/dialogue-generator.ts:84,114`                                                                                                                                                                                                                                 |
| 7   | V8 case-report "signature" is an unkeyed SHA-256                                                                                                     | `libs/yemaya/case-verifier/src/report.ts:48-58` (`signReport` = `createHash('sha256')` over canonical JSON; docstring is honest about it)                                                                                                                                                       |
| 8   | V8 asset binding is locator-hashed and honestly labeled; the aggregate drops the labels and overclaims byte scope                                    | `libs/yemaya/case-assets/src/bind.ts:61,71-72` (`sha256(url::seed)`, `hashScope: 'locator'`, `signed: false`) vs `libs/yemaya/case-pipeline/src/provenance.ts:16-19,60` (comments claim the hash covers "the asset bytes"; `hashScope`/`signed` not carried)                                    |
| 9   | V8 StageJournal sequence/attempt counters are process-local                                                                                          | `libs/yemaya/case-pipeline/src/journal.ts:48-50` — instance-private `Map`s                                                                                                                                                                                                                      |
| 10  | V8 HITL edits are free-form records with no apply/re-gate/resume                                                                                     | `libs/yemaya/case-pipeline/src/hitl.ts:17,37-38` — `edits?: Record<string, unknown>`, no application code                                                                                                                                                                                       |
| 11  | V8 docs-drift gate is red                                                                                                                            | Re-ran `node scripts/verify-v8-docs-drift.mjs` in this session: still fails, `v5-target.generated.ts` stale                                                                                                                                                                                     |
| 12  | V9 Python mastery heuristic: `+0.03`/`−0.05`, `correct_attempts` on no-marker, `0.6*prior + 0.4*score`, `+0.08` confidence even on failed assessment | `apps/metis/service/src/metis/services/learner_mastery_helpers.py:144,153,175,179` — all four exact                                                                                                                                                                                             |
| 13  | V9 rich generator returns the retry "as-is even if it also fails"                                                                                    | `libs/metis/llm-client/src/content-generator/content-generator.ts:1164-1190` (the code comment says so verbatim)                                                                                                                                                                                |
| 14  | V9 agent runtime plans DAGs but has no executor                                                                                                      | `apps/metis/service/src/metis/services/agent_runtime_service.py` — logs "planned with N DAG nodes"; no execution path                                                                                                                                                                           |
| 15  | Sophia binding scores declared citations without retrieval; rights auto-satisfied                                                                    | `apps/oshun/bff/src/agentic/autonomy-bindings/sophia-judge.ts` (hardcodes `stance: 'supports'`, `relevance: 0.8`; judgeId honestly says `declared-citations`); `publish-readiness-gates.ts:87` unconditionally `satisfied.add(EDITORIAL_GATE_IDS.rights)`                                       |
| 16  | Media critic throws on video without a still frame; no generator populates one                                                                       | `media-critic.ts:94-119` (its own comment says "none" populate `thumbnailUrl`/`posterUrl`/`keyFrameUrl` today)                                                                                                                                                                                  |
| 17  | CQJ has a real untrusted-content fence; critique/revise does not                                                                                     | `libs/shared/content-quality-judge/src/judge-engine.ts:43-60` (fence markers + immunity clause + sanitizer) vs zero fence matches in `libs/oshun/creative-orchestrator/src/critique-revise.ts`                                                                                                  |
| 18  | V6 runs/budgets/quarantine are in-memory                                                                                                             | `libs/iris/agents/core/src/agentic/agent-run.ts:102,163,225-227,521` — kill switches, failures, runs, controllers, quarantine, cost caps all instance Maps                                                                                                                                      |
| 19  | Director bandit decay `0.98`; ideation temp `0.8` / 2,000 tokens / 30-topic cap                                                                      | `director.ts:189,290`; `llm-synthesizer.ts:106,110-111`                                                                                                                                                                                                                                         |
| 20  | Theseus skill model `p = 0.35 + 0.65·s`                                                                                                              | `libs/yemaya/case-eval/src/solver.ts:83`                                                                                                                                                                                                                                                        |
| 21  | FSRS-5: 19 canonical weights, `FACTOR = 19/81`, `R(S,S) = 0.9`                                                                                       | `libs/metis/adaptive/src/path/fsrs.ts:11-54` — exact, including the derivation comment                                                                                                                                                                                                          |
| 22  | V4 director `0.42·damage …` and procgen `Seed + Attempt*7919` are real UE C++                                                                        | `V4/ue/Source/V4Gameplay/Private/V4AIDirectorSubsystem.cpp:175`; `V4/ue/Source/V4Procgen/Private/V4ProcgenSubsystem.cpp`                                                                                                                                                                        |
| 23  | V2 latency decomposition 420/240/…/40 ms                                                                                                             | `apps/v2/iris-commentary-orchestration/src/v2-ai-commentary.ts:24-27`                                                                                                                                                                                                                           |
| 24  | 28 documented generation surfaces                                                                                                                    | `V1/architecture/generation-audience-tiers.md:71-100`                                                                                                                                                                                                                                           |
| 25  | `V_SERIES.md` dated 2026-06-12; capability matrix last updated 2026-06-18                                                                            | File headers confirm both                                                                                                                                                                                                                                                                       |
| 26  | Verification record: `@oshun/creative-autonomy` 248/248                                                                                              | Reproduced this session: 16 files, 248 passed                                                                                                                                                                                                                                                   |
| 27  | Verification record: V9 176 tests vs ledger 173                                                                                                      | Reproduced this session: 159 across `libs/v9/*` + 17 in `libs/contracts/src/v9` = 176; `V9/V9_TODOS.md:35` still says 173                                                                                                                                                                       |
| 28  | External references                                                                                                                                  | All arXiv IDs (Self-Refine 2303.17651, Reflexion 2303.11366, ToT 2305.10601, LATS 2310.04406, GEPA 2507.19457, AgentRewardBench 2504.08942, GenEval 2310.11513, VBench++ 2411.13503) match the real papers; Anthropic/OpenAI/LangGraph/OTel/NIST/MCP/C2PA links are the correct primary sources |

The report's most consequential architectural finding — that the guarded
tool-plan chain enforces kill switches and pre-call budget checks but the
envelope is threaded through unmodified, so the persisted "executed run" never
accumulates tool calls, evidence, or cost — is correct at every level of the
call stack.

## 3. Errors and staleness found in the report

### 3.1 V3 launch-gate figure is wrong (inherited, not re-derived)

The report states (§6.3, §3 examples) that
`V3/launch/launch-readiness-release-gate.json` is "red at 29/40 with
`gaReady: false`". The actual file — unchanged in git since commit `8095244e26`
on 2026-06-12 — reads:

```
status: red, requiredGateCount: 41, passingGateCount: 30,
manifestNamedGateCount: 39, passingNamedGateCount: 24, gaReady: false
```

Neither 30/41 (required) nor 24/39 (named) is 29/40. The figure was copied from
`V3_AUTONOMOUS_CONTENT_AGENTS_AUDIT_2026-07-14.md:19`, which is itself wrong
about the file it cites. The substance (red, not GA-ready) holds, but this is a
concrete failure of the report's "corroborated" evidence class: the upstream
audit was trusted without opening the referenced artifact. Both documents should
be corrected to 30/41 required (24/39 named).

### 3.2 The V3 gate evidence is itself a stale truth source

The same JSON's `generatedAt` was `2026-06-12T19:03:08Z` — over a month old at
review time. The report (correctly) flags `V_SERIES.md` and the capability
matrix as stale truth sources, but treated the stored V3 gate file as current
fact. That timestamp proved the consumed artifact had not been rewritten; it did
**not** prove that nobody ran the verifier without `--write-evidence`.
Machine-generated evidence rots exactly like prose. The report's P0-06
acceptance criterion should add: **evidence artifacts carry generation
timestamps and a max-age policy, and a gate consuming evidence older than its
freshness SLO is itself red.** AUD-001 subsequently regenerated this record and
added a 24-hour freshness contract.

### 3.3 Host-ambiguous remediation advice

§6.4 says "this repository has a local UE5.5 toolchain and should use it for the
closure slice." That is true of the Linux sandbox
(`/root/workspace/UnrealEngine-5.5/`) and false of this macOS checkout, which
has no UE toolchain and cannot build one practically. The recommendation is
right but should name the execution host; a reader on this Mac cannot act on it.

Resolution: AUD-004 defines the `ue55-linux` self-hosted execution profile,
records missing local prerequisites as `not-run-on-this-host`, and provides a
dispatchable V4 remote job plus a required executed-evidence schema. A missing
binary on a Mac or another checkout is no longer treated as project-level
unavailability.

### 3.4 Verification record is not reproducible as written (minor)

§1.4 reports pass counts without the commands, package boundaries, or the HEAD
commit they were measured at. Reproducing "V9 packages plus V9 canonical
contracts — 176" required reverse-engineering the composition (12 suites under
`libs/v9/*` = 159, plus `libs/contracts/src/v9` = 17). For a report whose thesis
is typed, non-substitutable proof, its own evidence table should carry the exact
commands and the commit hash. (The counts themselves reproduced exactly; this is
a reproducibility gap, not an accuracy one.) The §1.4 environment description
("15 GiB RAM, 31 GiB swap") is likewise unverifiable after the fact on this
dynamically-swapped macOS host.

Resolution: AUD-003 now records the immutable source commit, exact commands and
package boundaries, runtime and host observations, authenticated bounded logs,
and a generated canonical table. Historical resource observations are explicitly
not generalized to another host.

## 4. Material additions the report missed

### 4.1 No in-repository operator UI consumer was found

Production-consumer search across `apps/oshun/web`, `apps/admin-mobile`, and
every other in-repository app found **zero consumers of
`/v1/agentic/autonomy/*`**. The matches for "review queue" in the web app
(`EvidenceReviewQueue.tsx`, lilith-studio) are unrelated features. The API and
its tests prove a real component, but the repository contains no human-usable
operator surface that drives it. An external private client is possible; no
repository or deployed-consumer evidence was available to prove one.

The report's own §15.1 principle — "a control does not govern a run merely
because both exist in the same package" — applies symmetrically to human
authority: a tested approve endpoint with no discovered operator consumer is a
component, not proof of an operating human-review capability. The scorecard's
"Strong in V1 output review" and V1 Human-control = 2 deserve that
qualification, and **P0-03 should explicitly require a human-usable operator
surface exercised end to end**, not only durable takeover semantics at the API
layer.

### 4.2 The live autonomy plane has partial token usage, not full cost accounting

The report's P0-09 correctly targets the Agentic Studio route's non-consumed
budget meter. The **live** V1 loop has a related but narrower gap:

- Portfolio selection enforces a knapsack over `idea.estimatedCostUnits` against
  `cycleBudgetUnits`
  (`libs/oshun/creative-autonomy/src/portfolio/selection.ts:180,202`) — a
  **planning-time estimate in abstract studio units**.
- `CycleReport.realTokenUsage` accumulates `run.planning.usage` for produced
  variants, is returned by the cycle route, retained in `cycleReports`, and
  persisted with the autonomy-plane snapshot. Director tests assert its numeric
  token/call totals. This is real, but principally planner-side accounting.
- Generator/research-loop, critic, judge, tool, media-provider, GPU, and
  monetary cost are not represented end to end, and there is no complete
  artifact-level provider-cost ledger.
- The concurrency governor bounds rate (`maxConcurrent: 2`,
  `launchesPerMinute: 30`), not spend.

An always-on autonomous loop whose full-cycle economics remain partly estimated,
with no complete measured-cost ledger, no daily/monthly monetary cap, and no
feedback from actual full-path cost to `estimatedCostUnits`, cannot bound its
own burn or report complete cost per accepted artifact (§13 requires exactly
that dashboard). **P0-09 should require complete token, provider, GPU, tool, and
monetary accounting, while retaining `realTokenUsage` as partial evidence.**

### 4.3 In-code truth-source drift: the runs-route documents behavior it does not have

`apps/oshun/bff/src/agentic/runs-route.ts:289-293` says:

> "C9: persist the EXECUTED envelope (tool calls, evidence, cost, final status)"

As §2 item 4 above verifies, the envelope never accumulates tool calls,
evidence, or cost — the persisted "executed run" always carries empty
`toolCalls`, `evidenceTrail`, and `emptyCostLedger()`. The comment is the same
defect class as the stale umbrella docs (P0-06), but **inside the enforcement
code itself**, where a future maintainer or auditor will trust it. Same class:
`libs/yemaya/case-pipeline/src/provenance.ts:19`'s "sha-256 of the asset bytes
the claim covers" over what may be a locator hash (the report caught this one).
P0-06's registry/drift-CI remedy should include doc-comments on enforcement
seams, or at minimum these two comments should be fixed when P0-09/P0-11 land.

Resolution: AUD-005 now states the route boundary consistently in the executor,
dispatcher, BFF route/store, and V1 architecture. AUTH-002 now closes the V1
executed-route runtime gap: its authoritative boundary persists pending and
completed/denied attempts, grant decisions, evidence/effects, measured usage,
and kill/budget audit events before another admission. V8 case provenance now
carries `hashScope` and signed state so a locator digest cannot become an
asset-byte claim by omission. Both boundaries have contract-generated field
references, real-path assertions, and one CI drift verifier spanning enforcement
comments, schema docs, and umbrella prose.

### 4.4 Hathor also fabricates `memoriesTriggered`

Alongside the constant `confidenceScore: 0.85` the report flagged,
`dialogue-generator.ts:113` returns `memoriesTriggered: []` unconditionally —
the response contract advertises memory integration that never occurs. Trivial
individually, but it belongs on the same remediation line item (§19.2 step 4) so
the fix covers the whole response envelope, not one field.

### 4.5 P0-01 could be seeded today from in-repo material

The report is right that no machine-readable coverage contract exists, but it
under-sells the raw material already present: the frozen 28-surface catalog
(`GENERATION_SURFACES` in V1), the four launch categories, the per-version TODOS
ledgers, and V9's gate contracts are all extractable. Phase A step 2 ("import
every launch requirement") should start as a mechanical extraction from these
sources rather than a blank-page inventory; that converts P0-01 from a design
task into a data-migration task with a measurable day-one count of registered vs
unregistered surfaces. The report criticizes the absence of a ledger without
seeding its first rows.

### 4.6 Declared production topology is auditable in-repo and bears on P1-08

The repository declares V1's canonical production substrate as a single Hetzner
box (compose + Caddy + GHCR/SSH CI). A live topology probe was not available in
this review, so the deployed environment cannot be asserted to match that
declaration. The report's P1-08 (exactly-once publication, checkpoint
versioning, disaster recovery) and the "Live proof" scorecard column never
engage with the declared design: if deployment matches it, the durable stores,
the BFF, and the cycle driver share one failure domain. That design risk is
auditable without production access, while confirming the live topology still
requires a probe. It materially changes what "durable across restart" must mean
(box loss, not just process loss). A one-paragraph topology finding plus a
backup/restore drill requirement would close this.

### 4.7 Smaller absences (flagging, not faulting)

- **Locale coverage of the live loop:** the four launch categories generate (as
  far as the code shows) untranslated single-locale content; the coverage
  contract models locales but the report never states the current live locale
  count. First registry import should record it.
- **Accessibility of generated artifacts:** R6 lists accessibility, and V2's
  caption contracts are praised, but no finding states whether V1's published
  text/image/audio artifacts carry alt text/captions today.
- **Retention/access policy for V1 review snapshots:** §18.3 rightly flags V6
  prompt logs and V9 learner data; the V1 durable review/learning snapshots
  (full artifacts + human feedback + standing direction) deserve the same
  retention/access treatment.
- **Test-coherence sampling:** the report's evidence ladder credits passing
  suites (e.g., 40/40 BFF autonomy tests) without sampling whether those tests
  assert domain correctness versus shape. Its conclusions don't depend on this,
  but the repo's own audit standard (adversarial verification, test coherence)
  would apply it.

## 5. Judgment calls I checked and endorse

- **Scoring governance at the invoked seam** (§15.1, §19.1-1) is the right
  standard and the deep pass applies it consistently — the runs-route and V9
  agent-runtime findings would be invisible under a package-level audit.
- **The four-way citation taxonomy** (§19.1-2: citation-shaped string / existing
  ID / retrieved passage / entailed passage) matches exactly what the code does
  in different places (Sophia declared-citations vs V9 lexical entailment vs V9
  citation-shape check) and is the correct P0-11 framing.
- **The hash-vs-signature distinction** (§19.1-3) is precisely borne out by
  `report.ts` and `bind.ts` vs the aggregate.
- **Not penalizing honest fail-loud seams** aligns with the repo's own
  quality-standards escape-valve doctrine; the report correctly treats V2's
  refusing translation path and V8's `requireSafetyClear` as strengths.
- **Phase ordering (A→E) and V8-first for level 4** are sound. One note: the red
  V8 docs-drift gate is a minutes-scale fix
  (`node libs/yemaya/case-contracts/tools/gen-v5-target.mjs` per the verifier
  output — regenerate and commit); it should be done immediately rather than
  waiting for Phase A, since it currently blocks the strongest product's release
  gate for a mechanical reason.

## 6. Consolidated recommendation deltas

Concrete edits to the report's register, in priority order:

1. **P0-03 (amend):** add "a human-usable operator surface exists and drives
   approve/reject/veto/tweak/direction end to end" — no in-repository consumer
   was found, and any external consumer still needs deployed evidence (§4.1).
2. **P0-09 (amend):** retain persisted `CycleReport.realTokenUsage` as partial
   planner evidence; require complete token/provider/GPU/tool/dollar accounting
   per cycle/artifact and a hard spend cap on the live plane (§4.2).
3. **P0-06 (amend):** correct the V3 figure to 30/41 (both in this report and in
   the V3 audit doc); add evidence-age/freshness SLOs for machine-generated gate
   artifacts (§3.1–3.2); include enforcement-seam doc comments in drift scope
   (§4.3).
4. **§1.4 (amend):** record exact verification commands + HEAD commit; note the
   host each recommendation assumes (UE closure slices run on the Linux box)
   (§3.3–3.4).
5. **Phase A (amend):** seed the coverage registry mechanically from the
   28-surface catalog, launch categories, and version TODOS (§4.5).
6. **P1-08 (amend):** add the repository-declared single-box topology, require a
   live topology probe, and add a box-loss restore drill (§4.6).
7. **Immediate mechanical fix:** regenerate `v5-target.generated.ts` to green
   the V8 docs-drift gate; fix the two overclaiming comments
   (`runs-route.ts:289`, `provenance.ts:19`) and Hathor's fabricated
   `confidenceScore`/`memoriesTriggered` pair (§4.3–4.4).

## 7. Bottom line

The audit is the most reliable single document about this portfolio's
autonomous-content reality that the repository currently contains. Its verdict
("SOTA-shaped at component and local-verification layers; not yet a unified,
production-observed autonomous content operating system") is supported by the
evidence and survived every adversarial re-check attempted here except one
inherited number. The additions above do not soften that verdict — they tighten
it: no in-repository human operator consumer was discovered, and partial planner
token counts do not amount to full-path economic governance. Both strengthen the
case for the report's central prescription: build the coverage contract, the
durable run spine, and the composed human-control plane before adding any new
generator.
