Scope: every system @oshun/creative-autonomy composes with, audited by reading the actual source (not docs), with each actionable gap closed in this pass and every remaining gap named honestly. Companion to V1/architecture/agentic-ai-studio.md § "The autonomous creative direction plane" and TODOS §18.15.

Verdict up front: the underlying stack is substantially real — the weaknesses were almost all integration seams that existed but were unbound, plus a handful of design gaps in the plane itself that binding exposed. All of those are now closed. What remains is exactly two kinds of gap: live provider credentials, and deeper integrations that need corpora/infrastructure that do not exist on this box.

Per-system findings#

1. @oshun/content-quality-judge (libs/shared) — REAL, now bound#

• Found: a serious judging stack: JudgeEngine.scorePointwise runs N≥2 samples per rubric dimension and reports mean + variance (judge-stability signal); comparePairwise evaluates BOTH orderings and only counts a consistent winner (the standard mitigation for LLM position bias); unparseable responses throw JudgeParseError (never a defaulted score); computeSlop/slopPenalty is a deterministic over-represented-diction detector; calibration.ts computes real Cohen's κ / Pearson against human anchors with drift checks.
• Issue found in MY plane (fixed): the compellingness panel forced every judge to score all 7 dimensions — which would have forced this binding to invent scores for dimensions the quality-judge rubrics do not measure (e.g. grounding). The panel now accepts partial scores per judge, aggregates each dimension over the judges that actually scored it, and throws IncompleteJudgePanelError when the panel as a whole leaves a dimension uncovered.
• Issue found (fixed): the engine consumes @iris/agents-core AgenticProvider while the orchestrator consumes @oshun/ai LLMProviderInterface — two provider protocols, no bridge. Built agenticProviderFromLlm (pure translation) so one env-keyed router powers planner, judges, and engine.
• Bound as: createCraftJudge (explicit per-content-type rubric-key → dimension mapping, verified against the real rubric keys — note hook exists only in the quest rubric, not commentary), createSlopNoveltyJudge (deterministic, credential-free), createQualityPairwiseJudge (tournament judge).
• Remaining (named, not actionable here): its calibration.ts κ pipeline needs a human-anchor gold set to run against; the plane keeps its own per-category calibration EMA until a deployment accumulates anchors.

2. Nyx (@oshun/domain-nyx) — REAL, now bound, credential-free#

• Found: buildNyxUpcomingAstronomicalEvents computes real upcoming events from the ephemeris (principal lunar phases ≈weekly, solar season markers, plus a source-backed featured list — eclipses, meteor showers), deterministic in now.
• Bound as: createNyxSkyEventSignalSource — events → the plane's triangular lead-time calendar model. Audience-estimate/ideal-lead priors per event kind are declared editorial constants the bandit then corrects with real human outcomes. This makes nyx.sky-event-brief the deployment's strongest end-to-end autonomous path (zero external credentials for sensing).

3. Editorial lifecycle (@oshun/studio-authoring) — REAL, now bound#

• Found: a pure, tested state machine (idea → … → published → … → archived) with tryEditorialAdvance gate enforcement; blocking authoring-evaluation findings genuinely refuse the publish transition. No store ships with it (pure functions).
• Bound as: createEditorialLifecyclePublisher — every autonomous publish walks draft → in-review → approved → published through the real machine under the deterministic Lilith content-QA gate; a blocked gate throws EditorialGateRefusedError with the missing gate ids. Autonomous content can no longer bypass editorial law.
• Issue found in MY plane (fixed): a publisher throw aborted the whole cycle. Publish failures are now isolated per item (publishFailures in the cycle report, publish-failed disposition); the item stays approved and retries next cycle.

4. Generation-job pipeline (BFF src/generation/) — REAL, now bound#

• Found: enqueue → executor → Isis release gate (evaluateGenerationRelease, fail-closed blocked without governance evidence) → released-output catalog (gallery/lineage/provenance). Real provider executors for illustration/explainer (Stability), music (Suno), narration (ElevenLabs), sky-briefing (Meshy), video (fal LTX-Video), caption-dub, accessibility-pass — all env-credential-gated, fail-closed.
• Gap found (fixed): the queue drained ONLY via the operator HTTP tick — no programmatic path. Extracted processQueuedGenerationJobs (+ getGenerationJob) so the autonomy producer drains its own jobs through the identical executor + release-gate + catalog path.
• Gap found (fixed): no text kind exists — text was unproducible. Built createLlmTextGenerator: schema-validated {title, text, citations[]} structured output with critique feedback threaded into revision passes, so grounding judges read declared citations and the revise loop genuinely revises.
• Bound as: createGenerationJobGenerators — image→illustration, audio→narration (dependency-text aware), video→video. A blocked or held job throws with its real governance reasons; the plane never treats an ungoverned output as produced.
• Remaining (named): music and caption-dub kinds are unbound (no launch category uses them; binding is mechanical when one does). Video outputs currently block at the release gate by design until the deploy-bound video safety/watermark step exists — that is the pipeline being honest, not a plane gap.

5. Lilith (@oshun/persona-policy-lilith) — REAL, now bound#

• Found: deterministic content-QA hooks (runLilithQaUnsafeClaimScan, runLilithQaVoiceAbuseLanguageScan) with severity-typed findings and a block-publish verdict; real unsafe-claim rule engine behind them.
• Bound as: createLilithToneJudge (partial judge: a blocker zeroes audience-fit; each advisory deducts a documented 0.15) and lilithEditorialGate (the publisher's required gate). Scans run under the scholar-historian/lectio posture — the least restrictive published-content pairing; category-specific postures are a wiring parameter.

6. Sophia (@oshun/evidence-sophia) — REAL, bound with an honest scope#

• Found: buildSophiaEducationalClaimGrounding — real support/contradiction counting, warning folding, and grounding-status derivation per claim.
• Bound as: createSophiaGroundingJudge — each text-bearing plan node becomes a claim whose citations are the node's own declared citations output, run through the real Sophia model; statuses map to documented score constants; citationPolicy: 'optional' supports self-grounded categories (ephemeris-derived sky briefs) at a documented 0.6.
• Honest scope limitation (named in the judge's id): this measures declared-citation coverage, not stance verification against source text. Verifying that a cited source actually supports the claim is the sophia.ground/sophia.fact_check tool path, which needs a retrieval corpus + creds; when a deployment wires those tools, a stance-verifying judge should replace/augment this one. That is the single biggest remaining quality gap for factual content.

7. Hathor (libs/hathor/narrative-generation) — REAL, deliberately unbound#

• Found: NarrativeGenerationService — a deterministic arc planner (tension models, motifs, seeded) with LLM prose per beat, provenance records, lore-consistency checks, and its own review workflow + staged volumes with STOP-on-regression.
• Decision: NOT bound this pass. Its request shape requires characters, canon, and motifs — none of the launch categories carry a character canon, so a binding today would fabricate inputs. When a narrative category with a real canon exists (e.g. a Hathor lore volume category), the binding is a straightforward DomainGenerator and Hathor's own review workflow slots in as the domain-local quality loop inside the generator, with the plane's queue staying the cross-domain human gate.

8. Agentic studio governance — REAL, now bound to the plane#

• Found: the admin agentic-operations kill-switch store is live-read by the runs route on every request; gold-set promotion machinery exists behind operator admin routes.
• Bound as: createAdminKillSwitchBinding — an armed global switch or a switch on agent-family:creative-autonomy aborts the next cycle. Gold-set promotion stays operator-owned by design (auto-promoting approvals to gold sets is a policy decision, not a default).

Plane-level SOTA gaps found by this audit (all fixed)#

1. No self-correction before parking — a gate failure parked immediately. Now: bounded gate-retry (gateRetries, default 1) re-produces with the failing dimensions and gate notes fed back into the brief; gateAttempts reported per item.
2. Forced-full-panel fabrication pressure — fixed with partial specialist panels + IncompleteJudgePanelError (above).
3. Publish failures aborted cycles — fixed with per-item isolation (above).
4. No shipped LLM defaults — every deployment had to write its own synthesizer/judges. Now shipped, all fail-loud via @oshun/ai/agent-loop: createStructuredIdeaSynthesizer, createLlmRubricJudge, and createLlmPairwiseJudge (with the same both-orderings position-bias mitigation the quality-judge engine uses).
5. No canonical text extraction — judges each needed artifact-walking logic. Now variantText/extractCitations (deterministic, conventional keys, never paraphrases).

What still stands between this and "absolutely great content"#

Honest, ordered by impact:

1. Live credentials — OPENROUTER_API_KEY binds the director end to end (server.ts logs BOUND vs fail-closed); media executors additionally need their provider keys (Stability/ElevenLabs/fal/Suno). Everything downstream of the keys is wired and tested against boundary doubles.
2. Stance-verified grounding — wire sophia.ground/sophia.fact_check over a real retrieval corpus and upgrade the grounding judge from declared-citation coverage to verified support (finding §6).
3. Human-anchor calibration — accumulate operator gold-set anchors and run the quality-judge κ calibration against the plane's judges; today the per-category EMA calibrates against review outcomes only (finding §1).
4. Engagement + search demand signals — the engagement-decay and catalog-gap sources are built and tested but need the analytics warehouse and search query-log feeds wired as inputs at deploy (real data, not new code).
5. Narrative categories — bind Hathor when a canon-bearing category exists (finding §7).
6. Video release path — the fal video executor produces, but outputs block at the Isis gate until the deploy-bound safety/watermark step lands (finding §4); autonomous video stays parked-by-governance until then.

Test evidence for this pass: @oshun/creative-autonomy 118/118 (10 files); BFF agentic + bindings + jobs 77/77; both tsc --noEmit clean.

Deep implementation review + SOTA comparison — 2026-07-02 (second pass)#

A prompt-and-implementation-level review of the plane itself, benchmarked against the disciplines SOTA agentic-content systems use (judge ensembles with anchoring and bias mitigation, non-stationary/contextual bandits, self-refine loops, injection defense, durable state, self-scheduling). Every issue below was verified against the source and fixed in this pass; the closing table names what deliberately still differs from SOTA and why.

Process bugs found and fixed#

1. Tweak-loss on production failure — the director consumed a human's tweak directive before producing; a provider outage silently swallowed the direction. Now consumed only when the regeneration actually got judged (any disposition except production-failed); otherwise it stays pending and retries. Test: a scripted outage across exactly the regeneration's produce calls.
2. Dependency-blind generators — createLlmTextGenerator ignored ctx.dependencies: a plan node depending on an outline/research node never saw the upstream artifact. Now a bounded dependencyContext (per-artifact + total clamps, identification fields first) threads into text prompts, and a one-line excerpt into image/video prompts.
3. Specialist-panel overconfidence — partial panels made single-scorer dimensions contribute std 0, inflating confidence to 1.0 with zero cross-checking. Confidence is now agreement × coverage (coverage = mean(min(1, scorers/2))): a thin panel is honestly less confident; the report exposes coverage separately.
4. Nyx provenance mis-attribution (caught by the new end-to-end test) — nyx signals carried the generic calendar-event sourceId of the mechanism they delegate to; they now carry nyx-sky-events.

Prompt-engineering upgrades (LLM-judge discipline)#

• Score anchors in the rubric ("0.9+ exceptional … 0.5 mediocre"), the standard fix for judge score-compression; hook judges the OPENING explicitly; verbosity guard ("never word count") on both rubric and pairwise judges — length bias is the best-documented LLM-judge failure after position bias (which both pairwise paths already mitigate via both-orderings consistency).
• Multi-sample rubric judging (samples option, temperature-perturbed, per-dimension means) — the same stability discipline the shared JudgeEngine uses.
• Deterministic judging clamp (clampForJudging, head+tail with an explicit elision marker) — long texts no longer overflow judge context or silently truncate the ending the structure dimension needs.
• Synthesizer dedupe context — the ideation model now sees recently produced/queued topics ("do NOT propose near-duplicates"), instead of wasting proposals the novelty scorer would kill anyway.
• Deployment-constrained planner — the bound producer tells the planner which domains actually have live generators, so plans stop naming unproducible nodes.

Algorithmic upgrades#

• Discounted Thompson sampling (decayArms, γ per cycle, default 0.98): taste is non-stationary; old evidence decays toward the prior instead of accumulating forever.
• Two-level (category, angle) arms: outcomes update an angle arm alongside the category arm; selection blends the angle draw as a bounded factor (0.5 + 0.5·draw) — the plane learns that an audience loves quick-takes but rejects deep-dives within a category. (A full contextual bandit over topic embeddings stays out — see the deliberate-differences table.)
• Trigram-augmented novelty (topicSimilarity = max of token and char-trigram Jaccard): catches morphological near-duplicates ('eclipse'/'eclipses') embeddings would catch, without credentials.
• Failed-topic cooldown: topics humans rejected/vetoed (or the gate parked) within a window (default 7d) are suppressed at candidate stage and reported (suppressedByCooldown) — the director stops relitigating fresh failures every cycle.
• Retry-After honoring: the governor extends its AIMD cooldown to the provider's own retryAfterSeconds/retry-after hint when longer.
• Topic sanitization at admission (single-line, 160-char cap): topics can originate from user-shaped input and flow into prompts; they are now data, not a prompt-injection channel (Lilith/editorial gates remain the content-level defense).

Infrastructure upgrades#

• Durable plane state (wireDurableAutonomyPlane over the same DurableSnapshotStore the jobs queue uses): review queue, bandit posteriors, learner calibration, steering document, and cycle reports survive a restart. Without this, every deploy erased everything the humans had taught the plane. Snapshot/restore round-trip is JSON-safe and tested.
• Autonomous cycle driver (createAutonomyCycleDriver, started in server.ts when the director is bound; OSHUN_AUTONOMY_AUTOCYCLE=0 disables): cycles now fire on the plane's schedule (quiet hours included) with no operator trigger — the plane is autonomous in fact, not just in name. Non-reentrancy is a single latch shared with the HTTP trigger; driver errors are logged and never kill the loop.
• Full-stack end-to-end test with only the model scripted: one content-routing fake provider serves planner, writer, artifact critic, synthesizer, rubric judge, craft judge, and pairwise judge, and the test drives real Nyx ephemeris sensing → planning → writing → the five-judge panel → tournament → gate → review queue, asserting the declared citation the Sophia judge scored.

SOTA techniques deliberately not adopted (and why)#

Test evidence for this pass: @oshun/creative-autonomy 131/131; BFF agentic 65/65 (incl. the bound-plane e2e, persistence round-trip, and cycle-driver suites); both tsc --noEmit clean.