Disciplines · Audits

Cross-Modal Creative-Ledger Lessons — Generalizing the Long-Form Video Playbook (2026-07-04)

The video implementation deliberately separated the modality-independent skeleton from video-specific anatomy.

13sections22 minread

On this page

Scope: AGENTIC_LONGFORM_VIDEO_LESSONS_2026-07-04.md researched and then implemented (same day, see its implementation addendum) the agentic long-form video playbook: compositional generation from pieces, granular per-piece evaluation and surgical re-generation, storyboarding as the load-bearing artifact, and hierarchical JSON scene ledgers as the compression agents navigate. This document answers the follow-on question: do those techniques and lessons generalize to images, audio, songs, 3D models, speech, worldbuilding, storylines, and the rest of the creative surface — and what should we build for each?

Answer up front: yes, almost all of it generalizes — because the ten lessons were never really about video. They are about managing any expensive, stochastic, multi-step creative process with an agent in the loop. Video merely forced the discipline first (highest cost per attempt). Only two things change per modality — what an anchor is and what a seam means — plus one structural change (sequence → graph) for narrative and worlds.

Honesty note on sources: the video sections rest on the primary-source research passes recorded in the lessons report (arXiv/GitHub/vendor docs, UNVERIFIED items flagged there). The cross-modal mapping below is analysis over that research plus our own in-repo systems (each named library was inspected in prior audited sessions). Where a recommendation depends on an external capability we have NOT wire-verified (marked wire-verify first), verification precedes authoring — the same discipline that preceded every catalog workflow we shipped.


1. The universal core (modality-independent, largely already built)#

The video implementation deliberately separated the modality-independent skeleton from video-specific anatomy. These pieces operate on nodes with recipes and measurements, not on pixels, and transfer as-is:

Piece Where it lives What is video-specific in it
Ledger pattern: intent / recipe / artifacts / measurements / index / lineage per node; retakes archive, never erase; explicit seeds with conflict protection libs/isis/workflows/src/ledger/scene-ledger.ts Only field names (clipUri, temporal-QA summary shape)
Critic→regen loop: deterministic detectors as hard floor → scaffolded judge with neighbor context → regenerate worst-N under lifetime caps → invalidate stale seam measurements → failures reported, never disguised ledger/shot-regen-loop.ts Only the detector (temporal QA) and the seam definition
Candidate budgeting: importance × uncertainty water-filling, exact totals, floors/caps, unspent reported ledger/candidate-allocation.ts Only deriveAllocationSignals heuristics
Stage gates on autonomy levels ('human-approval' / 'review-window' / 'full-auto'); first assemblies always reviewed ledger/gate-policy.ts Stage names only
Index-not-pixels digest: budget-bounded hierarchical text view; structure never dropped, descriptions shrink first ledger/media-index.ts Nothing material
Reference bank: validated entries, explicit per-workflow input bindings, explicit-inputs-win injection ledger/reference-bank.ts Entry kinds (character/style) — extend, don't rewrite
Bounded critique→revise with best-draft selection and honest met:false @oshun/creative-orchestrator critique-revise.ts Already artifact-generic
Real timeline assembly + CIE76 seam measurement; audio stems are first-class @shared/encoding/timeline-assembly.ts Works for AV; audio-only timelines need a small extension (below)
ε-BT pairwise tournaments, Thompson bandit portfolio, calibrated multi-dim gate, review queue, preference learner @oshun/creative-autonomy Already modality-agnostic at the idea/variant level

The two genuinely modality-specific axes:

  1. Anchors — the cheap-to-audit artifact you gate BEFORE spending on the expensive fill. Video: keyframe stills. Everything else has its own (see per-modality sections).
  2. Seams — where independently generated pieces meet, and what "drift" means there. Video: color/motion continuity at cuts. Music: key/tempo/ loudness across sections. Story: contradictions across chapters. 3D: texture/scale/style boundaries between assets.

The one structural difference: video is linear-temporal, so its ledger is a sequence and seams are boundaries in time. Stories and worlds are graphs (a character appears in chapters 2, 7, 9; a location constrains three plotlines) — the ledger generalizes from sequence to DAG and the seam measurement becomes constraint violation across an edge. 3D scenes are spatial hierarchies — and there the industry already has the ledger (USD composition), so the lesson is adopt and wrap, not invent.

The ten research lessons, restated modality-neutrally:

  1. Anchor-first hierarchy: plan top-down in cheap anchors, fill bottom-up with expensive synthesis; the anchor set IS the plan.
  2. Piece-wise generation with continuity conditioning (the previous piece's tail state conditions the next).
  3. Anchor/reference pinning against drift at long horizons.
  4. Reference-bank identity locking (characters, voices, motifs, materials, canon entities).
  5. Critic-gated per-piece retry — regenerate the node, not the work; escalate downward (region → node → group) before any full re-roll.
  6. Detectors first, decomposed dimensions, tournaments over lone zero-shot judges; score pieces in neighbor context.
  7. The outline/board is the load-bearing artifact and the human gate.
  8. Own your ledger schema; treat vendor payloads as compilation targets (no cross-tool standard is coming).
  9. Dual provenance: regen-exact recipes vs authenticity marking.
  10. Index-not-pixels navigation: agents hold the digest, drill into raw media on demand; the plan-side tree and review-side index are the same tree.

2. Images & image series#

The least novel case — image tooling invented half these patterns first (inpainting IS surgical regen; ControlNet IS anchor conditioning) — and our stills tier already shipped most of it in the 2026-07-03 remediation.

Lesson mapping. Anchors: thumbnails/comps and structure maps (depth/ pose/edges) before full-res; for a SERIES, the style bible + character refs are the anchors. Seams: for single images, spatial (inpaint/outpaint boundaries — our differential-diffusion chain exists precisely to blend them); for series (NFT collections, campaign sets, storyboard frames), the seam is cross-image consistency: palette/style/identity drift across the set. Surgical regen: region inpaint (composition/inpaint-edit), then whole-image re-roll, then set-level re-roll — the downward-escalation order. Detectors: artifact detection, aesthetic scores, golden perceptual regression (consistency/golden-output-perceptual-regression), identity drift (identity-drift.ts). Reference bank: shipped and bound (reference-bank.ts → FaceID/IPAdapter inputs).

Already own: the full composition/ family (union ControlNet, regional prompting, style-reference-set, inpaint/outpaint, image-variation), FaceID identity chain, batch pipelines, calibrated image gates + moderation in the BFF, per-workflow taxonomy with controlModalities.

Build recommendations:

  • I1 (P1) — Series ledger profile: a thin ImageSetLedger usage of the existing scene-ledger (nodes = images, scene = set), with set-level seam measurement = pairwise palette/style/identity drift (CIE76 palette delta reuses the assembly module's Lab math; identity via existing identity-drift scorer). Drives the shipped critic→regen loop for collections and campaign sets.
  • I2 (P1) — Anchor-first for stills sets: thumbnail-tier pass (fast, low-res or Z-Image-Turbo) gated before full-res hires-fix renders — the keyframe-first planner already expresses this; add an image-set preset (defaultKeyframeWorkflowId = fast tier, stage B = hires tier).
  • I3 (P2) — Set-level style lock verification: a deterministic set-consistency detector (palette histogram + CLIP-embedding dispersion via the sophia embedder seam) recorded as ledger measurements.

3. Music & songs#

The closest structural cousin to video: linear, sectioned, multi-track, with an industry-native decomposition (sections × stems) that maps 1:1 onto the ledger.

Lesson mapping. Hierarchy: song → sections (intro/verse/chorus/bridge/ outro) → stems (drums/bass/harmony/lead/vocals). Anchors: motif/hook first — audit an 8-bar hook before arranging 3 minutes (the exact keyframe-first economics: a rejected hook costs a fraction of a rejected arrangement). Chaining: section transitions with carried key/tempo/ instrumentation state; seam = key clash, tempo drift, loudness jump, timbre discontinuity at section boundaries. Surgical regen: re-roll ONE section (keeping neighbors) or ONE stem (keeping the mix) — never the song. Detectors first: we already have real BS.1770 loudness, limiter, de-esser DSP (isis audio_dsp nodes); add key/tempo estimation and clipping/spectral-artifact checks; LMM judging of audio is even weaker than video judging — tournaments + detectors only. Reference bank: leitmotif bank (audio refs) + artist-style entries; ACE-Step 1.5 already exposes bpm/key/time-signature control inputs — those are the lock surface. Storyboard analog: the song sheet (section list with bars, key, energy curve, lyric slots) is the load-bearing human gate.

Already own: ACE-Step 1.5 + Stable Audio 3 catalog workflows (bpm/key/timesig controls), audio_dsp detectors, music-generation, foley-studio, av-narrative-studio stem vocabulary (dialogue/music/effects/ambient), the euterpe SOTA DAW (sections, tracks, mixing — the natural host for the song ledger), assembly module stem mixing, encodeWavPcm16/ffmpeg audio path.

Build recommendations:

  • M1 (P0 for this modality) — Song ledger: nodes = sections, node payload = per-stem artifacts; recipe = catalog workflow + bpm/key/ timesig locks; seam measurement = key/tempo continuity + inter-section loudness delta (extend audio_dsp with a key/tempo estimator — wire-verify library choice first; librosa-class chromagram methods are the standard). This is the recommended FIRST adoption of the shared spine outside video: sequential, all pure pieces reusable, euterpe provides the editing surface.
  • M2 (P1) — Hook-first pipeline: generate K hook candidates (budget allocator; the chorus gets the candidates), tournament-judge them (ε-BT exists), gate, then arrange full sections conditioned on the winning hook. Needs a "continue/extend from audio" workflow — ACE-Step's repaint/extend capabilities are the likely vehicle (wire-verify exact node contracts before authoring, exactly as we did for LTX/VACE).
  • M3 (P1) — Section-surgical regen: the shipped critic→regen loop pointed at song-section nodes; regen delta = same seed policy + carry boundary bars from neighbors as conditioning where the model supports it (wire-verify); re-mix via assembly.
  • M4 (P1) — Audio-only timelines in timeline-assembly: small extension — today the module assumes a video stream; add an audio-only mode (concat/acrossfade of sections + stem mix + loudness measurement pass reusing the existing two-pass loudnorm posture: measure, never silently "fix").
  • M5 (P2) — Leitmotif reference bank kind: motif entries (audio URI
    • key/bpm metadata) with bindings into audio workflows' reference inputs when models with audio-reference conditioning land (LTX-2's LTXVReferenceAudio speaker-identity node is the first shipped hook).

4. Audio SFX / soundscapes / foley#

Different enough from songs to note separately: no harmonic seams, but event alignment (foley hits matching visual/story events) and bed/ layer composition.

Lesson mapping. Hierarchy: soundscape → beds (ambient loops) + events (one-shots). Anchors: the event LIST (timestamped cue sheet) — literally the storyboard lesson; audit the cue sheet, not the audio. Seams: loop points (bed must loop seamlessly — a real detector: cross-correlation at the loop boundary) and event-to-timeline alignment. Surgical regen: re-roll one cue. Detectors: loudness (have), loop-seam correlation, silence/clipping checks.

Already own: SFX catalog workflow (Stable Audio 3 SFX), foley-studio, audio_dsp, av-narrative effects/ambient stems, video-probe machinery for the paired-video case.

Build recommendations:

  • F1 (P1) — Cue-sheet ledger profile: nodes = cues with timestamp ranges (the ledger's timestampRange index field already exists); assembly = place cues over beds via the stem mixer.
  • F2 (P2) — Loop-seam detector: boundary cross-correlation measurement recorded as the node's seam metric for beds.

5. Speech / narration / dialogue#

Lesson mapping. Hierarchy: production → segments → utterances. Anchors: script + a 5–10s voice sample per character — both cheap to audit; batch TTS only after both gates pass (the automate-then-gate matrix applies verbatim). Seams: prosody/energy/pacing continuity across segment boundaries, and voice identity drift across a long production. Surgical regen: re-render ONE utterance (TTS is cheap enough that per-utterance retakes are the default, not the exception — the budget math inverts vs video, but the loop is identical). Detectors first — and speech has the best deterministic detector in any modality: ASR round-trip WER (synthesize → transcribe → align to script), plus duration-vs-expected, loudness, silence-gap checks. Reference bank: voice entries (the bank schema already reserved voiceRefUri).

Already own: real local TTS (Piper/espeak/Kokoro incl. the Kokoro worker node), voice-cloning, visual-dubbing, caption/dub executors with dub-faithfulness checks in the BFF, ElevenLabs narration path (fail-closed), av-narrative dialogue stems.

Build recommendations:

  • S1 (P1) — Utterance ledger + WER floor: nodes = utterances; recipe = TTS engine + voice ref + text; measurement = ASR round-trip WER as the deterministic hard floor (the temporal-QA analog). We have no in-repo ASR — the seam stays injectable and fail-closed until a deployment binds one (wire-verify Whisper-class local options for the worker).
  • S2 (P1) — Voice bank activation: populate voiceRefUri entries and bind them into TTS/cloning workflow inputs via WORKFLOW_REFERENCE_BINDINGS (the table exists; add the rows as the speech workflows join the catalog).
  • S3 (P2) — Prosody continuity seam metric across segment boundaries (pitch/energy contour deltas — deterministic, librosa-class; wire-verify).

6. 3D models & scenes#

The modality where "own your ledger" is WRONG at the scene level — the industry already has one.

Lesson mapping. Hierarchy: scene → assets → parts → materials. Anchors: blockout/proxy meshes before high-poly sculpt + texture (the exact keyframe economics — auditing silhouettes and scale is nearly free); for single assets, the multiview turntable render is the anchor (cheap 2D audit of an expensive 3D artifact). Seams: spatial — scale consistency between adjacent assets, texel density, style coherence, tiling texture borders, terrain chunk boundaries. Surgical regen: part-level (regenerate one part, one material, one texture — never the asset; never the scene for one asset). Detectors first — 3D is detector-RICH and judge-poor: manifoldness, UV overlap, polycount budgets, PBR validation, scale sanity. Reference bank: material/style libraries; for characters, the SAME character entries as images (one bank across modalities is the point).

Already own (unusually deep): hunyuan3d-asset (shape→GLB, gated), part-level-3d, text-mesh-editing, 3d-semantic-editing, 3d-quality-gates, 3d-scene-assembly, universal-rigging, mesh-transformers, gaussian-splatting suite, 3d-post-pipeline, real USD/LIVRPS + variant work (Phase 5.2, real pxr runtime on-box), UE5.5 on-box with the C++ .uasset authoring server, scene-experience.

Build recommendations:

  • D1 (P1) — USD as the scene ledger: do NOT extend SceneLedger to spatial scenes. Wrap USD composition (layers/references/variants = recipe+lineage; variants literally ARE retakes) with a thin adapter exposing the same read-side surface our agents use elsewhere (digest, measurements, gate status per prim). Our LIVRPS work is the foundation.
  • D2 (P1) — Blockout-first asset pipeline: proxy/blockout stage gated (turntable renders judged with existing image gates + human gate per the matrix) before sculpt/texture stages; per-part candidate budgeting via the shipped allocator; 3d-quality-gates wired in as the deterministic floor of the critic→regen loop, with part-level regen as the surgical arm (libraries exist; the LOOP binding is the gap).
  • D3 (P2) — Asset-seam metrics for scenes: scale/texel-density/style deltas between adjacent placed assets recorded as edge measurements on the USD-wrapped graph.
  • D4 (P2) — Hunyuan3D texture stage stays a deliberate endpoint decision (compiled C++ extensions, ~21GB VRAM — unchanged verdict from the video report).

7. Video (reference implementation — done)#

Covered by AGENTIC_LONGFORM_VIDEO_LESSONS_2026-07-04.md and implemented: scene ledger, critic→regen loop, keyframe-first planner, reference bank, budgeting, gates, media index/digest, ffmpeg assembly + CIE76 seams, wan-segment-chain, VACE edit/join, LTX-2 joint-AV + keyframe interpolation, distill fast_mode. Remaining deploy-boundary items live in that report's addendum (endpoint smokes; BFF stores/routes for ledger and bank).


8. Storylines / narrative (scripts, prose, episodic fiction)#

Lesson mapping. Hierarchy: logline → treatment → beat sheet → chapters/scenes → prose — the artifact ladder IS the anchor hierarchy, and the research is unambiguous about where the gate goes: the beat sheet/outline is the human gate (product convergence) and structured multi-role critique at the plan stage measurably pays (FilmAgent ablation 2.63 → 3.98/5). Soap2Soap's persistent scene-aware JSON screenplay and CoAgent's entity memory + verifier-triggered selective regen are the direct precedents for a story ledger. MAViS adds a subtle, important lesson: constrain the script stage to what downstream generators can actually execute — scriptwriting guidelines as a compile-target contract, not free prose. Seams: continuity — carried state (who knows what, who has what, where everyone is) across scene boundaries; contradictions are the color drift of narrative. Surgical regen: rewrite ONE scene under explicit constraints imported from its graph neighbors (entering state + required exit state), never the manuscript. Detectors first: deterministic continuity checkers over a structured state ledger (entity positions, knowledge, props, timeline) catch what zero-shot judges miss; LLM judging with tournaments for prose quality dimensions. Neighbor-context judging is even more essential than in video — a scene is only good GIVEN its neighbors.

Already own: V8 Ariadne is the existing proof — solve-first deterministic core, case ledger, seven fairness/solvability gates, DPLL verification before rendering: precisely "plan top-down, verify deterministically, then spend on prose." Also: metis narrator, crew critique→revise, av-narrative shot/beat vocabulary, prompt-pipeline, agentic-studio plan DAG.

Build recommendations:

  • N1 (P1) — Story ledger (the graph variant): nodes = scenes/beats with intent (beat, POV, required entering/exiting state), recipe (model + prompt template + seed for reproducible drafts), measurements (continuity check results, judge tournaments), lineage (retakes) — scene-ledger architecture with blockedBy/entity-edge structure instead of pure sequence. Generalize the shipped sequence ledger only as far as this consumer needs (edges + constraint seams), not into a grand abstraction.
  • N2 (P1) — Deterministic continuity detector: a typed world-state delta per scene (characters/props/locations/knowledge, declared as part of scene intent, MAViS-style) checked mechanically across edges — the WER/temporal-QA analog. V8's constraint machinery is the in-repo pattern to follow.
  • N3 (P2) — Scene-surgical rewrite loop: critic→regen pointed at scene nodes; the delta planner imports neighbor constraints into the rewrite prompt (an LLM-planner seam — the deterministic default only bumps seeds/temperature honestly).
  • N4 (P2) — Outline gate wiring: beat sheet review rides the shipped gate-policy matrix in the autonomy plane (the storyboard stage IS the outline stage for prose).

9. Worldbuilding / lore / living game worlds#

The fully-graph case, and the one where "index-not-pixels" is the primary lesson rather than a supporting one.

Lesson mapping. Hierarchy: world → domains (geography, cultures, economies, pantheons, history) → entities → relations. There is no single linear order — the ledger is an entity/relation graph with a timeline axis, exactly the shape VideoRAG's knowledge-graph index demonstrated on the understanding side. Anchors: the world bible seed (map, timeline skeleton, pantheon, naming rules) — cheap documents gated before mass content generation. Seams: canon consistency — every new artifact (quest, NPC, item, chapter) is an edge into the graph, and the seam check is contradiction against canon (names, geography, chronology, established facts). Surgical regen: regenerate one entity/quest/region description against its fixed neighbors. Reference bank: this modality IS the reference bank generalized — the canon graph is the bank, and every generator in every OTHER modality should consume it (character entries for images/video, voice entries for speech, palace geometry for 3D). Detectors: name-collision, timeline-ordering, geography-reachability checks are all deterministic. Index-not-pixels: agents navigate the world via the graph digest, never by re-reading generated corpora.

Already own: hathor's real culture/economy/character/MDA modeling, maya engine + world federation/interoperability work, world registration

  • health monitoring, V8's self-authoring universe (a bounded worldbuilding system with hard gates), V9's provenance-stamped knowledge forging, nyx ephemeris (real computed facts as canon inputs), sophia grounding/retrieval.

Build recommendations:

  • W1 (P1) — Canon graph ledger: entities/relations/timeline with per-node provenance (who/what generated it, from which seeds) and canon-status (proposed → gated → canon — the approval vocabulary reused). Sophia's retrieval + embeddings serve as the index layer; hathor models supply typed entity payloads for cultures/economies.
  • W2 (P1) — Contradiction detector as the hard floor: deterministic checks (unique names, timeline partial-order, geographic adjacency, fact table lookups) before any judge; new content that fails is a regen candidate for the critic loop, not a silent merge.
  • W3 (P2) — Cross-modal bank projection: one canon entity projects into every modality's reference surface (image refs, voice ref, 3D asset ref, style notes) — the character bank rows in reference-bank.ts become VIEWS over canon entities rather than standalone records once W1 exists.
  • W4 (P2) — World digest for agents: budget-bounded graph digest (regions → entities → one-line canon facts + edge counts), the ledgerDigest pattern over a graph traversal.

10. Cross-modal composition (the film case, and the autonomous director)#

A finished film is the composition of ALL of the above: story ledger → scene ledger → per-shot video + speech + music + SFX stems → assembly. The lessons compose vertically:

  • The story ledger (N1) is upstream of the scene ledger — beats compile into shot intents (the storyboard compiler already takes this role for the video layer).
  • Stems bind modalities at the shot node: the shot's audioStemUris already carry speech/music/SFX outputs into assembly; the song and cue-sheet ledgers (M1, F1) produce those stems with their own gates before the AV assembly gate.
  • One reference bank feeds everything (W3): the same character entry locks the face in keyframes, the voice in dialogue, and the model sheet in 3D.
  • The autonomous director (@oshun/creative-autonomy) gains shot/section-level budget allocation (L8) as a policy change — its bandit + governor already allocate at the idea/variant level; the allocator generalizes that to intra-work granularity.

Build recommendation:

  • X1 (P2, after M1/N1) — Production ledger: a thin composition layer referencing child ledgers by id (story, scenes, song, cues) with cross-ledger gates (no assembly until every child's gates clear). Not a new abstraction — an id-reference convention plus one gate evaluator.

11. Anti-lessons (unchanged, restated cross-modally)#

  • No premature grand unification. Do not build the universal MediaLedger now. Keep the pure pieces shared (they already are), and generalize the ledger core only when a second modality adopts it. The field's evidence: every attempted cross-tool standard stalled; every working system owns a concrete schema.
  • Adopt USD for spatial scenes; don't reinvent it. (D1.)
  • Don't trust lone zero-shot judges anywhere — the measured video finding (0.56 vs 0.73 Spearman) is, if anything, worse for audio and 3D, where LMM judging is younger. Detectors + tournaments everywhere.
  • Escalate regen downward (region/part/utterance → node → group → work) in every modality; the compute ordering the ablations support.
  • Wire-verify before authoring anything that touches an external contract (ACE-Step repaint/extend, key/tempo estimators, local ASR, prosody metrics are the flagged items above).
  • Fail loud, never fabricate: unmeasured nodes are "unmeasured," not passing; missing detectors block, they don't default to green — the posture every shipped piece already enforces.

12. Consolidated build sequence (across modalities)#

Priority Item Modality Depends on
1 M1 song ledger + M4 audio-only assembly Music shipped spine; key/tempo detector (wire-verify)
2 N1 story ledger (graph variant) + N2 continuity detector Narrative shipped spine; V8 patterns
3 S1 utterance ledger + WER floor + S2 voice bank rows Speech ASR seam (wire-verify)
4 I1 series ledger profile + I2 anchor-first sets Images shipped spine only
5 D1 USD wrapper + D2 blockout-first loop w/ 3d-quality-gates 3D existing USD/gates libs
6 W1 canon graph + W2 contradiction floor Worlds sophia/hathor
7 M2/M3 hook-first + section regen; N3/N4; F1 cue sheets Music/Narrative/SFX 1–2 above
8 W3 cross-modal bank projection; X1 production ledger Cross-modal W1 + M1/N1
9 I3, D3, S3, F2, W4 seam/digest deepenings various their parents

The through-line, once more: same skeleton, same reflexes, different anatomy. In music, narrative, speech, and image series we are one bridging module away — the organs exist and the spine is now shipped; in 3D the spine already exists in the industry (USD) and we wrap it; in worldbuilding the graph variant is the one genuinely new structure, and its closest working precedent is already in this repo (V8).


Implementation addendum (same day): all 24 recommendations landed#

Every item in §12 is implemented and tested (D4 remains the explicitly deliberate deferral it was documented as). Where things live:

Music (M1–M5). ledger/song-ledger.ts: sections with bpm/key/ time-signature locks, per-stem artifacts, retake archival; seam detectors are REAL music theory — circle-of-fifths distance over key-signature indexes (relative major/minor share a signature; C→F# = 6 steps, tested against theory), tempo-ratio and time-signature checks liftable only by DECLARED transitions, measured-LUFS deltas, and true-peak clipping as a hard failure. Hook-first planning (budgeted hook candidates on deterministic seed streams; arrangement stage hard-blocked until the hook gate clears) + section scoring/regen candidates + song stage gates. audio/song-section joins the catalog (64 workflows / 78 endpoint refs / 0 violations): ACE-Step 1.5 section renderer whose hook conditioning was wire-verified at the v0.27.0 tag — core ReferenceTimbreAudio + VAEEncodeAudio lock the section's timbre on the approved hook. The motif bank kind (M5) landed in the reference bank with per-kind invariants. @shared/encoding/audio-analysis.ts supplies the measured side with REAL DSP: radix-2 FFT, chromagram → Krumhansl–Schmuckler key estimation (C-E-G fixture → C major r=0.82), spectral-flux/autocorrelation tempo (120-BPM clicks → 120.06), and real loudnorm-JSON loudness parsing (a −20 dB copy measures exactly 20.00 LU lower).

SFX (F1–F2). ledger/cue-sheet.ts: timestamped cue validation (past-end, overlaps, loop quality) compiling to offset stems; assembly gained offsetSeconds per stem (adelay) with REAL placement verification (an 880 Hz blip at offset 1.0 s is detected by Goertzel in its window and absent before it), and measureLoopSeam gives beds a real tail↔head correlation (perfect loop → 1.000; noise → 0.03).

Speech (S1–S3). ledger/utterance-ledger.ts: utterances with a REAL WER implementation (DP backtrace with exact S/I/D decomposition, tested on classical cases) behind an injectable, fail-closed transcriber seam — the ASR floor blocks hard at 8% default; voice bank entries (S2) enforce voiceRefUri; prosody contours (S3) come from real per-frame RMS + NCC pitch tracking (120 vs 240 Hz fixtures measure exactly 12.000 semitones).

Images (I1–I3). ledger/image-series.ts: set ledger with palette outlier detection in REAL Lab (local sRGB→Lab + CIE76; red↔blue = 176.3, matching the assembly module's on-pixel measurement), embedding-cosine style cohesion, identity-score floors, worst-first set regen candidates, and the thumbnail→finals anchor-first pipeline on the shared allocator and gate matrix.

3D (D1–D3). ledger/usd-scene-adapter.ts wraps USD instead of reinventing it: prim records flow from a real pxr helper (scripts/isis/usd_list_prims.py; the gated integration test runs the REAL USD 0.26.5 runtime on-box and round-trips a variant-bearing stage), measurements/approvals attach by prim path, and retake recommendations speak USD's own vocabulary (variant flip → reference re-author → prim re-author, most surgical first). ledger/asset-ledger.ts: blockout-first stages with turntable gating, the structural quality-report contract as a detector floor that blocks the detail gate at EVERY autonomy level, and spatial seam metrics (AABB-adjacent scale/texel-density/style checks).

Narrative (N1–N4) + Worlds (W1–W4). story-ledger (graph variant: acts/scenes/entities with kind-checked references), story-continuity (typed world-state walk: nine violation kinds incl. dead-character-*, prop custody, knowledge-before-learned — each tested both ways), story-gates (continuity blocks at every autonomy level), canon-graph (proposed→gated→canon ladder with explicit approval records, provenance, crossModalRefs), canon-contradictions (name/alias collisions, timeline DAG cycles, order coherence, adjacency symmetry, located-in cycles, dead-participant, fact conflicts), canon-digest (W4), and canon-projection.ts (W3): gated/canon entities project into character AND voice bank entries — proposed canon never locks identity.

Cross-modal (X1). ledger/production-ledger.ts: children referenced by id + kind, gate state flowing through each child's own evaluator (an unevaluable child is blocked, never passing), required-vs-optional children, and recordProductionAssembly refusing to record over a blocked gate.

Gates at completion: workflows lib 312 tests (27 files) + tsc clean

  • 64/64 validate; encoding lib 99 tests (real-ffmpeg fixtures throughout)
  • tsc clean; Python render contract 21 tests / 2,684 subtests; supply chain 16 endpoints / 78 refs / 0 violations; adversarial stub + Math.random scans clean across all new files.

Honest boundaries: the transcriber (ASR), image-mean-color/embedding extraction, 3D validators, and the USD runtime are injectable seams bound at deployments (the pxr seam is additionally exercised for real on-box); song-section's hook conditioning is graph-verified, not yet pixel/audio-smoked on a live endpoint (same first-deploy smoke posture as the whole catalog); BFF persistence for the new ledgers remains the next slice, as before.