Eve library

Phase 182 Repository Capability Audit and Gap Ledger

Guide

Revision 1

On this page

Audited 2026-09-09. Scope: 182.C.02.01–182.C.02.12.

This is the honest inventory of what the repository can do today for controlled human video, what it only appears to do, and what must be built. Every verdict below was reached by reading the source, not by file existence, grep counts, or a module's own doc comment. Where a comment and the code disagree, the code wins and the disagreement is recorded as a finding.

Severity: H = ships a fabricated or false capability signal; M = real but unreachable, duplicated, or stale; L = cosmetic or documentation.

Summary of Findings#

# Sev Finding Item
F-01 H @isis/seedance-provider silently substituted a fabricated phoneme track when no lip-sync service was injected 02.03
F-02 M Five provider abstractions already exist for video; a sixth (libs/yemaya/video-generation) was not in the plan 02.03/.10
F-03 M 117 .js and 118 .d.ts build artifacts are committed beside source under libs/isis 02.03
F-04 H apps/psyche/avatar-engine has four unrunnable Nx targets (three missing modules, one silent no-op) and zero tests 02.06
F-05 M Every Veritas video provider except HeyGen is unreachable; four service modules that construct them have no importer 02.07
F-06 L video-provider-env.ts claimed there is no video watermarker or C2PA signer in-repo; both exist and are wired 02.02
F-07 M The OpenRouter adapter's download path uses unsigned_urls, not the documented /content endpoint, and is unprobed 02.02
F-08 M generate_audio defaults to true on audio-capable models and the adapter never sets it 02.02
F-09 M @nous/portrait-animation is 22 modules and 12,248 lines with zero inference I/O — every module is plan, score or metadata 02.05
F-10 M The nine release gates carry no speech, sync, speaker, identity, or consent gate 02.11
F-11 L scoreShapeMatch's constants are an uncalibrated ordinal ranking presented without that caveat 02.03
F-12 M Four incompatible interfaces named VideoGenerationRequest, with three different duration units and no converter 02.03/.10
F-13 M video-workflow-manager.ts holds the queue/batch/scene/template primitives worth reusing — but its queue is in-process 02.03/.11
F-14 L The capability matrix's generation-video row cites a RunPod probe; the deployable lane is fal/OpenRouter 02.09
F-15 M apps/oshun/web type-checks @isis/ai-video against a shim whose DurationPreset admits number — a fifth vocabulary 04.03
F-16 H @isis/ai-video's ensemble-hdr.ts fabricates VMAF, tone mapping, HDR validation and EXR export when no seam is injected 04.03

182.C.02.01 Curated video card types and validation path#

Files read: libs/isis/curated-cards/src/card-types.ts (228 lines), card-validators.ts (354 lines).

Verdict: real, and correctly narrow. Two card kinds exist, video (text-to-video) and video-from-image. Every curated field is checked against its own enum in validateSharedVideoParams, and CARD_INPUT_FIELD_ALLOWLIST checks the set of submitted fields, not only their values — so a client cannot smuggle providerKey, steps, or a raw scheduler through the form. The image-to-video validator additionally constrains referenceImage to an https: URL or a data:image/ URI. This is a good foundation and must not be widened carelessly.

What is missing for human video, field by field:

Needed by Phase 182 Present today
Script / exact dialogue text, per-speaker turns none
Approved speech audio asset + digest none
Multiple typed references (identity, face-angle, full-body, wardrobe, …) one referenceImage: string
Cast and speaker IDs none
Pronunciation lexicon none
Performance direction (blocking, gaze, gesture, emotion, beat timing) one cameraMovement enum
Shot plan / multi-shot none
requiredCapabilities vs preferences none
Quality policy (thresholds, candidate count, retry budget) one boolean hero
Output preferences (renditions, captions, stems) none
Consent / authorization binding none

Bounds that will constrain routing: VIDEO_DURATIONS_SECONDS is [4, 5, 10] only; VIDEO_RESOLUTIONS is ['720p','1080p']; VideoAspect is four ratios. The live OpenRouter catalog offers 2–30 s, 480p–4K, and nine aspect ratios, so the curated card is currently the tighter constraint, and any duration a human video needs beyond 10 s has no card representation at all.

182.C.02.02 BFF video executor, provider environment, router, storage, governance, release gates#

Files read: video-executor.ts (186), openrouter-video-provider.ts (364), video-provider-env.ts (358), provider-quality-router.ts (184), video-governance.ts (117), video-watermark.ts (219), jobs-route.ts (768, status enum and job envelope).

Real code paths. The executor validates before any provider call, fails closed with provider_not_configured when no provider is injected, and refuses a result with an empty URL or missing key frame. video-governance.ts composes a real spread-spectrum video watermarker and the shared C2PA signer, each independently fail-closed to null. provider-quality-router.ts is a genuine Beta–Thompson sampler over measured gate-pass rates with a JSON-safe snapshot. The OpenRouter adapter implements create → poll → download with real backoff, a real deadline, and loud failures — no fabricated URL, no fabricated completion.

Seams and fixtures, not capability. VideoProviderGenerate is injected at the deployable boundary; with no key the whole video path is inert. Nothing in this stack has a committed live-probe fixture, so under the tracker's status vocabulary the OpenRouter route is implemented, never live-probed.

F-06 (L). The video-provider-env.ts header stated "There is still NO video watermarker and NO C2PA video signer in-repo, so provenanceManifestPresent stays honestly false and watermarkCoverage stays honestly null." Both now exist (video-watermark.ts, audio-c2pa-signer.ts) and are wired through video-governance.ts:115. The comment was stale and read as a capability claim in the wrong direction. Fixed in this pass.

F-07 (M). The adapter downloads from unsigned_urls entries on the poll response. The current documented contract of record is GET /api/v1/videos/{id}/content?index=0. Both may work; neither is proved here. The adapter must be reconciled against a live probe at 182.C.11, and the frame_images entry shape it sends — { type, image_url: { url }, frame_type } — must be probed too, since the documented input_references entry is { type: 'image_url', image_url } with image_url not obviously an object.

F-08 (M). generate_audio defaults to true on audio-capable models and the adapter never sets it. The default model is alibaba/wan-2.7, whose catalog record has generate_audio: true, so every clip generated today comes back with model-invented audio that nothing asked for. On Wan 2.7's flat duration_seconds price this costs nothing extra, but on Veo 3.1 the same omission is the difference between $0.20/s and $0.40/s, and for exact-dialogue mode an unrequested generated audio track is a correctness problem, not only a cost one.

Base64 delivery. download() returns data:video/mp4;base64,…. A 10-second 1080p clip is tens of megabytes, inflated ~33% by base64, and it travels through the job record. This is the single biggest blocker to real evaluation and is already scoped as 182.C.11.08 / 182.C.16.02.

Job envelope. GenerationJobStatus is exactly six states — queued, running, complete, failed, blocked, needs_review — with blocked and needs_review already carrying the release-gate decision. Jobs live in an in-memory Map with a PostgreSQL snapshot and a documented disposition record. This is the right thing to extend at 182.C.05.10, not to replace.

182.C.02.03 @isis/ai-video, @isis/ai-providers, @isis/seedance-provider#

F-01 (H) — the fabricated lip-sync track#

libs/isis/seedance-provider/src/lip-sync-validation.ts is 811 lines. Its visual half is real: extractFramesWithFfmpeg spawns ffmpeg, classifyMouthShape uses sharp on decoded frames, and the mouth-cue analyzer measures actual pixels.

Its audio half fabricated. validateSeedanceLipSync resolved options.lipSyncService ?? createDefaultLipSyncService(), and that default called estimateLipSyncFromAudio, which:

  • derived duration as byteLength / 16000 with no container parse, so the number is wrong for any real WAV, MP3, or AAC payload;
  • generated a phoneme track by cycling the fixed pattern ['m','aa','s','ow','t','eh','n','uw'] — the phonemes bear no relation to the audio, which is invented ground truth, not a measurement;
  • attached confidence: 0.7 + (index % 3) * 0.08, a value derived from the array index (line 324, the one the plan already flagged);
  • returned syncQualityScore: clamp(0.78 + durationSecs / 60 - 1 / frameRate, …), a quality score computed from duration and frame rate alone.

The consequence was worse than any single line. The real mouth-cue analyzer then scored real video frames against that invented phoneme track, and phonemeAccuracy, utteranceConfidenceScore, and overallConfidenceScore came back looking like measurements. Both in-repo tests inject a real lipSyncService, and SeedanceVideoProvider passes this.lipSyncService, which is optional — so the fabricated path was precisely the production path.

Fixed in this pass: the silent default is removed and the seam now throws SeedanceLipSyncServiceNotConfiguredError. See "Fixes applied" below.

F-02 (M) — provider abstractions#

Six, not five. 182.C.02.10's consolidation list must include the one the plan did not name:

Abstraction Size Status
libs/isis/ai-video 22 TS files Runway client, next-gen adapters, SVD self-hosted server, workflow mgr
libs/isis/ai-providers/.../video-generation 5 TS + 10 build artifacts LTX (used by the BFF), Hunyuan, Wan providers
libs/isis/seedance-provider 92 files (46 src + specs) fal, PiAPI, Volcengine Ark clients + editing/reference modules
apps/veritas/video/src/providers 13 files HeyGen, Tavus, D-ID, Runway, OpusClip, Shotstack, stock, unified, ASR
apps/oshun/bff/.../video-provider-env.ts 358 lines The deployable seam that actually resolves a client
libs/yemaya/video-generation 18 TS files Its own model registry and provider router — not in the plan

libs/yemaya/video-generation/src/model-registry/model-registry.ts maps wan-2.6-t2v → wan-ai/wan2.6-t2v and similar AIML endpoints. Those are fal/AIML-style ids and are unrelated to OpenRouter's alibaba/wan-2.6; the same stale-id pattern the plan flagged in the Wan provider recurs here. Both id vocabularies must be reconciled against the live catalog rather than trusted.

F-03 (M) — committed build artifacts#

find libs/isis -name '*.js' -not -path '*/dist/*' → 117 files; -name '*.d.ts' → 118. libs/isis/ai-providers/src/providers/video-generation/ carries hunyuan-provider.js, hunyuan-provider.d.ts, index.js, index.d.ts, ltx-provider.js, ltx-provider.d.ts, types.js, types.d.ts, wan-provider.js, wan-provider.d.ts beside their sources. A stale .js beside a .ts can win module resolution and silently serve last month's code. These are build output and belong in dist.

F-12 (M) — four incompatible VideoGenerationRequest contracts#

The name is shared; the shape is not. Nothing converts between them, so every new caller re-derives the mapping by hand and the duration unit changes meaning across the boundary:

Contract Duration field Model field Other required fields
@isis/ai-video video.types.ts:VideoGenerationRequest duration: DurationPreset (4|5|10|16) none aspectRatio; fields mutable
@isis/ai-providers video-generation/types.ts:276 duration?: number (seconds) model required readonly throughout; adds width, height, fps, steps, guidance
@yemaya/video-generation types.ts:324 durationSeconds: number modelId | null id, mode, fps, steps, guidanceScale, seed, six presets
BFF video-executor.ts:25:ProviderVideoRequest durationSeconds: number none (bound at env) aspect, resolution, cameraMovement, stylePreset, locale

@isis/ai-video's DurationPreset admits 16, which no curated card can express (VIDEO_DURATIONS_SECONDS is [4, 5, 10]) and which OpenRouter's default lane, Wan 2.7, cannot render (2–10 s). The BFF contract is the one the deployable path actually uses and is the narrowest; the Phase 182 contract at 182.C.05 must be defined once and the other three adapted to it or deleted per 182.C.02.10 — not extended in place.

F-13 (M) — reusable workflow primitives worth keeping#

@isis/ai-video's video-workflow-manager.ts (866 lines) is the one piece of the duplicated stack with primitives Phase 182 should reuse rather than rewrite: a priority queue with concurrency control and QueueEvents, enqueueBatch, generateFromScenes (a scene list → per-shot job fan-out, which is the shot plan of 182.C.09 in embryo), executeWorkflow over declarative WorkflowTemplate/WorkflowStepTemplate records with injectable WorkflowStepHandlers, and cancelJob/waitForJob. Its execution is injected through the VideoGenerationExecutor port (submit/getJob/cancelJob), so it carries no provider coupling and can sit above the BFF job envelope.

Two caveats before reuse: its queue is in-process and non-durable, so it is not a replacement for the durable job envelope in jobs-route.ts (the extend-or-replace decision in 182.C.02.11 stands), and its WORKFLOW_TEMPLATES constant encodes pre-Phase-182 stages with no evaluation, retry, or release-gate step. Reuse the queue, the batch/scene fan-out, and the template execution model; re-author the templates against the 182.C stage order.

F-15 (M) — the web app type-checks against a shim that contradicts the library#

Found while inventorying consumers for 182.C.04.03. @isis/ai-video has exactly one consumer, libs/yemaya/remote-film-capture, and it imports types only (isis-ai-video-integration.ts:1-10). But the two build paths disagree about where those types come from:

Build path @isis/ai-video resolves to DurationPreset there
apps/oshun/web/tsconfig.json:159 remote-film-capture/src/isis-ai-video-shim.d.ts 5 | 10 | 15 | 30 | number
apps/oshun/web/vitest.config.ts:589 the real libs/isis/ai-video/src/index.ts 4 | 5 | 10 | 16

The shim's | number makes the union unconstrained, and its VideoProvider, AspectRatio, VideoResolution, and VideoStyle are all widened with | string the same way. So the web app's tsc accepts durations and providers that the real library rejects, while its vitest run compiles against the real types — the two can diverge silently, and the typecheck is the weaker of the two.

This is F-12's problem with an extra twist: it is a fifth duration vocabulary, and it is the one that shadows the real library in the build that gates CI. It must be reconciled before @isis/ai-video is retired, because retiring the real library while the shim stands would leave the web app type-checking against a declaration file for a package that no longer exists.

F-16 (H) — the retirement scope was assessed on the provider clients alone#

Found while executing 182.C.04.03 step 4, reading libs/isis/ai-video before deleting it. F-02's one-line inventory ("Runway client, next-gen adapters, SVD self-hosted server, workflow mgr") did not cover src/advanced/, which is 3,691 lines across three modules. They do not share a disposition:

Module Lines Disposition on reading
lens-physics.ts 727 Kept. Real optics, no provider coupling, no fabricated value — relocated to @isis/video-enhancement.
ensemble-hdr.ts 728 Deleted. F-01's pattern again: fabricated defaults behind optional seams (below).
video-generation-sota.ts 2,236 Deleted. Option builders over the retiring VideoProviderClient/RunwayClient; provider-shaped, no unique math.

lens-physics.ts computes thin-lens image distance, hyperfocal distance, circle of confusion in millimetres and in sensor pixels, cat-eye and oval bokeh from blade geometry, shutter-angle motion blur, focus breathing, Brown–Conrady distortion with per-channel chromatic offsets, and anamorphic squeeze. Every output is derived from the inputs. Its nearest neighbour, @aphrodite/auteur-camera/lens-simulation.ts (785 lines), manages renderer parameters for a live rig rather than computing optics, so it is not a duplicate and deleting the library would have lost the work.

ensemble-hdr.ts reads as the opposite once its defaults are followed:

  • estimatePerceptualSignals returns vmaf: quality * 100, where quality is a number the provider put in its own result metadata. scorePerceptualQuality then publishes that as vmaf — a named algorithm that requires a reference and a distorted video and was never run. The reader seam is optional, so this is the default path.
  • defaultToneMap "tone maps" by rewriting the file extension to .hdr.mp4 and echoing the requested peakNits back as fact.
  • defaultHdrValidation reports measuredPeakNits equal to the value defaultToneMap just echoed from the config, and passes or fails against a supportsHdr flag from a hand-written table.
  • defaultExrExport returns an .exr path for a file it never wrote.
  • DEFAULT_PROVIDER_PROFILES hard-codes price and estimatedQuality for twelve providers with no date and no source — the mutable-fact-frozen-in-code problem ADR-0006 forbids, and superseded by provider-quality-router.ts, which selects on measured release rates.

Both in-repo tests inject the real seams, so the tests pass while the default path fabricates — the same shape as F-01, and the reason this module was deleted with the provider stack rather than relocated beside lens-physics.ts.

182.C.02.04 Identity, scene, shot, wardrobe, and temporal consistency modules#

libs/isis/workflows/src/consistency/ holds ten modules, each with a spec: identity-profiles, identity-drift, scene-profiles, shot-profiles, wardrobe-profiles, temporal-drift, temporal-quality-qa, golden-output-perceptual-regression, orchestration, sequence-orchestration.

Mapping to the Phase 182 contract:

Existing module Maps to
identity-profiles 182.C.07.01 character pack (extend, don't fork)
identity-drift, temporal-drift 182.C.19.01 temporal identity drift
scene-profiles, shot-profiles 182.C.09.01 shot-plan schema
wardrobe-profiles 182.C.07.04 mutable shot traits
golden-output-perceptual-regression 182.C.34 bakeoff regression
sequence-orchestration 182.C.22 long-form continuity ledger
temporal-quality-qa 182.C.19.06 motion/temporal consistency

These are the designated integration targets for 182.C.07.06.

Where the boundary actually falls. The first pass called these "bookkeeping over supplied scores," which understates identity-drift.ts and overstates the gap. The real split is between measurement over supplied features and feature extraction from pixels:

  • Real math, already here. identity-drift.ts computes cosine similarity over embedding vectors (:112-131) and mean euclidean landmark displacement (:134-152), weights them (:98-99), and resolves a severity and a review decision against configured thresholds. temporal-quality-qa.ts computes pairwise frame metrics and a weighted assessment the same way.
  • Not here, and the reason 182.C.19 cannot be satisfied by these modules. Nothing in the directory extracts a face embedding or a landmark set from a decoded frame. Every vector and every landmark point arrives as an input, so the accuracy of the whole assessment is inherited from an extractor that does not yet exist in the repo — which is precisely the GPU-hosted work 182.C.19 scopes and 182.C.01.10 marks blocked-local.

So 182.C.19 must supply the extractor and may reuse these assessors rather than reimplementing the comparison math.

182.C.02.05 @nous/portrait-animation#

22 exported modules, 12,248 lines of non-spec source. F-09 (M): not one performs inference. A scan for process spawning, HTTP, or model runtime across every non-spec module returns zero hits except three incidental matches in portrait-animation-streaming-output-pipeline.ts.

Labels per 182.C.02.05:

Label Modules
planning-only hallo3-portrait-animation-inference-pipeline, hallo3-video-dit-model-loader, im-portrait-3d-aware-video-diffusion, liveportrait-expression-transfer-pipeline, liveportrait-stitching-animation-pipeline, multi-style-…, multi-person-…, musetalk-video-dubbing-pipeline, echomimic-audio-to-facial-motion-pipeline, chatanyone-realtime-portrait-pipeline, rap-realtime-audio-driven-portrait-pipeline, realtime-audio-stream-portrait-pipeline, partial-face-editing-pipeline, portrait-animation-expression-amplitude-control, portrait-animation-gpu-batching-orchestrator, portrait-reference-preprocessing-pipeline, cross-language-dubbing-pipeline, dubbing-batch-processor
score-ingestion-only portrait-animation-quality-metrics-pipeline, dubbing-quality-validation-pipeline
executable none
adapter-backed none
stale none found; the modules are internally consistent
metadata-only portrait-animation-library (17 lines: library name, Nx project name, tags, and a metadata getter — no animation behaviour at all)

The first pass's table labelled 20 of the 22 modules. The two it omitted, read and labelled here:

  • portrait-animation-library.ts — metadata-only. Seventeen lines of Nx identity constants and getPortraitAnimationLibraryMetadata(). It is not a pipeline and should not be counted as one in any capability claim.
  • portrait-animation-streaming-output-pipeline.ts — planning-only, same as the other eighteen. Its 653 lines resolve a WHIP/WHEP/LL-HLS transport plan through resolvePlan (:399) and define a …Stats intake type; the three "HTTP" matches the scan flagged are spec URL constants (:5-11) and the literal 'single-http-post-with-ice-patch-updates' describing WHIP signalling, not a request the module makes. Nothing here opens a socket or a peer connection.

With those two placed, the count is 22: nineteen planning-only, two score-ingestion-only, one metadata-only, and still zero executable or adapter-backed modules.

The naming is honest — the public method is resolvePlan and the return type is …Plan — so this is not a fabrication. But it is also not evidence of lip sync. PortraitAnimationQualityMetricsPipeline.resolvePlan takes request.observedScores and compares them to thresholds (lseC 6, lseD 8, ArcFace cosine 0.35, FVD 120, head-pose MAE 8°); it cites the SyncNet, Wav2Lip, ArcFace, FVD, and 6DRepNet references but never runs any of them. Under 182.C.01.05 a score this library was handed can never populate a metric field. This is exactly what 182.C.31.03–182.C.31.05 must resolve.

182.C.02.06 apps/psyche/avatar-engine#

F-04 (H). 157 files of real-looking Python: 3D Gaussian Splatting, FLAME face modelling, NeRF, a lip-sync subpackage with a genuine lazily-imported Whisper phoneme extractor plus NLTK CMUdict and librosa. Dependencies are torch, pytorch3d, open3d, insightface, mediapipe, face-alignment, smplx, chumpy.

Four of its declared Nx targets cannot run. Three point at modules that do not exist; the fourth points at real code with no entry point:

Target Command Reality
serve / serve-prod uvicorn avatar_engine.main:app no main.py in src/avatar_engine/
download-models python -m avatar_engine.scripts.download_models no scripts/ package
train python -m avatar_engine.training.trainer trainer.py exists (692 lines) but defines only classes — no __main__
test / -no-gpu/-cov pytest zero test files; tests/ holds only a .gitkeep

The train row corrects the first pass, which reported trainer.py as missing. It is there and it is substantial — AvatarDataset(Dataset), a BlendshapeModel(nn.Module), AvatarTrainer, and a DistributedTrainer subclass. What it lacks is a main() and an if __name__ == '__main__' guard, so python -m avatar_engine.training.trainer imports the module, runs nothing, and exits 0 — a silent no-op, which is worse than the import error the other two targets produce, because it looks like a successful training run.

So the service cannot be started, cannot acquire weights, cannot be trained through its own target, and has never been tested. It is reachable in principle from libs/psyche/avatar-training/src/backend.ts, which speaks HTTP to it (:399 health, :611 request) — but only against the main:app that does not exist.

Licensing is unresolved and material. FLAME (models/flame.py loads a .pkl by path) and smplx are distributed under research licenses that restrict commercial use; insightface models carry their own non-commercial terms. libs/shared/audit-platform/src/model-license-review.ts is the right place to record this — it already models weight-redistribution rights, commercial-output rights, and quarantine, and names avatar-engine weights explicitly — but no review record exists.

Recommendation for 182.C.31.06: retire it from capability claims for this phase. Hardening it means writing an entry point, a weight-acquisition script, a licence review, and an inference test suite for a GPU service on hardware this project does not have. Nothing in Phase 182's critical path needs it.

182.C.02.07 Veritas avatar/video providers#

F-05 (M). apps/veritas/video/src/providers/ contains heygen.ts, tavus.ts, did.ts, runway.ts, opusclip.ts, shotstack.ts, stock-footage.ts, unified.ts, plus the three ASR clients.

Only HeyGen is constructed at runtime: main.ts:138 and main.ts:231 call createHeyGenClient(...) behind hasHeyGenKey(). createUnifiedVideoProvider, getUnifiedVideoProvider, and createUnifiedProviderFromEnv have no callers anywhere in the repository outside unified.ts and its own barrel re-export. server.ts only uses the string union 'heygen' | 'tavus' | 'did' as a route parameter for looking up stored avatar profiles.

So Tavus, D-ID, Runway, OpusClip, Shotstack, and the whole unified abstraction are unreachable code presented in the barrel as runtime capabilities — but they are unreachable in two different ways, and the first pass collapsed them into one. Re-traced from the entry points:

Provider Constructed? Reachable?
HeyGen main.ts:138, main.ts:231 behind hasHeyGenKey() Yes — the only live one
Tavus, D-ID only in their own lazy default-singletons and in unified.ts:247 No — no construction outside providers/
Runway services/broll.ts:188 No — nothing imports services/broll
OpusClip services/youtube-shorts.ts:469 No — nothing imports services/youtube-shorts
Shotstack services/video-rendering.ts:243, services/video-pipeline.ts:879 No — nothing imports either module
unified.* factories only inside unified.ts and re-exported by providers/index.ts:30-33 No

main.ts imports exactly three services — article-video, avatar-profiles, video-script — and server.ts imports the same three as types. The barrel (src/index.ts) re-exports only those three service modules. Four service modules with live provider construction inside them (broll, youtube-shorts, video-rendering, video-pipeline) have zero importers anywhere in the repo, so the dead weight is a layer above the provider clients, not in them. That matters for the wire-or-retire decision: wiring Runway or Shotstack means adopting an unimported service module, not just constructing a client.

Per 182.C.13.02 and 182.C.31.07 the decision is wire-or-retire — and both are V1.2-gated, since Veritas does not ship until V1_DEFERRED_DOMAIN_RELEASE.

182.C.02.08 V1 persona / avatar / voice documentation vs code#

Claim area Documented Code reality Verdict
Persona/avatar/voice packs V1/architecture/persona-avatar-voice-packs.md (294 lines) persona-lifecycle-store.ts (886 lines) persists through @oshun/persistence/durable-snapshot-store delivered
Voice-consent evaluation implied by the packs doc voice-consent-store.ts (102 lines) is a pure evaluator, wired at routes/admin-studio-voice-consent.ts:99 delivered, but no ledger store
Voice cloning implied apps/oshun/bff/src/isis/voice-cloning-store.ts is an unbound projection: no consent ledger, no biometric store missing
Viseme mapping 39 phonemes → 11 visemes animation data in libs/oshun/embodiment-psyche/src/avatar-sync.ts delivered, but not a renderer
Avatar rendering Psyche avatar engine cannot start (F-04) missing
Talking-head provider HeyGen via Veritas constructed, but Veritas is V1.2-deferred provider-gated
Active-scene human video — nothing missing
Exact-dialogue lip sync — nothing measured; F-01 fabricated the only in-repo score missing

Two things the first pass's table got wrong by reading only file names:

  • voice-consent-store.ts is not a store. It is a deterministic evaluator (evaluateVoiceConsent, :68) over a ledger it is handed: it classifies each record active/pending/expired/revoked at a logical sequence number, names the covering records, and explains denial as no_scope_match or no_active_consent. Real domain logic, genuinely wired through routes/admin-studio-voice-consent.ts:99 — but it persists nothing, so the consent ledger 182.C.26.01 needs still has no home. Consent evaluation is delivered; consent custody is missing.
  • persona-lifecycle-store.ts really does persist (886 lines over @oshun/persistence/durable-snapshot-store, with a persona_lifecycle_persistence_failed fail-closed path at :110), so the packs row is delivered, not partial.

A second evaluateVoiceConsent exists in libs/euterpe/provenance/src/rights-consent/ with an incompatible signature — (record: VoiceConsentRecord, use: UsageClass, territory: TerritoryCode, at: string) (rights-consent.ts:164) against the BFF's single input object and sequence-number clock. Same name, different clock model (wall-clock ISO vs logical seq) and different scope vocabulary. This is F-12's pattern in the consent layer, and 182.C.26 must choose one before either is treated as the authorization of record.

182.C.02.09 Capability matrix#

CONTENT_GENERATION_CAPABILITY_MATRIX.md already records "Video generation and encoding — missing — No real generated-video plus encoder evidence is registered," and "CLIP/aesthetic/VMAF/PESQ/STOI/NLI — seam-only." Both are accurate and stay as they are: this audit produced no live generation and no encoder receipt, so nothing may be upgraded. Video temporal consistency is recorded as implemented against libs/shared/content-eval/src/video/temporal.test.ts, which asserts real inter-frame MSE — that claim survives. Re-verified: the test asserts exact per-transition values ([100, 400], mean 250, max 400) and a typed throw on a single-frame input, so it would fail against a stub.

No hand-edit was made, and none was warranted. The matrix carries the banner "Generated capability truth — do not hand-edit status claims"; it is rendered from scripts/audit/capability-truth-registry.json. This pass produced no live generation and no encoder receipt, so no status may move. The matrix also has no lip-sync, avatar, or human-video row at all, so there was nothing for the F-01 fix to falsify.

F-14 (L) — the video row points at the wrong probe. The registry entry generation-video lists exactly one evidence ref, runpod-provider-probe, but the deployable video lane resolves fal or OpenRouter (video-provider-env.ts), never RunPod. The missing status is correct today either way, so nothing is being misreported — but a future RunPod probe would satisfy this row without proving anything about the lane Phase 182 actually ships, and the OpenRouter probe that 182.C.11 will produce has no ref to land in. 182.C.11 must register an OpenRouter video-probe evidence ref and attach it here; until then the row stays missing.

182.C.02.10 Deletion and consolidation list#

The rule for Phase 182 is that it adds no seventh abstraction (six exist, see F-02). Provisional dispositions, to be ratified by the ADR at 182.C.04.03:

Path Disposition
apps/oshun/bff/.../video-provider-env.ts Keep as the deployable seam. It is the only thing that resolves a real client.
libs/isis/ai-providers/.../video-generation Keep as provider SDK of record (the BFF already imports LTX from it); delete the committed .js/.d.ts.
libs/isis/seedance-provider Keep as the direct lane for audio/video references, edit, and extend — the capabilities OpenRouter does not expose. Audit each module.
libs/isis/ai-video Retired 2026-09-10. Workflow primitives lifted (F-13), lens-physics.ts relocated, the rest deleted — see F-16 for what advanced/ held.
libs/yemaya/video-generation Out of scope, quarantine the ids. Yemaya is film/game; reconcile its registry against the live catalog before anyone reuses it.
apps/veritas/video/src/providers V1.2-gated wire-or-retire (182.C.13.02).
apps/psyche/avatar-engine Retire from capability claims (182.C.31.06).
libs/nous/portrait-animation Relabel planning-only; do not count as lip-sync capability (182.C.31.03–.05).

Consolidation is a contract problem before it is a deletion problem. Per F-12, four live interfaces are named VideoGenerationRequest and three of them disagree about what "duration" means. Deleting a library does not fix that; the ADR at 182.C.04.03 must name one request/result contract — the BFF's ProviderVideoRequest is the narrowest and the only one on the deployable path, so it is the natural base — and every kept abstraction adapts to it at its edge. Two consequences for the table above:

  • libs/isis/ai-video cannot be retired until video-workflow-manager.ts (F-13) is lifted out. Lift the queue, enqueueBatch, generateFromScenes, and the WorkflowTemplate execution model; leave WORKFLOW_TEMPLATES behind and re-author the templates against the 182.C stage order.
  • The committed .js/.d.ts under libs/isis/ai-providers/.../video-generation (F-03) are deleted as part of that ADR, not before it — a stale .js winning module resolution is exactly the failure mode that makes an unratified deletion risky.

182.C.02.11 BFF generation primitives — extend or replace#

Every one of these is real, fail-loud, and an extension point. The decision for all fourteen is extend; nothing here is rebuilt.

Primitive Lines What it already does Phase 182 extension
jobs-route.ts job envelope + status 768 Six-state enum, PostgreSQL snapshot, tenant/owner attribution, subject deletion Add the fourteen human-video stages (182.C.05.10)
generation-budget-gate.ts 415 Per-tier spend gate, fail-closed Video cost estimate at admission (182.C.15.06)
generation-quota-gate.ts 323 Per-user quota Per-model, per-tenant concurrency (182.C.15.06)
request-validation.ts 411 Validates before any provider call Human-video request schema (182.C.05.01)
generated-artifact-erasure.ts 61 Artifact erasure Propagate to derivatives and provider copies (182.C.16.09)
media-releasability.ts 208 Releasability decision Add speech/sync/identity/consent gates (182.C.28.04)
promotion-gate.ts 167 Promotion decision Per-cohort promotion (182.C.34.07)
provider-measurement.ts 167 Builds the release measurement Carry evaluator versions and coverage (182.C.01.05)
synthesis-disclosure.ts 154 Synthetic-media disclosure Geography/channel policy (182.C.28.02)
video-watermark.ts 219 Real spread-spectrum mark; lossless H.264 4:4:4; measured coverage Keep; document the lossless tradeoff to creators
audio-c2pa-signer.ts 221 Media-agnostic C2PA signing, reused by video Bind script/audio digests (182.C.28.01)
audio-probe.ts / video-probe.ts 445 / 521 ffprobe-based real measurement Reject truncated/HTML/silent/zero-frame (182.C.11.09)
dub-faithfulness.ts 218 ASR seam that throws DubFaithfulnessNotConfiguredError when unset The exact-speech evaluator's home (182.C.18.01)
narration-executor.ts + env 95 / 289 TTS path with a disclosure gate Deterministic TTS for exact dialogue (182.C.08.03)

dub-faithfulness.ts is the model the rest of this phase should copy: a credential-gated capability behind an injectable seam that throws when unconfigured, with the credential-free part (token-cosine similarity) done for real.

F-10 (M). release-proof.test-fixtures.ts names nine gates — safety-scan, provenance-c2pa, watermark-coverage, synthesis-disclosure, quality-aggregate, policy-compliance, rights-and-license, output-shape-valid, human-review-ready — mapped onto ProofRecordSchema's fourteen proof types in libs/contracts/src/common/proof-types.ts. None of them covers exact speech, audiovisual sync, speaker attribution, identity preservation, or likeness consent. Those five are the gates 182.C.28.04 must add, and ConsentProofSchema and ObservedOutcomeProofSchema already exist to carry them.

182.C.02.12 ASR vendor decision — DECIDED AND FUNDED 2026-09-10#

Decided 2026-09-10: OpenRouter audio-input models, funded and live. The user supplied a working OpenRouter key, and a round-trip probe with known ground truth settled both halves of the question — see evidence/2026-09-10-openrouter-asr/.

  • Transcription is fit for purpose. mistralai/voxtral-small-24b-2507 (primary, dedicated speech model) and google/gemini-3.1-flash-lite (4.4× cheaper, equally correct) both transcribed the probe sentence exactly. 46 audio-input models are on the account, so the 182.C.18.07 ensemble is a routing choice, not a procurement problem.
  • Word timings are not. Three recognizers disagreed by up to 1013 ms on word onsets for the same 3.95 s audio — 8.1× the ITU-R BT.1359-1 reject boundary of 125 ms. They are generated timestamps from chat-shaped models, not forced alignment. 182.C.18.02 therefore still needs a real aligner (WhisperX / whisper-timestamped / MFA) on the GPU worker, and that residual blocker is now blocked-local rather than blocked-vendor.

The probe also found a real defect: two recognizers rendered the spoken "forty two dollars" as "$42", and the content-word normalizer stripped the symbol as punctuation — scoring CWER > 0 on a perfect transcription. Fixed, with the measured transcripts pinned as tests.

The constraint set this decision replaces:

  • The OpenAI account authenticates but has no credits (2026-08-10); there is no Deepgram account. apps/veritas/video/src/providers/{whisper,assemblyai,deepgram}.ts are the only ASR clients in the repository, and Veritas is V1.2-deferred.
  • apps/psyche/avatar-engine/src/avatar_engine/lipsync/phonemes.py already contains a WhisperPhonemeExtractor that lazily imports whisper and NLTK CMUdict — a self-hosted path with word- and phoneme-level output already partly exists in Python.
Option Cost shape Word timings Privacy Adds a vendor
Self-hosted Whisper on the RunPod GPU worker Shares 182.C.17.13's host yes Voice never leaves our infrastructure no
AssemblyAI New funded account yes Third-party processes consented voice audio yes
OpenRouter audio-input model Existing funded account unreliable Third-party; video is already non-ZDR no

Recommendation: self-hosted Whisper on the same RunPod worker 182.C.17.13 provisions. It reuses one hosting decision instead of adding a vendor, keeps consented voice audio inside our own boundary — which matters because 182.C.27 must classify voice recordings as biometric data — and gives the word-level timestamps that 182.C.08.07 and 182.C.18.02 need and that a chat-shaped audio model does not reliably return. It also collapses two blockers into one: blocked-vendor becomes part of blocked-local.

Superseded by the 2026-09-10 decision above. The self-hosted-Whisper recommendation is not discarded — it returns as the route for the forced aligner 182.C.18.02 needs, which is the one capability no OpenRouter audio model provides.


Fixes Applied in This Pass#

  1. F-01 — libs/isis/seedance-provider/src/lip-sync-validation.ts: deleted createDefaultLipSyncService, estimateLipSyncFromAudio, estimateAudioDurationSeconds, and buildEstimatedPhonemes, plus the now-unreachable phonemeToViseme helper and its duplicate viseme table (the canonical mapping lives in @oshun/embodiment-psyche's avatar-sync.ts). The lipSyncService option is now required, and its absence throws SeedanceLipSyncServiceNotConfiguredError rather than silently substituting an invented phoneme track. Both existing tests already injected a real service, and SeedanceVideoProvider forwards whatever it was constructed with, so the only behaviour that changed is that an unconfigured deployment now fails loud instead of returning a fabricated score.
  2. F-06 — apps/oshun/bff/src/generation/video-provider-env.ts: corrected the stale header claim that no video watermarker or C2PA signer exists. The first pass missed a second copy of the same claim in the adaptVideoProviderGenerate docstring ("There is still no in-repo video watermarker/C2PA signer, so those floors stay honestly absent"), which was false in a function that already takes a governance parameter and calls applyVideoGovernance. Corrected on re-verification of 182.C.02.02 to state the real condition: both capabilities exist and engage when their keys are bound and the provider returns inline bytes, and stay honestly null/false — still blocking release — otherwise.
  3. F-11 — scoreShapeMatch in the same seedance module scores agreement between a phoneme's implied mouth shape and the measured shape with hand-chosen constants (0.75 floor on an exact match, 0.55 for a confusable pair, 0.18 otherwise). Its inputs are real, but the constants were never fitted against human ratings. Now documented as an ordinal ranking that must never be reported as an LSE-class measurement, superseded by 182.C.17.

A regression test pins the fail-loud behaviour (lip-sync-validation.test.ts, "fails loud when no lip-sync service is bound, instead of inventing phonemes"), asserting both the typed error and that the validator refuses before spending anything on downloading media.

Everything else is recorded against its owning 182.C item rather than fixed here.