Audited 2026-09-09. Scope: 182.C.02.01–182.C.02.12.
This is the honest inventory of what the repository can do today for controlled human video, what it only appears to do, and what must be built. Every verdict below was reached by reading the source, not by file existence, grep counts, or a module's own doc comment. Where a comment and the code disagree, the code wins and the disagreement is recorded as a finding.
Severity: H = ships a fabricated or false capability signal; M = real but unreachable, duplicated, or stale; L = cosmetic or documentation.
Summary of Findings#
| # | Sev | Finding | Item |
|---|---|---|---|
| F-01 | H | @isis/seedance-provider silently substituted a fabricated phoneme track when no lip-sync service was injected |
02.03 |
| F-02 | M | Five provider abstractions already exist for video; a sixth (libs/yemaya/video-generation) was not in the plan |
02.03/.10 |
| F-03 | M | 117 .js and 118 .d.ts build artifacts are committed beside source under libs/isis |
02.03 |
| F-04 | H | apps/psyche/avatar-engine has four unrunnable Nx targets (three missing modules, one silent no-op) and zero tests |
02.06 |
| F-05 | M | Every Veritas video provider except HeyGen is unreachable; four service modules that construct them have no importer | 02.07 |
| F-06 | L | video-provider-env.ts claimed there is no video watermarker or C2PA signer in-repo; both exist and are wired |
02.02 |
| F-07 | M | The OpenRouter adapter's download path uses unsigned_urls, not the documented /content endpoint, and is unprobed |
02.02 |
| F-08 | M | generate_audio defaults to true on audio-capable models and the adapter never sets it |
02.02 |
| F-09 | M | @nous/portrait-animation is 22 modules and 12,248 lines with zero inference I/O — every module is plan, score or metadata |
02.05 |
| F-10 | M | The nine release gates carry no speech, sync, speaker, identity, or consent gate | 02.11 |
| F-11 | L | scoreShapeMatch's constants are an uncalibrated ordinal ranking presented without that caveat |
02.03 |
| F-12 | M | Four incompatible interfaces named VideoGenerationRequest, with three different duration units and no converter |
02.03/.10 |
| F-13 | M | video-workflow-manager.ts holds the queue/batch/scene/template primitives worth reusing — but its queue is in-process |
02.03/.11 |
| F-14 | L | The capability matrix's generation-video row cites a RunPod probe; the deployable lane is fal/OpenRouter |
02.09 |
| F-15 | M | apps/oshun/web type-checks @isis/ai-video against a shim whose DurationPreset admits number — a fifth vocabulary |
04.03 |
| F-16 | H | @isis/ai-video's ensemble-hdr.ts fabricates VMAF, tone mapping, HDR validation and EXR export when no seam is injected |
04.03 |
182.C.02.01 Curated video card types and validation path#
Files read: libs/isis/curated-cards/src/card-types.ts (228 lines),
card-validators.ts (354 lines).
Verdict: real, and correctly narrow. Two card kinds exist, video
(text-to-video) and video-from-image. Every curated field is checked against
its own enum in validateSharedVideoParams, and CARD_INPUT_FIELD_ALLOWLIST
checks the set of submitted fields, not only their values — so a client cannot
smuggle providerKey, steps, or a raw scheduler through the form. The
image-to-video validator additionally constrains referenceImage to an https:
URL or a data:image/ URI. This is a good foundation and must not be widened
carelessly.
What is missing for human video, field by field:
| Needed by Phase 182 | Present today |
|---|---|
| Script / exact dialogue text, per-speaker turns | none |
| Approved speech audio asset + digest | none |
| Multiple typed references (identity, face-angle, full-body, wardrobe, …) | one referenceImage: string |
| Cast and speaker IDs | none |
| Pronunciation lexicon | none |
| Performance direction (blocking, gaze, gesture, emotion, beat timing) | one cameraMovement enum |
| Shot plan / multi-shot | none |
requiredCapabilities vs preferences |
none |
| Quality policy (thresholds, candidate count, retry budget) | one boolean hero |
| Output preferences (renditions, captions, stems) | none |
| Consent / authorization binding | none |
Bounds that will constrain routing: VIDEO_DURATIONS_SECONDS is
[4, 5, 10] only; VIDEO_RESOLUTIONS is ['720p','1080p']; VideoAspect is
four ratios. The live OpenRouter catalog offers 2–30 s, 480p–4K, and nine aspect
ratios, so the curated card is currently the tighter constraint, and any
duration a human video needs beyond 10 s has no card representation at all.
182.C.02.02 BFF video executor, provider environment, router, storage, governance, release gates#
Files read: video-executor.ts (186), openrouter-video-provider.ts (364),
video-provider-env.ts (358), provider-quality-router.ts (184),
video-governance.ts (117), video-watermark.ts (219), jobs-route.ts (768,
status enum and job envelope).
Real code paths. The executor validates before any provider call, fails
closed with provider_not_configured when no provider is injected, and refuses
a result with an empty URL or missing key frame. video-governance.ts composes
a real spread-spectrum video watermarker and the shared C2PA signer, each
independently fail-closed to null. provider-quality-router.ts is a genuine
Beta–Thompson sampler over measured gate-pass rates with a JSON-safe snapshot.
The OpenRouter adapter implements create → poll → download with real backoff, a
real deadline, and loud failures — no fabricated URL, no fabricated completion.
Seams and fixtures, not capability. VideoProviderGenerate is injected at
the deployable boundary; with no key the whole video path is inert. Nothing in
this stack has a committed live-probe fixture, so under the tracker's status
vocabulary the OpenRouter route is implemented, never live-probed.
F-06 (L). The video-provider-env.ts header stated "There is still NO video
watermarker and NO C2PA video signer in-repo, so provenanceManifestPresent
stays honestly false and watermarkCoverage stays honestly null." Both now
exist (video-watermark.ts, audio-c2pa-signer.ts) and are wired through
video-governance.ts:115. The comment was stale and read as a capability claim
in the wrong direction. Fixed in this pass.
F-07 (M). The adapter downloads from unsigned_urls entries on the poll
response. The current documented contract of record is
GET /api/v1/videos/{id}/content?index=0. Both may work; neither is proved
here. The adapter must be reconciled against a live probe at 182.C.11, and the
frame_images entry shape it sends — { type, image_url: { url }, frame_type }
— must be probed too, since the documented input_references entry is
{ type: 'image_url', image_url } with image_url not obviously an object.
F-08 (M). generate_audio defaults to true on audio-capable models and
the adapter never sets it. The default model is alibaba/wan-2.7, whose catalog
record has generate_audio: true, so every clip generated today comes back with
model-invented audio that nothing asked for. On Wan 2.7's flat
duration_seconds price this costs nothing extra, but on Veo 3.1 the same
omission is the difference between $0.20/s and $0.40/s, and for exact-dialogue
mode an unrequested generated audio track is a correctness problem, not only a
cost one.
Base64 delivery. download() returns data:video/mp4;base64,…. A 10-second
1080p clip is tens of megabytes, inflated ~33% by base64, and it travels through
the job record. This is the single biggest blocker to real evaluation and is
already scoped as 182.C.11.08 / 182.C.16.02.
Job envelope. GenerationJobStatus is exactly six states — queued,
running, complete, failed, blocked, needs_review — with blocked and
needs_review already carrying the release-gate decision. Jobs live in an
in-memory Map with a PostgreSQL snapshot and a documented disposition record.
This is the right thing to extend at 182.C.05.10, not to replace.
182.C.02.03 @isis/ai-video, @isis/ai-providers, @isis/seedance-provider#
F-01 (H) — the fabricated lip-sync track#
libs/isis/seedance-provider/src/lip-sync-validation.ts is 811 lines. Its
visual half is real: extractFramesWithFfmpeg spawns ffmpeg,
classifyMouthShape uses sharp on decoded frames, and the mouth-cue analyzer
measures actual pixels.
Its audio half fabricated. validateSeedanceLipSync resolved
options.lipSyncService ?? createDefaultLipSyncService(), and that default
called estimateLipSyncFromAudio, which:
- derived duration as
byteLength / 16000with no container parse, so the number is wrong for any real WAV, MP3, or AAC payload; - generated a phoneme track by cycling the fixed pattern
['m','aa','s','ow','t','eh','n','uw']— the phonemes bear no relation to the audio, which is invented ground truth, not a measurement; - attached
confidence: 0.7 + (index % 3) * 0.08, a value derived from the array index (line 324, the one the plan already flagged); - returned
syncQualityScore: clamp(0.78 + durationSecs / 60 - 1 / frameRate, …), a quality score computed from duration and frame rate alone.
The consequence was worse than any single line. The real mouth-cue analyzer then
scored real video frames against that invented phoneme track, and
phonemeAccuracy, utteranceConfidenceScore, and overallConfidenceScore came
back looking like measurements. Both in-repo tests inject a real
lipSyncService, and SeedanceVideoProvider passes this.lipSyncService,
which is optional — so the fabricated path was precisely the production
path.
Fixed in this pass: the silent default is removed and the seam now throws
SeedanceLipSyncServiceNotConfiguredError. See "Fixes applied" below.
F-02 (M) — provider abstractions#
Six, not five. 182.C.02.10's consolidation list must include the one the plan
did not name:
| Abstraction | Size | Status |
|---|---|---|
libs/isis/ai-video |
22 TS files | Runway client, next-gen adapters, SVD self-hosted server, workflow mgr |
libs/isis/ai-providers/.../video-generation |
5 TS + 10 build artifacts | LTX (used by the BFF), Hunyuan, Wan providers |
libs/isis/seedance-provider |
92 files (46 src + specs) | fal, PiAPI, Volcengine Ark clients + editing/reference modules |
apps/veritas/video/src/providers |
13 files | HeyGen, Tavus, D-ID, Runway, OpusClip, Shotstack, stock, unified, ASR |
apps/oshun/bff/.../video-provider-env.ts |
358 lines | The deployable seam that actually resolves a client |
libs/yemaya/video-generation |
18 TS files | Its own model registry and provider router — not in the plan |
libs/yemaya/video-generation/src/model-registry/model-registry.ts maps
wan-2.6-t2v → wan-ai/wan2.6-t2v and similar AIML endpoints. Those are
fal/AIML-style ids and are unrelated to OpenRouter's alibaba/wan-2.6; the same
stale-id pattern the plan flagged in the Wan provider recurs here. Both id
vocabularies must be reconciled against the live catalog rather than trusted.
F-03 (M) — committed build artifacts#
find libs/isis -name '*.js' -not -path '*/dist/*' → 117 files;
-name '*.d.ts' → 118.
libs/isis/ai-providers/src/providers/video-generation/ carries
hunyuan-provider.js, hunyuan-provider.d.ts, index.js, index.d.ts,
ltx-provider.js, ltx-provider.d.ts, types.js, types.d.ts,
wan-provider.js, wan-provider.d.ts beside their sources. A stale .js
beside a .ts can win module resolution and silently serve last month's code.
These are build output and belong in dist.
F-12 (M) — four incompatible VideoGenerationRequest contracts#
The name is shared; the shape is not. Nothing converts between them, so every new caller re-derives the mapping by hand and the duration unit changes meaning across the boundary:
| Contract | Duration field | Model field | Other required fields |
|---|---|---|---|
@isis/ai-video video.types.ts:VideoGenerationRequest |
duration: DurationPreset (4|5|10|16) |
none | aspectRatio; fields mutable |
@isis/ai-providers video-generation/types.ts:276 |
duration?: number (seconds) |
model required |
readonly throughout; adds width, height, fps, steps, guidance |
@yemaya/video-generation types.ts:324 |
durationSeconds: number |
modelId | null |
id, mode, fps, steps, guidanceScale, seed, six presets |
BFF video-executor.ts:25:ProviderVideoRequest |
durationSeconds: number |
none (bound at env) | aspect, resolution, cameraMovement, stylePreset, locale |
@isis/ai-video's DurationPreset admits 16, which no curated card can
express (VIDEO_DURATIONS_SECONDS is [4, 5, 10]) and which OpenRouter's
default lane, Wan 2.7, cannot render (2–10 s). The BFF contract is the one the
deployable path actually uses and is the narrowest; the Phase 182 contract at
182.C.05 must be defined once and the other three adapted to it or deleted per
182.C.02.10 — not extended in place.
F-13 (M) — reusable workflow primitives worth keeping#
@isis/ai-video's video-workflow-manager.ts (866 lines) is the one piece of
the duplicated stack with primitives Phase 182 should reuse rather than rewrite:
a priority queue with concurrency control and QueueEvents, enqueueBatch,
generateFromScenes (a scene list → per-shot job fan-out, which is the shot
plan of 182.C.09 in embryo), executeWorkflow over declarative
WorkflowTemplate/WorkflowStepTemplate records with injectable
WorkflowStepHandlers, and cancelJob/waitForJob. Its execution is injected
through the VideoGenerationExecutor port (submit/getJob/cancelJob), so
it carries no provider coupling and can sit above the BFF job envelope.
Two caveats before reuse: its queue is in-process and non-durable, so it is
not a replacement for the durable job envelope in jobs-route.ts (the
extend-or-replace decision in 182.C.02.11 stands), and its
WORKFLOW_TEMPLATES constant encodes pre-Phase-182 stages with no evaluation,
retry, or release-gate step. Reuse the queue, the batch/scene fan-out, and the
template execution model; re-author the templates against the 182.C stage
order.
F-15 (M) — the web app type-checks against a shim that contradicts the library#
Found while inventorying consumers for 182.C.04.03. @isis/ai-video has
exactly one consumer, libs/yemaya/remote-film-capture, and it imports types
only (isis-ai-video-integration.ts:1-10). But the two build paths disagree
about where those types come from:
| Build path | @isis/ai-video resolves to |
DurationPreset there |
|---|---|---|
apps/oshun/web/tsconfig.json:159 |
remote-film-capture/src/isis-ai-video-shim.d.ts |
5 | 10 | 15 | 30 | number |
apps/oshun/web/vitest.config.ts:589 |
the real libs/isis/ai-video/src/index.ts |
4 | 5 | 10 | 16 |
The shim's | number makes the union unconstrained, and its VideoProvider,
AspectRatio, VideoResolution, and VideoStyle are all widened with
| string the same way. So the web app's tsc accepts durations and providers
that the real library rejects, while its vitest run compiles against the real
types — the two can diverge silently, and the typecheck is the weaker of the
two.
This is F-12's problem with an extra twist: it is a fifth duration
vocabulary, and it is the one that shadows the real library in the build that
gates CI. It must be reconciled before @isis/ai-video is retired, because
retiring the real library while the shim stands would leave the web app
type-checking against a declaration file for a package that no longer exists.
F-16 (H) — the retirement scope was assessed on the provider clients alone#
Found while executing 182.C.04.03 step 4, reading libs/isis/ai-video before
deleting it. F-02's one-line inventory ("Runway client, next-gen adapters, SVD
self-hosted server, workflow mgr") did not cover src/advanced/, which is 3,691
lines across three modules. They do not share a disposition:
| Module | Lines | Disposition on reading |
|---|---|---|
lens-physics.ts |
727 | Kept. Real optics, no provider coupling, no fabricated value — relocated to @isis/video-enhancement. |
ensemble-hdr.ts |
728 | Deleted. F-01's pattern again: fabricated defaults behind optional seams (below). |
video-generation-sota.ts |
2,236 | Deleted. Option builders over the retiring VideoProviderClient/RunwayClient; provider-shaped, no unique math. |
lens-physics.ts computes thin-lens image distance, hyperfocal distance, circle
of confusion in millimetres and in sensor pixels, cat-eye and oval bokeh from
blade geometry, shutter-angle motion blur, focus breathing, Brown–Conrady
distortion with per-channel chromatic offsets, and anamorphic squeeze. Every
output is derived from the inputs. Its nearest neighbour,
@aphrodite/auteur-camera/lens-simulation.ts (785 lines), manages renderer
parameters for a live rig rather than computing optics, so it is not a duplicate
and deleting the library would have lost the work.
ensemble-hdr.ts reads as the opposite once its defaults are followed:
estimatePerceptualSignalsreturnsvmaf: quality * 100, wherequalityis a number the provider put in its own result metadata.scorePerceptualQualitythen publishes that asvmaf— a named algorithm that requires a reference and a distorted video and was never run. The reader seam is optional, so this is the default path.defaultToneMap"tone maps" by rewriting the file extension to.hdr.mp4and echoing the requestedpeakNitsback as fact.defaultHdrValidationreportsmeasuredPeakNitsequal to the valuedefaultToneMapjust echoed from the config, and passes or fails against asupportsHdrflag from a hand-written table.defaultExrExportreturns an.exrpath for a file it never wrote.DEFAULT_PROVIDER_PROFILEShard-codes price andestimatedQualityfor twelve providers with no date and no source — the mutable-fact-frozen-in-code problem ADR-0006 forbids, and superseded byprovider-quality-router.ts, which selects on measured release rates.
Both in-repo tests inject the real seams, so the tests pass while the default
path fabricates — the same shape as F-01, and the reason this module was deleted
with the provider stack rather than relocated beside lens-physics.ts.
182.C.02.04 Identity, scene, shot, wardrobe, and temporal consistency modules#
libs/isis/workflows/src/consistency/ holds ten modules, each with a spec:
identity-profiles, identity-drift, scene-profiles, shot-profiles,
wardrobe-profiles, temporal-drift, temporal-quality-qa,
golden-output-perceptual-regression, orchestration,
sequence-orchestration.
Mapping to the Phase 182 contract:
| Existing module | Maps to |
|---|---|
identity-profiles |
182.C.07.01 character pack (extend, don't fork) |
identity-drift, temporal-drift |
182.C.19.01 temporal identity drift |
scene-profiles, shot-profiles |
182.C.09.01 shot-plan schema |
wardrobe-profiles |
182.C.07.04 mutable shot traits |
golden-output-perceptual-regression |
182.C.34 bakeoff regression |
sequence-orchestration |
182.C.22 long-form continuity ledger |
temporal-quality-qa |
182.C.19.06 motion/temporal consistency |
These are the designated integration targets for 182.C.07.06.
Where the boundary actually falls. The first pass called these "bookkeeping
over supplied scores," which understates identity-drift.ts and overstates the
gap. The real split is between measurement over supplied features and feature
extraction from pixels:
- Real math, already here.
identity-drift.tscomputes cosine similarity over embedding vectors (:112-131) and mean euclidean landmark displacement (:134-152), weights them (:98-99), and resolves a severity and a review decision against configured thresholds.temporal-quality-qa.tscomputes pairwise frame metrics and a weighted assessment the same way. - Not here, and the reason
182.C.19cannot be satisfied by these modules. Nothing in the directory extracts a face embedding or a landmark set from a decoded frame. Every vector and every landmark point arrives as an input, so the accuracy of the whole assessment is inherited from an extractor that does not yet exist in the repo — which is precisely the GPU-hosted work182.C.19scopes and182.C.01.10marksblocked-local.
So 182.C.19 must supply the extractor and may reuse these assessors rather
than reimplementing the comparison math.
182.C.02.05 @nous/portrait-animation#
22 exported modules, 12,248 lines of non-spec source. F-09 (M): not one
performs inference. A scan for process spawning, HTTP, or model runtime across
every non-spec module returns zero hits except three incidental matches in
portrait-animation-streaming-output-pipeline.ts.
Labels per 182.C.02.05:
| Label | Modules |
|---|---|
| planning-only | hallo3-portrait-animation-inference-pipeline, hallo3-video-dit-model-loader, im-portrait-3d-aware-video-diffusion, liveportrait-expression-transfer-pipeline, liveportrait-stitching-animation-pipeline, multi-style-…, multi-person-…, musetalk-video-dubbing-pipeline, echomimic-audio-to-facial-motion-pipeline, chatanyone-realtime-portrait-pipeline, rap-realtime-audio-driven-portrait-pipeline, realtime-audio-stream-portrait-pipeline, partial-face-editing-pipeline, portrait-animation-expression-amplitude-control, portrait-animation-gpu-batching-orchestrator, portrait-reference-preprocessing-pipeline, cross-language-dubbing-pipeline, dubbing-batch-processor |
| score-ingestion-only | portrait-animation-quality-metrics-pipeline, dubbing-quality-validation-pipeline |
| executable | none |
| adapter-backed | none |
| stale | none found; the modules are internally consistent |
| metadata-only | portrait-animation-library (17 lines: library name, Nx project name, tags, and a metadata getter — no animation behaviour at all) |
The first pass's table labelled 20 of the 22 modules. The two it omitted, read and labelled here:
portrait-animation-library.ts— metadata-only. Seventeen lines of Nx identity constants andgetPortraitAnimationLibraryMetadata(). It is not a pipeline and should not be counted as one in any capability claim.portrait-animation-streaming-output-pipeline.ts— planning-only, same as the other eighteen. Its 653 lines resolve a WHIP/WHEP/LL-HLS transport plan throughresolvePlan(:399) and define a…Statsintake type; the three "HTTP" matches the scan flagged are spec URL constants (:5-11) and the literal'single-http-post-with-ice-patch-updates'describing WHIP signalling, not a request the module makes. Nothing here opens a socket or a peer connection.
With those two placed, the count is 22: nineteen planning-only, two score-ingestion-only, one metadata-only, and still zero executable or adapter-backed modules.
The naming is honest — the public method is resolvePlan and the return type is
…Plan — so this is not a fabrication. But it is also not evidence of lip sync.
PortraitAnimationQualityMetricsPipeline.resolvePlan takes
request.observedScores and compares them to thresholds (lseC 6, lseD 8,
ArcFace cosine 0.35, FVD 120, head-pose MAE 8°); it cites the SyncNet, Wav2Lip,
ArcFace, FVD, and 6DRepNet references but never runs any of them. Under
182.C.01.05 a score this library was handed can never populate a metric field.
This is exactly what 182.C.31.03–182.C.31.05 must resolve.
182.C.02.06 apps/psyche/avatar-engine#
F-04 (H). 157 files of real-looking Python: 3D Gaussian Splatting, FLAME face modelling, NeRF, a lip-sync subpackage with a genuine lazily-imported Whisper phoneme extractor plus NLTK CMUdict and librosa. Dependencies are torch, pytorch3d, open3d, insightface, mediapipe, face-alignment, smplx, chumpy.
Four of its declared Nx targets cannot run. Three point at modules that do not exist; the fourth points at real code with no entry point:
| Target | Command | Reality |
|---|---|---|
serve / serve-prod |
uvicorn avatar_engine.main:app |
no main.py in src/avatar_engine/ |
download-models |
python -m avatar_engine.scripts.download_models |
no scripts/ package |
train |
python -m avatar_engine.training.trainer |
trainer.py exists (692 lines) but defines only classes — no __main__ |
test / -no-gpu/-cov |
pytest |
zero test files; tests/ holds only a .gitkeep |
The train row corrects the first pass, which reported trainer.py as missing.
It is there and it is substantial — AvatarDataset(Dataset), a
BlendshapeModel(nn.Module), AvatarTrainer, and a DistributedTrainer
subclass. What it lacks is a main() and an if __name__ == '__main__' guard,
so python -m avatar_engine.training.trainer imports the module, runs nothing,
and exits 0 — a silent no-op, which is worse than the import error the other two
targets produce, because it looks like a successful training run.
So the service cannot be started, cannot acquire weights, cannot be trained
through its own target, and has never been tested. It is reachable in principle
from libs/psyche/avatar-training/src/backend.ts, which speaks HTTP to it
(:399 health, :611 request) — but only against the main:app that does not
exist.
Licensing is unresolved and material. FLAME (models/flame.py loads a
.pkl by path) and smplx are distributed under research licenses that
restrict commercial use; insightface models carry their own non-commercial
terms. libs/shared/audit-platform/src/model-license-review.ts is the right
place to record this — it already models weight-redistribution rights,
commercial-output rights, and quarantine, and names avatar-engine weights
explicitly — but no review record exists.
Recommendation for 182.C.31.06: retire it from capability claims for this
phase. Hardening it means writing an entry point, a weight-acquisition script,
a licence review, and an inference test suite for a GPU service on hardware this
project does not have. Nothing in Phase 182's critical path needs it.
182.C.02.07 Veritas avatar/video providers#
F-05 (M). apps/veritas/video/src/providers/ contains heygen.ts,
tavus.ts, did.ts, runway.ts, opusclip.ts, shotstack.ts,
stock-footage.ts, unified.ts, plus the three ASR clients.
Only HeyGen is constructed at runtime: main.ts:138 and main.ts:231 call
createHeyGenClient(...) behind hasHeyGenKey(). createUnifiedVideoProvider,
getUnifiedVideoProvider, and createUnifiedProviderFromEnv have no callers
anywhere in the repository outside unified.ts and its own barrel re-export.
server.ts only uses the string union 'heygen' | 'tavus' | 'did' as a route
parameter for looking up stored avatar profiles.
So Tavus, D-ID, Runway, OpusClip, Shotstack, and the whole unified abstraction are unreachable code presented in the barrel as runtime capabilities — but they are unreachable in two different ways, and the first pass collapsed them into one. Re-traced from the entry points:
| Provider | Constructed? | Reachable? |
|---|---|---|
| HeyGen | main.ts:138, main.ts:231 behind hasHeyGenKey() |
Yes — the only live one |
| Tavus, D-ID | only in their own lazy default-singletons and in unified.ts:247 |
No — no construction outside providers/ |
| Runway | services/broll.ts:188 |
No — nothing imports services/broll |
| OpusClip | services/youtube-shorts.ts:469 |
No — nothing imports services/youtube-shorts |
| Shotstack | services/video-rendering.ts:243, services/video-pipeline.ts:879 |
No — nothing imports either module |
unified.* factories |
only inside unified.ts and re-exported by providers/index.ts:30-33 |
No |
main.ts imports exactly three services — article-video, avatar-profiles,
video-script — and server.ts imports the same three as types. The barrel
(src/index.ts) re-exports only those three service modules. Four service
modules with live provider construction inside them (broll, youtube-shorts,
video-rendering, video-pipeline) have zero importers anywhere in the
repo, so the dead weight is a layer above the provider clients, not in them.
That matters for the wire-or-retire decision: wiring Runway or Shotstack means
adopting an unimported service module, not just constructing a client.
Per 182.C.13.02 and 182.C.31.07 the decision is wire-or-retire — and both
are V1.2-gated, since Veritas does not ship until
V1_DEFERRED_DOMAIN_RELEASE.
182.C.02.08 V1 persona / avatar / voice documentation vs code#
| Claim area | Documented | Code reality | Verdict |
|---|---|---|---|
| Persona/avatar/voice packs | V1/architecture/persona-avatar-voice-packs.md (294 lines) |
persona-lifecycle-store.ts (886 lines) persists through @oshun/persistence/durable-snapshot-store |
delivered |
| Voice-consent evaluation | implied by the packs doc | voice-consent-store.ts (102 lines) is a pure evaluator, wired at routes/admin-studio-voice-consent.ts:99 |
delivered, but no ledger store |
| Voice cloning | implied | apps/oshun/bff/src/isis/voice-cloning-store.ts is an unbound projection: no consent ledger, no biometric store |
missing |
| Viseme mapping | 39 phonemes → 11 visemes | animation data in libs/oshun/embodiment-psyche/src/avatar-sync.ts |
delivered, but not a renderer |
| Avatar rendering | Psyche avatar engine | cannot start (F-04) | missing |
| Talking-head provider | HeyGen via Veritas | constructed, but Veritas is V1.2-deferred | provider-gated |
| Active-scene human video | — | nothing | missing |
| Exact-dialogue lip sync | — | nothing measured; F-01 fabricated the only in-repo score | missing |
Two things the first pass's table got wrong by reading only file names:
voice-consent-store.tsis not a store. It is a deterministic evaluator (evaluateVoiceConsent,:68) over a ledger it is handed: it classifies each record active/pending/expired/revoked at a logical sequence number, names the covering records, and explains denial asno_scope_matchorno_active_consent. Real domain logic, genuinely wired throughroutes/admin-studio-voice-consent.ts:99— but it persists nothing, so the consent ledger182.C.26.01needs still has no home. Consent evaluation is delivered; consent custody is missing.persona-lifecycle-store.tsreally does persist (886 lines over@oshun/persistence/durable-snapshot-store, with apersona_lifecycle_persistence_failedfail-closed path at:110), so the packs row isdelivered, notpartial.
A second evaluateVoiceConsent exists in
libs/euterpe/provenance/src/rights-consent/ with an incompatible signature —
(record: VoiceConsentRecord, use: UsageClass, territory: TerritoryCode, at: string)
(rights-consent.ts:164) against the BFF's single input object and
sequence-number clock. Same name, different clock model (wall-clock ISO vs
logical seq) and different scope vocabulary. This is F-12's pattern in the
consent layer, and 182.C.26 must choose one before either is treated as the
authorization of record.
182.C.02.09 Capability matrix#
CONTENT_GENERATION_CAPABILITY_MATRIX.md already records "Video generation and
encoding — missing — No real generated-video plus encoder evidence is
registered," and "CLIP/aesthetic/VMAF/PESQ/STOI/NLI — seam-only." Both are
accurate and stay as they are: this audit produced no live generation and no
encoder receipt, so nothing may be upgraded. Video temporal consistency is
recorded as implemented against
libs/shared/content-eval/src/video/temporal.test.ts, which asserts real
inter-frame MSE — that claim survives. Re-verified: the test asserts exact
per-transition values ([100, 400], mean 250, max 400) and a typed throw on
a single-frame input, so it would fail against a stub.
No hand-edit was made, and none was warranted. The matrix carries the banner
"Generated capability truth — do not hand-edit status claims"; it is rendered
from scripts/audit/capability-truth-registry.json. This pass produced no live
generation and no encoder receipt, so no status may move. The matrix also has no
lip-sync, avatar, or human-video row at all, so there was nothing for the F-01
fix to falsify.
F-14 (L) — the video row points at the wrong probe. The registry entry
generation-video lists exactly one evidence ref, runpod-provider-probe, but
the deployable video lane resolves fal or OpenRouter
(video-provider-env.ts), never RunPod. The missing status is correct today
either way, so nothing is being misreported — but a future RunPod probe would
satisfy this row without proving anything about the lane Phase 182 actually
ships, and the OpenRouter probe that 182.C.11 will produce has no ref to land
in. 182.C.11 must register an OpenRouter video-probe evidence ref and attach
it here; until then the row stays missing.
182.C.02.10 Deletion and consolidation list#
The rule for Phase 182 is that it adds no seventh abstraction (six exist,
see F-02). Provisional dispositions, to be ratified by the ADR at 182.C.04.03:
| Path | Disposition |
|---|---|
apps/oshun/bff/.../video-provider-env.ts |
Keep as the deployable seam. It is the only thing that resolves a real client. |
libs/isis/ai-providers/.../video-generation |
Keep as provider SDK of record (the BFF already imports LTX from it); delete the committed .js/.d.ts. |
libs/isis/seedance-provider |
Keep as the direct lane for audio/video references, edit, and extend — the capabilities OpenRouter does not expose. Audit each module. |
libs/isis/ai-video |
Retired 2026-09-10. Workflow primitives lifted (F-13), lens-physics.ts relocated, the rest deleted — see F-16 for what advanced/ held. |
libs/yemaya/video-generation |
Out of scope, quarantine the ids. Yemaya is film/game; reconcile its registry against the live catalog before anyone reuses it. |
apps/veritas/video/src/providers |
V1.2-gated wire-or-retire (182.C.13.02). |
apps/psyche/avatar-engine |
Retire from capability claims (182.C.31.06). |
libs/nous/portrait-animation |
Relabel planning-only; do not count as lip-sync capability (182.C.31.03–.05). |
Consolidation is a contract problem before it is a deletion problem. Per
F-12, four live interfaces are named VideoGenerationRequest and three of them
disagree about what "duration" means. Deleting a library does not fix that; the
ADR at 182.C.04.03 must name one request/result contract — the BFF's
ProviderVideoRequest is the narrowest and the only one on the deployable path,
so it is the natural base — and every kept abstraction adapts to it at its edge.
Two consequences for the table above:
libs/isis/ai-videocannot be retired untilvideo-workflow-manager.ts(F-13) is lifted out. Lift the queue,enqueueBatch,generateFromScenes, and theWorkflowTemplateexecution model; leaveWORKFLOW_TEMPLATESbehind and re-author the templates against the182.Cstage order.- The committed
.js/.d.tsunderlibs/isis/ai-providers/.../video-generation(F-03) are deleted as part of that ADR, not before it — a stale.jswinning module resolution is exactly the failure mode that makes an unratified deletion risky.
182.C.02.11 BFF generation primitives — extend or replace#
Every one of these is real, fail-loud, and an extension point. The decision for all fourteen is extend; nothing here is rebuilt.
| Primitive | Lines | What it already does | Phase 182 extension |
|---|---|---|---|
jobs-route.ts job envelope + status |
768 | Six-state enum, PostgreSQL snapshot, tenant/owner attribution, subject deletion | Add the fourteen human-video stages (182.C.05.10) |
generation-budget-gate.ts |
415 | Per-tier spend gate, fail-closed | Video cost estimate at admission (182.C.15.06) |
generation-quota-gate.ts |
323 | Per-user quota | Per-model, per-tenant concurrency (182.C.15.06) |
request-validation.ts |
411 | Validates before any provider call | Human-video request schema (182.C.05.01) |
generated-artifact-erasure.ts |
61 | Artifact erasure | Propagate to derivatives and provider copies (182.C.16.09) |
media-releasability.ts |
208 | Releasability decision | Add speech/sync/identity/consent gates (182.C.28.04) |
promotion-gate.ts |
167 | Promotion decision | Per-cohort promotion (182.C.34.07) |
provider-measurement.ts |
167 | Builds the release measurement | Carry evaluator versions and coverage (182.C.01.05) |
synthesis-disclosure.ts |
154 | Synthetic-media disclosure | Geography/channel policy (182.C.28.02) |
video-watermark.ts |
219 | Real spread-spectrum mark; lossless H.264 4:4:4; measured coverage | Keep; document the lossless tradeoff to creators |
audio-c2pa-signer.ts |
221 | Media-agnostic C2PA signing, reused by video | Bind script/audio digests (182.C.28.01) |
audio-probe.ts / video-probe.ts |
445 / 521 | ffprobe-based real measurement | Reject truncated/HTML/silent/zero-frame (182.C.11.09) |
dub-faithfulness.ts |
218 | ASR seam that throws DubFaithfulnessNotConfiguredError when unset |
The exact-speech evaluator's home (182.C.18.01) |
narration-executor.ts + env |
95 / 289 | TTS path with a disclosure gate | Deterministic TTS for exact dialogue (182.C.08.03) |
dub-faithfulness.ts is the model the rest of this phase should copy: a
credential-gated capability behind an injectable seam that throws when
unconfigured, with the credential-free part (token-cosine similarity) done for
real.
F-10 (M). release-proof.test-fixtures.ts names nine gates — safety-scan,
provenance-c2pa, watermark-coverage, synthesis-disclosure,
quality-aggregate, policy-compliance, rights-and-license,
output-shape-valid, human-review-ready — mapped onto ProofRecordSchema's
fourteen proof types in libs/contracts/src/common/proof-types.ts. None of them
covers exact speech, audiovisual sync, speaker attribution, identity
preservation, or likeness consent. Those five are the gates 182.C.28.04 must
add, and ConsentProofSchema and ObservedOutcomeProofSchema already exist to
carry them.
182.C.02.12 ASR vendor decision — DECIDED AND FUNDED 2026-09-10#
Decided 2026-09-10: OpenRouter audio-input models, funded and live. The user
supplied a working OpenRouter key, and a round-trip probe with known ground
truth settled both halves of the question — see
evidence/2026-09-10-openrouter-asr/.
- Transcription is fit for purpose.
mistralai/voxtral-small-24b-2507(primary, dedicated speech model) andgoogle/gemini-3.1-flash-lite(4.4× cheaper, equally correct) both transcribed the probe sentence exactly. 46 audio-input models are on the account, so the182.C.18.07ensemble is a routing choice, not a procurement problem. - Word timings are not. Three recognizers disagreed by up to 1013 ms on
word onsets for the same 3.95 s audio — 8.1× the ITU-R BT.1359-1 reject
boundary of 125 ms. They are generated timestamps from chat-shaped models, not
forced alignment.
182.C.18.02therefore still needs a real aligner (WhisperX / whisper-timestamped / MFA) on the GPU worker, and that residual blocker is nowblocked-localrather thanblocked-vendor.
The probe also found a real defect: two recognizers rendered the spoken "forty two dollars" as "$42", and the content-word normalizer stripped the symbol as punctuation — scoring CWER > 0 on a perfect transcription. Fixed, with the measured transcripts pinned as tests.
The constraint set this decision replaces:
- The OpenAI account authenticates but has no credits (2026-08-10); there is no
Deepgram account.
apps/veritas/video/src/providers/{whisper,assemblyai,deepgram}.tsare the only ASR clients in the repository, and Veritas is V1.2-deferred. apps/psyche/avatar-engine/src/avatar_engine/lipsync/phonemes.pyalready contains aWhisperPhonemeExtractorthat lazily importswhisperand NLTK CMUdict — a self-hosted path with word- and phoneme-level output already partly exists in Python.
| Option | Cost shape | Word timings | Privacy | Adds a vendor |
|---|---|---|---|---|
| Self-hosted Whisper on the RunPod GPU worker | Shares 182.C.17.13's host |
yes | Voice never leaves our infrastructure | no |
| AssemblyAI | New funded account | yes | Third-party processes consented voice audio | yes |
| OpenRouter audio-input model | Existing funded account | unreliable | Third-party; video is already non-ZDR | no |
Recommendation: self-hosted Whisper on the same RunPod worker 182.C.17.13
provisions. It reuses one hosting decision instead of adding a vendor, keeps
consented voice audio inside our own boundary — which matters because 182.C.27
must classify voice recordings as biometric data — and gives the word-level
timestamps that 182.C.08.07 and 182.C.18.02 need and that a chat-shaped
audio model does not reliably return. It also collapses two blockers into one:
blocked-vendor becomes part of blocked-local.
Superseded by the 2026-09-10 decision above. The self-hosted-Whisper
recommendation is not discarded — it returns as the route for the forced
aligner 182.C.18.02 needs, which is the one capability no OpenRouter audio
model provides.
Fixes Applied in This Pass#
- F-01 —
libs/isis/seedance-provider/src/lip-sync-validation.ts: deletedcreateDefaultLipSyncService,estimateLipSyncFromAudio,estimateAudioDurationSeconds, andbuildEstimatedPhonemes, plus the now-unreachablephonemeToVisemehelper and its duplicate viseme table (the canonical mapping lives in@oshun/embodiment-psyche'savatar-sync.ts). ThelipSyncServiceoption is now required, and its absence throwsSeedanceLipSyncServiceNotConfiguredErrorrather than silently substituting an invented phoneme track. Both existing tests already injected a real service, andSeedanceVideoProviderforwards whatever it was constructed with, so the only behaviour that changed is that an unconfigured deployment now fails loud instead of returning a fabricated score. - F-06 —
apps/oshun/bff/src/generation/video-provider-env.ts: corrected the stale header claim that no video watermarker or C2PA signer exists. The first pass missed a second copy of the same claim in theadaptVideoProviderGeneratedocstring ("There is still no in-repo video watermarker/C2PA signer, so those floors stay honestly absent"), which was false in a function that already takes agovernanceparameter and callsapplyVideoGovernance. Corrected on re-verification of182.C.02.02to state the real condition: both capabilities exist and engage when their keys are bound and the provider returns inline bytes, and stay honestlynull/false— still blocking release — otherwise. - F-11 —
scoreShapeMatchin the same seedance module scores agreement between a phoneme's implied mouth shape and the measured shape with hand-chosen constants (0.75 floor on an exact match, 0.55 for a confusable pair, 0.18 otherwise). Its inputs are real, but the constants were never fitted against human ratings. Now documented as an ordinal ranking that must never be reported as an LSE-class measurement, superseded by182.C.17.
A regression test pins the fail-loud behaviour (lip-sync-validation.test.ts,
"fails loud when no lip-sync service is bound, instead of inventing phonemes"),
asserting both the typed error and that the validator refuses before spending
anything on downloading media.
Everything else is recorded against its owning 182.C item rather than fixed
here.