Status: Accepted
Date: 2026-09-09
Authors: Isis Platform, Oshun BFF, Contracts
Reviewers: @oshun/core-team, @oshun/isis-team, @oshun/platform,
@oshun/trust-safety
Context and Problem Statement#
Phase 182 adds controlled human video — authorized likeness, exact dialogue,
directed performance, quality-gated delivery. The repository already contains
six video provider abstractions and, worse, four live interfaces named
VideoGenerationRequest that disagree about what "duration" means (F-12 in
docs/domains/isis/human-video/capability-audit.md):
| Contract | Duration field | Model field |
|---|---|---|
@isis/ai-video video.types.ts |
duration: DurationPreset (4|5|10|16) |
none |
@isis/ai-providers video-generation/types.ts:276 |
duration?: number (seconds) |
model required |
@yemaya/video-generation types.ts:324 |
durationSeconds: number |
modelId | null |
BFF video-executor.ts:25 ProviderVideoRequest |
durationSeconds: number |
none (bound at env) |
Nothing converts between them. Every new caller re-derives the mapping by hand,
and @isis/ai-video's DurationPreset admits 16 — a value no curated card
can express (VIDEO_DURATIONS_SECONDS is [4, 5, 10]) and the default lane,
alibaba/wan-2.7, cannot render (2–10 s).
Adding a seventh abstraction for human video would make this permanent. So would extending one of the four in place.
Decision#
One canonical contract per concern, defined once, adapted at every edge. Five contracts, with a named owner and a named home for each.
1. Request — HumanVideoRequest#
Base: the BFF's ProviderVideoRequest. It is the narrowest of the four, the
only one on the deployable path, and the only one already bounded by the curated
card validators.
Normative rules:
durationSeconds: numberis the only duration representation. No presets, noduration, no unit ambiguity. Lane limits are enforced against the live capability registry, not against a hard-coded union.modeis a required enumerated field carrying one of the seven modes in182.C.03.01. It is never inferred and never rewritten by the router.- Typed references, not one
referenceImage: string: each reference carries a role (identity, face-angle, full-body, wardrobe, scene), a digest, and its authorization record id. script(canonical text + SHA-256 digest + language tags + per-speaker turns + optional pronunciation lexicon) andspeechAudio(asset ref + digest) are separate fields. Exact-dialogue mode requires both.requiredCapabilitiesare distinct from preferences. A required capability that no healthy lane offers fails the request (human_video_unsatisfiable,182.C.03.07); a preference is advisory.qualityTiercarries the candidate and retry budget of182.C.03.06.
2. Provider capability — VideoLaneCapability#
Discovered, never hard-coded. Sourced from GET /api/v1/videos/models for
OpenRouter lanes and from the direct clients' own declarations, cached with a
revalidation cadence (182.C.01.06), and intersected with the request's
requiredCapabilities at admission.
The catalog carries no reference-count limit, retention, region, or
data-handling fields, so those are maintained as dated manual records
alongside the discovered record and are marked as such. A capability the catalog
advertises but no committed probe fixture covers is unprobed, and a request
depending on it fails with capability_unprobed rather than being attempted.
3. Job state — extend GenerationJobStatus, do not replace it#
The existing six-state enum (queued, running, complete, failed,
blocked, needs_review — jobs-route.ts:59-68) already carries the
release-gate decision and is already durable. Human-video stages are added
inside that envelope (182.C.05.10); the status enum itself does not grow a
seventh member. blocked and needs_review already express every gate outcome
this phase needs.
4. Measurement — one record, computed from released bytes#
Extends provider-measurement.ts. Every measurement carries the six conditions
of 182.C.03.05 with an explicit tri-state per condition — pass, fail, or
unknown — plus the evaluator identity and version that produced it.
unknown is never coerced to pass. A condition with no producer today (C1,
C4, C6 have thresholds but no evaluator until 182.C.17/.19/.20) reports
unknown, and the release gate treats that as blocking.
5. Release proof — extend ProofRecordSchema#
The nine named gates in release-proof.test-fixtures.ts:12-20 map onto the
fourteen PROOF_TYPES in libs/contracts/src/common/proof-types.ts. None
covers exact speech, audiovisual sync, speaker attribution, identity
preservation, or likeness consent (F-10). Those five gates are added to the
existing schema — consent is already a proof type, and ConsentProofSchema
(:181) and ObservedOutcomeProofSchema (:295) already carry the shapes
needed. No parallel proof system is created.
6. Where each type lives (182.C.04.02)#
Three homes, and the test for each is who must agree on it, not who wrote it first.
| Home | Rule | Human-video types that belong there |
|---|---|---|
@oshun/contracts (shared) |
Two or more deployables must agree on the wire shape, or it appears in a release proof | HumanVideoRequest, mode enum, typed reference roles, script/lexicon shape, qualityTier, the six-condition measurement record, the five new release-proof gates, human_video_unsatisfiable reasons |
Isis implementation libs (@isis/*) |
Provider-shaped or algorithm-shaped; no other deployable needs the type to compile | Lane adapters, passthrough option shapes per model, capability-discovery parsing, seedance direct-lane request/plan types, evaluator internals |
Oshun BFF (apps/oshun/bff) |
Boundary-only: request admission, credential resolution, job envelope, route DTOs | VideoProviderGenerate seam, env→provider resolution, per-tenant admission decisions, route-level validation results |
Three rules that follow:
- Provider-specific option shapes never enter
@oshun/contracts. A passthrough parameter list is a property of one model on one platform on one date; putting it in shared contracts freezes a mutable fact into every consumer's build. Shared contracts carry the capability vocabulary (accepts-supplied-audio,accepts-image-references), never the vendor's parameter names. - Anything that appears in a release proof is shared, without exception. A proof is read by the BFF, the worker, admin review, and any future auditor, so its shape is a four-party agreement by definition.
- Credentials and biometric payload types never leave the BFF. Not because
of layering aesthetics — because a type in shared contracts is importable by a
browser bundle, and
182.C.04.04forbids that reachability.
Consequences#
@isis/ai-providers/providers/video-generationis the provider SDK of record (182.C.04.03); the BFF already importsLTXProviderfrom it (video-provider-env.ts:37).@isis/seedance-providerstays as the direct lane for the reference, edit, and extend capabilities OpenRouter does not expose.@isis/ai-videois migrate-then-retire, and cannot retire untilvideo-workflow-manager.tsis lifted out (F-13): its queue,enqueueBatch,generateFromScenes, andWorkflowTemplateexecution model are the pieces worth keeping.WORKFLOW_TEMPLATESitself is left behind and re-authored against the182.Cstage order, since it encodes no evaluation, retry, or release step.libs/yemaya/video-generationis out of scope and its model ids are quarantined: it mapswan-2.6-t2v → wan-ai/wan2.6-t2v(model-registry.ts:45), an AIML-style id unrelated to OpenRouter'salibaba/wan-2.6, and@isis/ai-providers's Wan provider carries a third vocabulary (wan-provider.ts:49,alibaba/wan-2-6-t2v). All three must be reconciled against the live catalog before reuse.- The committed
.js/.d.tsbuild artifacts beside source invideo-generation/(F-03: 117.js, 118.d.tsacrosslibs/isis) are deleted as part of this migration, not ahead of it — a stale.jswinning module resolution is exactly why an unsequenced deletion is risky. - A second, incompatible
evaluateVoiceConsentexists inlibs/euterpe/provenance/src/rights-consent/rights-consent.ts:164(wall-clock ISO, territory-scoped) beside the BFF's logical-sequence evaluator.182.C.26must select one as the authorization of record; this ADR does not pre-empt that choice, but records that the video path may not depend on both.
Migration and Retirement Sequence (182.C.04.03)#
@isis/ai-providers/providers/video-generation is the SDK of record.
@isis/ai-video is retired — in this order, because the consumer inventory says
the cheap-looking step is the dangerous one.
Consumer inventory (verified 2026-09-09). @isis/ai-video has exactly one
consumer, libs/yemaya/remote-film-capture, and it imports types only
(isis-ai-video-integration.ts:1-10). No runtime code outside the library calls
it. But apps/oshun/web/tsconfig.json:159 maps the package to a shim
(remote-film-capture/src/isis-ai-video-shim.d.ts) whose types are widened with
| number and | string, while apps/oshun/web/vitest.config.ts:589 maps it
to the real source — F-15.
Sequence:
- Reconcile the shim first. Retiring the library while the shim stands would leave the web app type-checking against a declaration for a package that no longer exists, and the widened unions mean the failure would not surface as a type error. Narrow the shim to the real unions, or delete it and point the web tsconfig at the real source as vitest already does.
- Lift
video-workflow-manager.ts(F-13) into the Phase 182 workflow home: the queue,enqueueBatch,generateFromScenes, and theWorkflowTemplateexecution model. LeaveWORKFLOW_TEMPLATESbehind. - Re-point the type-only consumer at the canonical contracts from
@oshun/contracts, which is whereHumanVideoRequestand the mode vocabulary live per section 6. - Then delete the library, together with the F-03 build artifacts under
video-generation/.
Nothing in steps 2–4 may start before step 1 lands, and none of it may start
before the canonical contracts of sections 1–5 exist. Until then
@isis/ai-video stays in place — retired in decision, not yet in code.
Executed 2026-09-10 — two deviations, both recorded#
The sequence ran in order and @isis/ai-video is deleted. Two steps did not go
as written, and the reasons are part of the decision record:
Step 1 kept the shim rather than deleting it. Pointing the web tsconfig at
the real source would have pulled the library's runtime graph into an app
typecheck that the shim exists to keep small. Re-exporting the real declarations
from inside the ambient declare module block is illegal TypeScript (TS2439)
and, under skipLibCheck, silently resolves the whole surface to any — worse
than the widening it replaced, and measured before choosing. The shim was
therefore narrowed to the library's exact unions, and it was deleted at step 3
when the last import went away.
Step 3 did not re-point the consumer at @oshun/contracts/human-video.
Reading remote-film-capture says it should not: that contract is the canonical
surface for controlled human video — authorized likeness, exact dialogue,
consent context — and the consumer reports readiness for establishing shots,
background plates, and transitions, where no person is depicted from a
reference. The two vocabularies also disagree where they overlap
(ResolutionTier is 2160p; the plate fixtures are 4k), so adopting the
contract would have rewritten fixture data. The eight types moved into
isis-ai-video-types.ts beside the consumer under a Plate prefix, so F-12's
four disagreeing VideoGenerationRequest interfaces do not gain a fifth
unqualified one. Consequence for this ADR's section 6: "shared contracts"
means shared by deployables that must agree, and a single-consumer,
single-domain vocabulary is not made canonical by being adjacent to one.
Step 4 split, on evidence. advanced/ was not covered by the inventory this
sequence was written from. lens-physics.ts is real optics with no provider
coupling and was relocated to @isis/video-enhancement rather than deleted;
ensemble-hdr.ts fabricates VMAF, tone mapping, HDR validation, and EXR export
whenever its optional seams are unbound, so it was deleted with the provider
stack. Both are written up as F-16 in
docs/domains/isis/human-video/capability-audit.md.
Server-Boundary Rules (182.C.04.04)#
Four classes of material never reach a browser or mobile bundle. This is stated as an invariant with an enforcement point each, because "keep it on the server" without a check is a wish.
| Material | Rule | Enforcement |
|---|---|---|
| Provider credentials | Resolved only in BFF/worker process env; never in a NEXT_PUBLIC_* variable, never in a bundle |
video-provider-env.ts resolves from process.env at the deployable boundary and returns null without a key |
| Webhook receipt | callback_url points at a server route; the HMAC-SHA256 X-OpenRouter-Signature is verified server-side before the payload is trusted |
182.C.11 webhook route; a client may never receive or verify a provider callback |
| Provider downloads | unsigned_urls entries require Bearer auth and are fetched server-side; the client receives our own storage URL, never a provider URL with our token |
openrouter-video-provider.ts:344 download path |
| Biometric uploads | Reference images and voice audio are uploaded to our boundary, digested, and bound to an authorization record before any provider submission | 182.C.26.01–.02 |
Verified 2026-09-09 — the boundary currently holds. No NEXT_PUBLIC_*
variable names a provider key, and no .tsx in apps/oshun/web/src reads a
provider secret. The one client component that reads process.env at all,
config/runtime-context.tsx, is restricted to nine NEXT_PUBLIC_* keys (app
env, URLs, PWA/offline/monitoring/analytics flags, sample rate, feature flags,
version) — none of them a credential. This ADR records that state as the
invariant to preserve, not as a change being introduced.
Consequence for type placement. This is why section 6 keeps credential and
biometric types out of @oshun/contracts: a type in shared contracts is
importable by a browser bundle, and reachability is the thing being prevented.
Ports (182.C.04.05)#
Ten ports. Each is an injected interface with one implementation per backend,
and each fails loud when unconfigured rather than returning a degraded
result — the pattern dub-faithfulness.ts already establishes
(DubFaithfulnessNotConfiguredError, :132-133) and the one every port here
copies.
| # | Port | Responsibility | Existing seam to extend | Unconfigured behaviour |
|---|---|---|---|---|
| P1 | AssetIngestion |
Accept references and speech audio, digest, virus/format check, bind authorization | generated-artifact-object-store.ts |
Throw — no submission without ingestion |
| P2 | SpeechProduction |
Produce approved audio from the canonical script (deterministic TTS) or accept it | narration-executor.ts + narration-provider-env.ts |
Throw — exact mode has no fallback |
| P3 | ShotPlanning |
Brief + script → shot plan honouring lane duration bounds | generateFromScenes in video-workflow-manager.ts (F-13) |
Throw |
| P4 | CapabilityDiscovery |
Fetch/cache lane capabilities; mark unprobed ones | new; feeds provider-quality-router.ts |
Fail closed — treat all lanes as unavailable |
| P5 | Generation |
Submit a shot to a lane, poll, retrieve bytes | VideoProviderGenerate (video-executor.ts:43) |
provider_not_configured (already) |
| P6 | Evaluation |
Compute the six conditions from delivered bytes | video-probe.ts, audio-probe.ts, dub-faithfulness.ts |
Report unknown, which blocks release |
| P7 | Correction |
Bounded, disclosed corrective processing of a near-miss take | new | Throw — never silently correct |
| P8 | Encoding |
Renditions, captions, stems from the approved master | video-probe.ts for verification |
Throw |
| P9 | Provenance |
Watermark + C2PA manifest over the released bytes | video-governance.ts, audio-c2pa-signer.ts |
null signals, floors block release (already) |
| P10 | Release |
Gate decision + proof record | media-releasability.ts, release-gate.ts, ProofRecordSchema |
Block |
Four rules that apply to every port:
- A port is a capability boundary, not a layer. P5 has an OpenRouter
implementation, a direct fal/Ark implementation, and a Comfy/RunPod
implementation; all three satisfy the same interface without collapsing to a
lowest-common-denominator payload (see
182.C.04.07). - No port may return a value it did not compute. P6 in particular returns
unknownwhere an evaluator could not run — never a default, never a neutral score. This is the F-01 lesson written into the architecture. - P7 is disclosed by construction. Any corrective processing that touches the delivered frames or audio is recorded in the provenance manifest and in the creator-visible evidence. Undisclosed correction is indistinguishable from fabrication.
- Ports are injected at the deployable boundary, not resolved at import.
This keeps credentials in one place (
182.C.04.04) and keeps every port unit-testable without a network.
Durable Event Ownership and Compatibility (182.C.04.06)#
Human-video events extend the existing Isis event family in
libs/contracts/src/events/isis.ts (IsisJobQueued, …Started, …Progress,
…Completed, …Failed, …Cancelled, IsisAssetGenerated) on the shared
envelope in events/envelope.ts — EventSource already enumerates isis, and
the envelope already carries correlationId, causationId, and traceId. No
new event bus, no new envelope.
Ownership#
One producer per event type. An event is owned by the component that knows the fact first-hand; nobody else may emit it.
| Producer | Owns | May consume |
|---|---|---|
| Oshun BFF | Request admission, validation outcome, budget/quota decision, job envelope state | worker + evaluation events |
| Generation worker | Shot submitted, candidate produced, retry taken, provider fault | BFF admission events |
| Evaluation worker | Per-condition measurement, unknown findings, selection decision |
candidate events |
| Release | Gate decision, proof record written, disclosure applied | measurement events |
| Studio clients | nothing — clients are consumers only | all events scoped to their tenant |
| Veritas | nothing in this phase — V1.2-gated | nothing yet |
| Admin tools | Human review decisions, waivers with owner + expiry (182.C.01.07) |
all events |
Two consequences worth stating: a Studio client never emits job state (it requests, the BFF decides), and a human review decision is an event with an author, not a mutation of a job row — the audit trail is the point.
Compatibility rules#
- Additive only within a major. New optional fields; never a removed field, never a narrowed type, never a re-used name with a new meaning.
- Enum values are additive and consumers must tolerate unknown members. A new mode or a new failure reason must not crash a Studio client built last month. Consumers switch with a default branch; producers never assume a consumer understands a new value.
- The status enum does not grow. Human-video stages are carried as a payload field inside the existing six-state envelope, per section 3 — so a consumer that only understands the six states stays correct.
- Digests, not payloads. Events carry the script digest, reference digests,
and artifact digests — never the script text, the reference images, or the
media bytes. This keeps biometric material out of the event log entirely
(
182.C.04.04) and keeps events small enough to retain. - A measurement event is immutable and versioned by evaluator. Re-running an evaluator emits a new measurement carrying the new evaluator version; it never overwrites the old one. Comparing the two is how evaluator drift becomes visible instead of silent.
- Breaking changes get a new event type, side-by-side, with both emitted through one deprecation window announced in the tracker.
Backend Parity Without Lowest-Common-Denominator Payloads (182.C.04.07)#
Four backend families must implement the same ports: direct providers (fal,
PiAPI, Volcengine Ark via @isis/seedance-provider), OpenRouter, self-hosted
Comfy on RunPod (comfy-video-provider.ts, already implementing the same
VideoProviderClient seam as the LTX and OpenRouter adapters), and future
self-hosted models. They have genuinely different surfaces — OpenRouter exposes
no edit or extend at all; Comfy TI2V-5B caps at ~5 s single-shot and reaches
longer takes only by chaining segments; only two OpenRouter lanes accept
supplied audio.
The tempting resolution — define the request as the intersection of what all
four support — is rejected. It would delete exact-dialogue mode (only two lanes
have audio), delete edit and extend (direct only), and cap everything at the
weakest duration bound. Three mechanisms replace it:
1. A capability vocabulary, not a parameter union#
The canonical request declares what it needs, in our vocabulary:
accepts-supplied-audio, accepts-image-references, accepts-final-frame,
supports-extend, supports-edit, duration-range, resolution-set,
deterministic-seed. Each backend adapter declares which it satisfies. The
router intersects requirement × declared capability, never payload ×
payload. A vendor parameter name never appears in the shared contract (section
6).
2. Typed per-lane options, carried but not interpreted#
Provider-specific power stays reachable through a typed options object scoped to
one lane — OpenRouter's allowed_passthrough_parameters per model, Comfy's
workflow inputs, Ark's request fields. These are typed at the adapter,
validated against the discovered capability record, and never widened into the
shared request. The creator experience stays provider-neutral (182.C.03.01's
modes); the power user reaches the specifics through the adapter's own typed
surface. This is the existing curated-card discipline —
CARD_INPUT_FIELD_ALLOWLIST refuses providerKey, steps, and raw schedulers
at the form — applied one layer down.
3. Capability gaps fail, they do not degrade#
When a mode needs a capability the selected family lacks, the request either
routes to a family that has it or fails with the specific reason from
182.C.03.07 (no_edit_lane, no_audio_input_lane, duration_out_of_range).
It is never satisfied with a weaker payload. Concretely: a 20-second
exact-dialogue take is not delivered as a 10-second one, and it is not delivered
by chaining Comfy segments unless the shot plan explicitly authored it that way
and the seam is evaluated (182.C.22).
The rule in one line#
Backends differ in what they can do, never in what a result means. Every family returns bytes that pass through the same P6 evaluation and the same P10 release gate, so "released" is one standard regardless of which backend produced it — while the set of requests each family can accept stays honestly different.
Decision Drivers#
- One duration unit, one mode vocabulary, one measurement record — the three places the current fragmentation actually causes wrong behaviour.
- Extension over replacement: every BFF generation primitive audited at
182.C.02.11is real, fail-loud, and already durable. - No contract may be able to express a capability the system cannot measure.
Alternatives Considered#
Adopt @isis/ai-providers's request as the base. Rejected: it requires
model, which pushes model selection up into the caller and fights the router.
Adopt @yemaya/video-generation's request. Rejected: it hard-codes six
preset vocabularies and a scheduler, exactly the raw-parameter surface the
curated card allowlist deliberately refuses.
Define a fresh seventh contract owned by Phase 182. Rejected: this is how the repository reached six.
Related#
TODOS/phase-182.md—182.C.04,182.C.05docs/domains/isis/human-video/capability-audit.md— F-01 … F-14docs/domains/isis/human-video/product-contract.md—182.C.03docs/domains/isis/human-video/implementation-tracker.md