Eve library

ADR-0006: Canonical Human-Video Request, Capability, Job-State, Measurement, and Release-Proof Contracts

ADR

Revision 1

On this page

Status: Accepted
Date: 2026-09-09
Authors: Isis Platform, Oshun BFF, Contracts
Reviewers: @oshun/core-team, @oshun/isis-team, @oshun/platform, @oshun/trust-safety

Context and Problem Statement#

Phase 182 adds controlled human video — authorized likeness, exact dialogue, directed performance, quality-gated delivery. The repository already contains six video provider abstractions and, worse, four live interfaces named VideoGenerationRequest that disagree about what "duration" means (F-12 in docs/domains/isis/human-video/capability-audit.md):

Contract Duration field Model field
@isis/ai-video video.types.ts duration: DurationPreset (4|5|10|16) none
@isis/ai-providers video-generation/types.ts:276 duration?: number (seconds) model required
@yemaya/video-generation types.ts:324 durationSeconds: number modelId | null
BFF video-executor.ts:25 ProviderVideoRequest durationSeconds: number none (bound at env)

Nothing converts between them. Every new caller re-derives the mapping by hand, and @isis/ai-video's DurationPreset admits 16 — a value no curated card can express (VIDEO_DURATIONS_SECONDS is [4, 5, 10]) and the default lane, alibaba/wan-2.7, cannot render (2–10 s).

Adding a seventh abstraction for human video would make this permanent. So would extending one of the four in place.

Decision#

One canonical contract per concern, defined once, adapted at every edge. Five contracts, with a named owner and a named home for each.

1. Request — HumanVideoRequest#

Base: the BFF's ProviderVideoRequest. It is the narrowest of the four, the only one on the deployable path, and the only one already bounded by the curated card validators.

Normative rules:

  • durationSeconds: number is the only duration representation. No presets, no duration, no unit ambiguity. Lane limits are enforced against the live capability registry, not against a hard-coded union.
  • mode is a required enumerated field carrying one of the seven modes in 182.C.03.01. It is never inferred and never rewritten by the router.
  • Typed references, not one referenceImage: string: each reference carries a role (identity, face-angle, full-body, wardrobe, scene), a digest, and its authorization record id.
  • script (canonical text + SHA-256 digest + language tags + per-speaker turns + optional pronunciation lexicon) and speechAudio (asset ref + digest) are separate fields. Exact-dialogue mode requires both.
  • requiredCapabilities are distinct from preferences. A required capability that no healthy lane offers fails the request (human_video_unsatisfiable, 182.C.03.07); a preference is advisory.
  • qualityTier carries the candidate and retry budget of 182.C.03.06.

2. Provider capability — VideoLaneCapability#

Discovered, never hard-coded. Sourced from GET /api/v1/videos/models for OpenRouter lanes and from the direct clients' own declarations, cached with a revalidation cadence (182.C.01.06), and intersected with the request's requiredCapabilities at admission.

The catalog carries no reference-count limit, retention, region, or data-handling fields, so those are maintained as dated manual records alongside the discovered record and are marked as such. A capability the catalog advertises but no committed probe fixture covers is unprobed, and a request depending on it fails with capability_unprobed rather than being attempted.

3. Job state — extend GenerationJobStatus, do not replace it#

The existing six-state enum (queued, running, complete, failed, blocked, needs_review — jobs-route.ts:59-68) already carries the release-gate decision and is already durable. Human-video stages are added inside that envelope (182.C.05.10); the status enum itself does not grow a seventh member. blocked and needs_review already express every gate outcome this phase needs.

4. Measurement — one record, computed from released bytes#

Extends provider-measurement.ts. Every measurement carries the six conditions of 182.C.03.05 with an explicit tri-state per condition — pass, fail, or unknown — plus the evaluator identity and version that produced it.

unknown is never coerced to pass. A condition with no producer today (C1, C4, C6 have thresholds but no evaluator until 182.C.17/.19/.20) reports unknown, and the release gate treats that as blocking.

5. Release proof — extend ProofRecordSchema#

The nine named gates in release-proof.test-fixtures.ts:12-20 map onto the fourteen PROOF_TYPES in libs/contracts/src/common/proof-types.ts. None covers exact speech, audiovisual sync, speaker attribution, identity preservation, or likeness consent (F-10). Those five gates are added to the existing schema — consent is already a proof type, and ConsentProofSchema (:181) and ObservedOutcomeProofSchema (:295) already carry the shapes needed. No parallel proof system is created.

6. Where each type lives (182.C.04.02)#

Three homes, and the test for each is who must agree on it, not who wrote it first.

Home Rule Human-video types that belong there
@oshun/contracts (shared) Two or more deployables must agree on the wire shape, or it appears in a release proof HumanVideoRequest, mode enum, typed reference roles, script/lexicon shape, qualityTier, the six-condition measurement record, the five new release-proof gates, human_video_unsatisfiable reasons
Isis implementation libs (@isis/*) Provider-shaped or algorithm-shaped; no other deployable needs the type to compile Lane adapters, passthrough option shapes per model, capability-discovery parsing, seedance direct-lane request/plan types, evaluator internals
Oshun BFF (apps/oshun/bff) Boundary-only: request admission, credential resolution, job envelope, route DTOs VideoProviderGenerate seam, env→provider resolution, per-tenant admission decisions, route-level validation results

Three rules that follow:

  • Provider-specific option shapes never enter @oshun/contracts. A passthrough parameter list is a property of one model on one platform on one date; putting it in shared contracts freezes a mutable fact into every consumer's build. Shared contracts carry the capability vocabulary (accepts-supplied-audio, accepts-image-references), never the vendor's parameter names.
  • Anything that appears in a release proof is shared, without exception. A proof is read by the BFF, the worker, admin review, and any future auditor, so its shape is a four-party agreement by definition.
  • Credentials and biometric payload types never leave the BFF. Not because of layering aesthetics — because a type in shared contracts is importable by a browser bundle, and 182.C.04.04 forbids that reachability.

Consequences#

  • @isis/ai-providers/providers/video-generation is the provider SDK of record (182.C.04.03); the BFF already imports LTXProvider from it (video-provider-env.ts:37).
  • @isis/seedance-provider stays as the direct lane for the reference, edit, and extend capabilities OpenRouter does not expose.
  • @isis/ai-video is migrate-then-retire, and cannot retire until video-workflow-manager.ts is lifted out (F-13): its queue, enqueueBatch, generateFromScenes, and WorkflowTemplate execution model are the pieces worth keeping. WORKFLOW_TEMPLATES itself is left behind and re-authored against the 182.C stage order, since it encodes no evaluation, retry, or release step.
  • libs/yemaya/video-generation is out of scope and its model ids are quarantined: it maps wan-2.6-t2v → wan-ai/wan2.6-t2v (model-registry.ts:45), an AIML-style id unrelated to OpenRouter's alibaba/wan-2.6, and @isis/ai-providers's Wan provider carries a third vocabulary (wan-provider.ts:49, alibaba/wan-2-6-t2v). All three must be reconciled against the live catalog before reuse.
  • The committed .js/.d.ts build artifacts beside source in video-generation/ (F-03: 117 .js, 118 .d.ts across libs/isis) are deleted as part of this migration, not ahead of it — a stale .js winning module resolution is exactly why an unsequenced deletion is risky.
  • A second, incompatible evaluateVoiceConsent exists in libs/euterpe/provenance/src/rights-consent/rights-consent.ts:164 (wall-clock ISO, territory-scoped) beside the BFF's logical-sequence evaluator. 182.C.26 must select one as the authorization of record; this ADR does not pre-empt that choice, but records that the video path may not depend on both.

Migration and Retirement Sequence (182.C.04.03)#

@isis/ai-providers/providers/video-generation is the SDK of record. @isis/ai-video is retired — in this order, because the consumer inventory says the cheap-looking step is the dangerous one.

Consumer inventory (verified 2026-09-09). @isis/ai-video has exactly one consumer, libs/yemaya/remote-film-capture, and it imports types only (isis-ai-video-integration.ts:1-10). No runtime code outside the library calls it. But apps/oshun/web/tsconfig.json:159 maps the package to a shim (remote-film-capture/src/isis-ai-video-shim.d.ts) whose types are widened with | number and | string, while apps/oshun/web/vitest.config.ts:589 maps it to the real source — F-15.

Sequence:

  1. Reconcile the shim first. Retiring the library while the shim stands would leave the web app type-checking against a declaration for a package that no longer exists, and the widened unions mean the failure would not surface as a type error. Narrow the shim to the real unions, or delete it and point the web tsconfig at the real source as vitest already does.
  2. Lift video-workflow-manager.ts (F-13) into the Phase 182 workflow home: the queue, enqueueBatch, generateFromScenes, and the WorkflowTemplate execution model. Leave WORKFLOW_TEMPLATES behind.
  3. Re-point the type-only consumer at the canonical contracts from @oshun/contracts, which is where HumanVideoRequest and the mode vocabulary live per section 6.
  4. Then delete the library, together with the F-03 build artifacts under video-generation/.

Nothing in steps 2–4 may start before step 1 lands, and none of it may start before the canonical contracts of sections 1–5 exist. Until then @isis/ai-video stays in place — retired in decision, not yet in code.

Executed 2026-09-10 — two deviations, both recorded#

The sequence ran in order and @isis/ai-video is deleted. Two steps did not go as written, and the reasons are part of the decision record:

Step 1 kept the shim rather than deleting it. Pointing the web tsconfig at the real source would have pulled the library's runtime graph into an app typecheck that the shim exists to keep small. Re-exporting the real declarations from inside the ambient declare module block is illegal TypeScript (TS2439) and, under skipLibCheck, silently resolves the whole surface to any — worse than the widening it replaced, and measured before choosing. The shim was therefore narrowed to the library's exact unions, and it was deleted at step 3 when the last import went away.

Step 3 did not re-point the consumer at @oshun/contracts/human-video. Reading remote-film-capture says it should not: that contract is the canonical surface for controlled human video — authorized likeness, exact dialogue, consent context — and the consumer reports readiness for establishing shots, background plates, and transitions, where no person is depicted from a reference. The two vocabularies also disagree where they overlap (ResolutionTier is 2160p; the plate fixtures are 4k), so adopting the contract would have rewritten fixture data. The eight types moved into isis-ai-video-types.ts beside the consumer under a Plate prefix, so F-12's four disagreeing VideoGenerationRequest interfaces do not gain a fifth unqualified one. Consequence for this ADR's section 6: "shared contracts" means shared by deployables that must agree, and a single-consumer, single-domain vocabulary is not made canonical by being adjacent to one.

Step 4 split, on evidence. advanced/ was not covered by the inventory this sequence was written from. lens-physics.ts is real optics with no provider coupling and was relocated to @isis/video-enhancement rather than deleted; ensemble-hdr.ts fabricates VMAF, tone mapping, HDR validation, and EXR export whenever its optional seams are unbound, so it was deleted with the provider stack. Both are written up as F-16 in docs/domains/isis/human-video/capability-audit.md.

Server-Boundary Rules (182.C.04.04)#

Four classes of material never reach a browser or mobile bundle. This is stated as an invariant with an enforcement point each, because "keep it on the server" without a check is a wish.

Material Rule Enforcement
Provider credentials Resolved only in BFF/worker process env; never in a NEXT_PUBLIC_* variable, never in a bundle video-provider-env.ts resolves from process.env at the deployable boundary and returns null without a key
Webhook receipt callback_url points at a server route; the HMAC-SHA256 X-OpenRouter-Signature is verified server-side before the payload is trusted 182.C.11 webhook route; a client may never receive or verify a provider callback
Provider downloads unsigned_urls entries require Bearer auth and are fetched server-side; the client receives our own storage URL, never a provider URL with our token openrouter-video-provider.ts:344 download path
Biometric uploads Reference images and voice audio are uploaded to our boundary, digested, and bound to an authorization record before any provider submission 182.C.26.01–.02

Verified 2026-09-09 — the boundary currently holds. No NEXT_PUBLIC_* variable names a provider key, and no .tsx in apps/oshun/web/src reads a provider secret. The one client component that reads process.env at all, config/runtime-context.tsx, is restricted to nine NEXT_PUBLIC_* keys (app env, URLs, PWA/offline/monitoring/analytics flags, sample rate, feature flags, version) — none of them a credential. This ADR records that state as the invariant to preserve, not as a change being introduced.

Consequence for type placement. This is why section 6 keeps credential and biometric types out of @oshun/contracts: a type in shared contracts is importable by a browser bundle, and reachability is the thing being prevented.

Ports (182.C.04.05)#

Ten ports. Each is an injected interface with one implementation per backend, and each fails loud when unconfigured rather than returning a degraded result — the pattern dub-faithfulness.ts already establishes (DubFaithfulnessNotConfiguredError, :132-133) and the one every port here copies.

# Port Responsibility Existing seam to extend Unconfigured behaviour
P1 AssetIngestion Accept references and speech audio, digest, virus/format check, bind authorization generated-artifact-object-store.ts Throw — no submission without ingestion
P2 SpeechProduction Produce approved audio from the canonical script (deterministic TTS) or accept it narration-executor.ts + narration-provider-env.ts Throw — exact mode has no fallback
P3 ShotPlanning Brief + script → shot plan honouring lane duration bounds generateFromScenes in video-workflow-manager.ts (F-13) Throw
P4 CapabilityDiscovery Fetch/cache lane capabilities; mark unprobed ones new; feeds provider-quality-router.ts Fail closed — treat all lanes as unavailable
P5 Generation Submit a shot to a lane, poll, retrieve bytes VideoProviderGenerate (video-executor.ts:43) provider_not_configured (already)
P6 Evaluation Compute the six conditions from delivered bytes video-probe.ts, audio-probe.ts, dub-faithfulness.ts Report unknown, which blocks release
P7 Correction Bounded, disclosed corrective processing of a near-miss take new Throw — never silently correct
P8 Encoding Renditions, captions, stems from the approved master video-probe.ts for verification Throw
P9 Provenance Watermark + C2PA manifest over the released bytes video-governance.ts, audio-c2pa-signer.ts null signals, floors block release (already)
P10 Release Gate decision + proof record media-releasability.ts, release-gate.ts, ProofRecordSchema Block

Four rules that apply to every port:

  • A port is a capability boundary, not a layer. P5 has an OpenRouter implementation, a direct fal/Ark implementation, and a Comfy/RunPod implementation; all three satisfy the same interface without collapsing to a lowest-common-denominator payload (see 182.C.04.07).
  • No port may return a value it did not compute. P6 in particular returns unknown where an evaluator could not run — never a default, never a neutral score. This is the F-01 lesson written into the architecture.
  • P7 is disclosed by construction. Any corrective processing that touches the delivered frames or audio is recorded in the provenance manifest and in the creator-visible evidence. Undisclosed correction is indistinguishable from fabrication.
  • Ports are injected at the deployable boundary, not resolved at import. This keeps credentials in one place (182.C.04.04) and keeps every port unit-testable without a network.

Durable Event Ownership and Compatibility (182.C.04.06)#

Human-video events extend the existing Isis event family in libs/contracts/src/events/isis.ts (IsisJobQueued, …Started, …Progress, …Completed, …Failed, …Cancelled, IsisAssetGenerated) on the shared envelope in events/envelope.ts — EventSource already enumerates isis, and the envelope already carries correlationId, causationId, and traceId. No new event bus, no new envelope.

Ownership#

One producer per event type. An event is owned by the component that knows the fact first-hand; nobody else may emit it.

Producer Owns May consume
Oshun BFF Request admission, validation outcome, budget/quota decision, job envelope state worker + evaluation events
Generation worker Shot submitted, candidate produced, retry taken, provider fault BFF admission events
Evaluation worker Per-condition measurement, unknown findings, selection decision candidate events
Release Gate decision, proof record written, disclosure applied measurement events
Studio clients nothing — clients are consumers only all events scoped to their tenant
Veritas nothing in this phase — V1.2-gated nothing yet
Admin tools Human review decisions, waivers with owner + expiry (182.C.01.07) all events

Two consequences worth stating: a Studio client never emits job state (it requests, the BFF decides), and a human review decision is an event with an author, not a mutation of a job row — the audit trail is the point.

Compatibility rules#

  1. Additive only within a major. New optional fields; never a removed field, never a narrowed type, never a re-used name with a new meaning.
  2. Enum values are additive and consumers must tolerate unknown members. A new mode or a new failure reason must not crash a Studio client built last month. Consumers switch with a default branch; producers never assume a consumer understands a new value.
  3. The status enum does not grow. Human-video stages are carried as a payload field inside the existing six-state envelope, per section 3 — so a consumer that only understands the six states stays correct.
  4. Digests, not payloads. Events carry the script digest, reference digests, and artifact digests — never the script text, the reference images, or the media bytes. This keeps biometric material out of the event log entirely (182.C.04.04) and keeps events small enough to retain.
  5. A measurement event is immutable and versioned by evaluator. Re-running an evaluator emits a new measurement carrying the new evaluator version; it never overwrites the old one. Comparing the two is how evaluator drift becomes visible instead of silent.
  6. Breaking changes get a new event type, side-by-side, with both emitted through one deprecation window announced in the tracker.

Backend Parity Without Lowest-Common-Denominator Payloads (182.C.04.07)#

Four backend families must implement the same ports: direct providers (fal, PiAPI, Volcengine Ark via @isis/seedance-provider), OpenRouter, self-hosted Comfy on RunPod (comfy-video-provider.ts, already implementing the same VideoProviderClient seam as the LTX and OpenRouter adapters), and future self-hosted models. They have genuinely different surfaces — OpenRouter exposes no edit or extend at all; Comfy TI2V-5B caps at ~5 s single-shot and reaches longer takes only by chaining segments; only two OpenRouter lanes accept supplied audio.

The tempting resolution — define the request as the intersection of what all four support — is rejected. It would delete exact-dialogue mode (only two lanes have audio), delete edit and extend (direct only), and cap everything at the weakest duration bound. Three mechanisms replace it:

1. A capability vocabulary, not a parameter union#

The canonical request declares what it needs, in our vocabulary: accepts-supplied-audio, accepts-image-references, accepts-final-frame, supports-extend, supports-edit, duration-range, resolution-set, deterministic-seed. Each backend adapter declares which it satisfies. The router intersects requirement × declared capability, never payload × payload. A vendor parameter name never appears in the shared contract (section 6).

2. Typed per-lane options, carried but not interpreted#

Provider-specific power stays reachable through a typed options object scoped to one lane — OpenRouter's allowed_passthrough_parameters per model, Comfy's workflow inputs, Ark's request fields. These are typed at the adapter, validated against the discovered capability record, and never widened into the shared request. The creator experience stays provider-neutral (182.C.03.01's modes); the power user reaches the specifics through the adapter's own typed surface. This is the existing curated-card discipline — CARD_INPUT_FIELD_ALLOWLIST refuses providerKey, steps, and raw schedulers at the form — applied one layer down.

3. Capability gaps fail, they do not degrade#

When a mode needs a capability the selected family lacks, the request either routes to a family that has it or fails with the specific reason from 182.C.03.07 (no_edit_lane, no_audio_input_lane, duration_out_of_range). It is never satisfied with a weaker payload. Concretely: a 20-second exact-dialogue take is not delivered as a 10-second one, and it is not delivered by chaining Comfy segments unless the shot plan explicitly authored it that way and the seam is evaluated (182.C.22).

The rule in one line#

Backends differ in what they can do, never in what a result means. Every family returns bytes that pass through the same P6 evaluation and the same P10 release gate, so "released" is one standard regardless of which backend produced it — while the set of requests each family can accept stays honestly different.

Decision Drivers#

  • One duration unit, one mode vocabulary, one measurement record — the three places the current fragmentation actually causes wrong behaviour.
  • Extension over replacement: every BFF generation primitive audited at 182.C.02.11 is real, fail-loud, and already durable.
  • No contract may be able to express a capability the system cannot measure.

Alternatives Considered#

Adopt @isis/ai-providers's request as the base. Rejected: it requires model, which pushes model selection up into the caller and fights the router.

Adopt @yemaya/video-generation's request. Rejected: it hard-codes six preset vocabularies and a scheduler, exactly the raw-parameter surface the curated card allowlist deliberately refuses.

Define a fresh seventh contract owned by Phase 182. Rejected: this is how the repository reached six.

  • TODOS/phase-182.md — 182.C.04, 182.C.05
  • docs/domains/isis/human-video/capability-audit.md — F-01 … F-14
  • docs/domains/isis/human-video/product-contract.md — 182.C.03
  • docs/domains/isis/human-video/implementation-tracker.md