Status: repository-complete / closed. Video-producing seams consumed by V1 critique now expose a real sampled frame or produce a typed hold; V8 asset minting runs a fail-closed best-of-N tournament with exact media, quality, rights, and candidate-retention evidence; and canonical media batteries keep external benchmarks diagnostic-only. This closure does not claim a deployed evaluator, live human comparison, production provider result, or population superiority.
Work-package charter#
- Task ID:
MEDIA-001 - Owner: V1 media generation/critique, canonical evaluation contracts, and V8 Loom asset-production maintainers; implementation owner: Elikem Agudogo
- Accountable products: V1 BFF media generation and autonomy critique; V8 Yemaya case-assets and Loom asset realization/release
- Priority: P0/P1 multimodal quality and release-integrity gap
- Target milestone: 2026-07-15 SOTA gap-closure program
- Risk tier: high multimodal release-quality, candidate-substitution, rights, accessibility, and experiential-evidence risk
- Dependencies: canonical media-evaluation contracts; V8 case contracts,
case-assets, and Loom; ffmpeg/ffprobe video sampling; V1 creative critique and
revision; artifact-rights receipts from
PROOF-003 - Affected schemas and content slots:
MediaEvaluationBundle, image/video/ audio/3D/gameplay probe and human-comparison records, V8 asset selection and manifest records, V1 video provider output, and composed media-critic result. The affected content slots are generated V8 image, video, audio, and 3D assets plus critic-consumed V1 video; no rendered UI layout changed. - Linked task IDs:
MEDIA-001,EVAL-005,PROOF-003,V1-002,V8-001, andV8-005 - Audit finding: video critique could receive no visible frame; V8 asset minting had no production best-of-N tournament or exact experiential evidence
- Rollout strategy: additive evaluation records and candidate evidence, followed by fail-closed requirements at V1 critique and V8 production/release
- Rollback strategy: revert source commit
f8046ba3a573f082aa24e95573cd30de8a5c31d6; this restores the single-candidate V8 mint path and the previous untyped missing-frame failure - Evidence-pack location: this file
Closure matrix#
| Ledger requirement | Closure proof |
|---|---|
| Frame output on every critic-consumed video producer | The BFF VideoProviderGenerate contract now requires both a clip URL and a representative keyframe URL. The only producer adapters feeding the video executor probe the real produced clip with ffmpeg, extract the first sampled PNG, expose it as a data URL plus SHA-256, and forward it through both text-to-video and image-to-video executors. Empty clip or frame values fail loud. |
| Typed missing-frame hold | The composed media critic resolves its visible image before dispatching either VLM call. An absent video thumbnail, poster, or keyframe returns disposition: held with code media_required_frame_missing, required and observed evidence, and regeneration guidance. The revision loop stops after that pass, preserves the hold on the artifact, and does not fabricate a provider-usage receipt for a pre-dispatch hold. |
| V8 best-of-N and three-axis quality | The real @yemaya/case-assets Loom path derives deterministic candidate seeds, submits every configured variant, requires exact media evaluation, artifact-rights receipts, and perceptual/period/canon evidence, and selects the highest passing composite with deterministic seed/ID tie breaks. Production defaults to three variants, permits 2–16, requires an injected or HTTPS evaluation service, and fails configuration when none exists. |
| Preserve winner and losers | Every selection record retains requested variant count and every candidate's ID, job, seed, rendered prompt, provider, model, locator, MIME type, filename, artifact digest, status, three-axis scores, canonical evaluation, rights receipts, hold reasons, and selected flag. Provider failures and unevaluated candidates remain failed/held records; all attempted jobs consume budget. Manifests retain all-held tournaments as blocking evidence. |
| Artifact-specific media batteries and human comparison | The canonical contract defines and executes complete image, video, audio, 3D, and gameplay batteries. It requires exact artifact subjects and SHA-256, versioned non-diagnostic probes, evidence references, and blinded side-randomized human comparison. Missing probes, failed thresholds, absent/losing human studies, subject substitution, duplicate/unknown candidate evidence, or unreported model/provider identities hold release. External benchmarks are diagnostic-only and cannot satisfy a missing probe or human study. |
Video frame and typed-hold proof#
probeVideoContent samples real frames from the produced clip rather than
inventing a poster locator. Its result now contains:
- a
data:image/png;base64,...keyframe derived from the first sampled PNG; - SHA-256 over those exact PNG bytes;
- the existing duration, dimensions, luma, temporal, and safety measurements.
The environment-backed provider refuses a missing clip URL, runs this probe for every successful provider response, and returns the keyframe through the required producer contract. The governed executor independently rejects blank clip or keyframe URLs before exposing output to autonomy bindings.
The composed critic treats visible-frame availability as a prerequisite rather than an exception path. A hold names the missing evidence and directs the next producer to emit a real extracted thumbnail, poster, or keyframe. Standalone VLM critics retain their fail-loud behavior for direct callers; the composed autonomy boundary is the typed-hold surface. Revision stops immediately on a hold instead of spending additional generation attempts against unscorable output.
Canonical experiential battery#
MediaEvaluationBundle binds the following required probes to one exact
artifact subject and digest:
| Modality | Required dimensions |
|---|---|
| Image | composition; prompt/source alignment; identity; typography; aesthetics; cultural/brand correctness; artifact detection; accessibility |
| Video | temporal continuity; direction; edit rhythm; lip sync; story comprehension; flicker/artifacts; captioning; accessibility |
| Audio | intelligibility; word error rate where applicable; speaker/consent match; timing; loudness; silence/artifacts; musical structure; originality/fingerprint safety; accessibility |
| 3D | topology; UV; rigging; collision; LOD/Nanite; cook; performance; compatibility; novelty; accessibility |
| Gameplay | playability; performance; compatibility; novelty; accessibility; fun; player comprehension |
runMediaEvaluationBattery refuses to run when any required modality runner or
the human comparator is absent. It executes every required probe and the
comparator, parses the resulting strict schema, then evaluates coverage. Human
records require a distinct baseline, blinded and randomized presentation,
cohort/instruction/locale metadata, exact vote totals, evidence references, and
a subject-preference win. Diagnostic external benchmark records are retained but
never contribute required coverage.
V8 production tournament and release binding#
For every surviving mint gap, Loom reserves N jobs, submits deterministic
candidate seeds, and records failures instead of collapsing the tournament. The
evaluation response must name each real candidate. Local selection rejects
unknown, duplicate, missing, substituted-subject, wrong-modality, incomplete,
failed-threshold, or human-losing evidence. It separately requires passing,
identity-bearing perceptual, period, and canon dimensions, plus a complete
artifact-rights proof bundle bound to the exact candidate subject.
Only an eligible winner becomes a bound asset. The manifest validator proves
that the selected record is unique, eligible, evidence-bearing, and matches the
bound locator and seed. It calculates minted-job cost from all winner, loser,
failed, and all-held attempts. A tournament with no winner is retained under
heldSelections, omitted from releasable assets, and produces a blocking
manifest problem.
Production composition requires either an injected evaluator or an HTTPS
V8_MEDIA_EVALUATION_API_URL; the adapter applies bearer authentication when
configured, time and response-size limits, and a strict candidate envelope.
Before release, Loom fetches the authoritative selected locator, verifies its
real C2PA claim, hashes the bytes, and requires that digest to equal the
selected evaluation digest. Thus evaluation evidence for one candidate cannot
authorize substituted bytes at another locator.
Adversarial fixture matrix#
| Fixture | Result |
|---|---|
| Provider returns a clip with no URL | Producer fails with video_provider_missing_output_url; no fake frame or quality evidence is attached |
| Executor receives a blank keyframe | Executor fails with video_provider_missing_key_frame |
| Critic receives only a video URL | Typed media_required_frame_missing hold; no VLM call and no retry loop |
| One best-of-N provider job fails | Failed candidate remains in the selection evidence; eligible evaluated candidates may still compete |
| Candidate has no evaluation | Candidate is held and cannot win by position or generation order |
| Evaluation names another artifact | Candidate is held for exact-subject mismatch |
| Evaluation service returns duplicate or unknown IDs | Provider response holds every produced candidate |
| Required modality probe is absent | Coverage fails even when an external benchmark score is high |
| Human baseline wins or no human study exists | Coverage fails; machine scores cannot compensate |
| Selected evaluation digest differs from fetched bytes | Loom release verification fails before provenance upgrade |
| Every candidate is held | Full tournament remains in the manifest as a blocking hold; every provider attempt is still charged |
Automated verification#
Current admitted refresh#
The current refresh ran against repository commit
8354d0897b597796fc4898f6c2d462427d552158 with lockfile SHA-256
08b055786e9b8a6104d5313601d7eef3205dedcf395066fd9a8264c9859d690f on host
app-dev-server, Ubuntu 24.04.4 LTS, Linux 6.8.0-134-generic x86_64. The
relevant toolchain was Node.js v24.15.0, pnpm 10.25.0, TypeScript 6.0.3,
Vitest 4.1.7, and esbuild 0.28.0. Tests used deterministic local fixtures
and temporary files; the video suites invoked real local ffmpeg/ffprobe
processing but no remote media provider or production evaluation service.
| UTC interval | Exit | Command and result |
|---|---|---|
| 2026-07-29T12:26:02.663Z–2026-07-29T12:26:03.582Z | 0 | pnpm exec vitest run --config libs/contracts/vitest.config.ts libs/contracts/src/common/media-evaluation.test.ts --maxWorkers=1 — exact-subject media batteries, required probes, human comparisons, and diagnostic-only benchmark behavior passed 9/9 |
| 2026-07-29T12:26:18.673Z–2026-07-29T12:26:19.609Z | 0 | (cd libs/yemaya/case-contracts && pnpm exec vitest run --maxWorkers=1) — asset-selection and manifest contracts passed 18/18 |
| 2026-07-29T12:26:24.462Z–2026-07-29T12:26:35.730Z | 0 | (cd libs/yemaya/case-assets && pnpm exec vitest run --maxWorkers=1) — complete V8 asset binding, budget, orchestration, best-of-N, exact media/rights evidence, and realization suite passed 65/65 across ten files |
| 2026-07-29T12:26:43.226Z–2026-07-29T12:26:46.880Z | 0 | pnpm exec vitest run --config libs/oshun/creative-orchestrator/vitest.config.ts --maxWorkers=1 — complete critique/revision and orchestration package passed 138/138 across 13 files |
| 2026-07-29T12:27:02.161Z–2026-07-29T12:27:20.233Z | 0 | (cd apps/v8/loom-service && pnpm exec vitest run --maxWorkers=1) — Loom evaluation, provenance, dependency, pipeline, and service suites passed 41, with 3 database-dependent cases explicitly skipped, across ten files |
| 2026-07-29T12:27:28.914Z–2026-07-29T12:28:01.402Z | 0 | pnpm exec vitest run --config apps/oshun/bff/vitest.config.ts apps/oshun/bff/src/agentic/autonomy-bindings/autonomy-bindings.test.ts apps/oshun/bff/src/agentic/autonomy-bindings/media-critic.test.ts apps/oshun/bff/src/generation/video-executor.test.ts apps/oshun/bff/src/generation/video-probe.test.ts apps/oshun/bff/src/generation/video-provider-env.test.ts --maxWorkers=1 — V1 media production, real-frame probes, typed holds, and autonomy integration passed 115/115 |
| 2026-07-29T12:28:13.474Z–2026-07-29T12:28:28.392Z | 0 | pnpm exec tsc -p libs/contracts/tsconfig.lib.json --noEmit |
| 2026-07-29T12:28:35.670Z–2026-07-29T12:28:51.723Z | 0 | pnpm exec tsc -p libs/yemaya/case-contracts/tsconfig.lib.json --noEmit followed by pnpm exec tsc -p libs/yemaya/case-contracts/tsconfig.spec.json --noEmit |
| 2026-07-29T12:28:58.050Z–2026-07-29T12:29:13.314Z | 0 | pnpm exec tsc -p libs/yemaya/case-assets/tsconfig.lib.json --noEmit followed by pnpm exec tsc -p libs/yemaya/case-assets/tsconfig.spec.json --noEmit |
| 2026-07-29T12:29:19.814Z–2026-07-29T12:29:22.579Z | 0 | pnpm exec tsc -p libs/oshun/creative-orchestrator/tsconfig.json --noEmit |
| 2026-07-29T12:29:27.807Z–2026-07-29T12:29:44.699Z | 0 | pnpm exec tsc -p apps/v8/loom-service/tsconfig.json --noEmit |
| 2026-07-29T12:30:01.738Z–2026-07-29T12:30:26.845Z | 0 | Exact canonical-media, V8 contract/asset/Loom, V1 video/critic, and critique/revision lint command shown below — 24 relevant files passed with zero warnings |
The exact lint invocation was:
pnpm exec eslint libs/contracts/src/common/media-evaluation.ts libs/contracts/src/common/media-evaluation.test.ts libs/yemaya/case-contracts/src/asset-manifest.ts libs/yemaya/case-contracts/src/contracts.test.ts libs/yemaya/case-assets/src/quality-selection.ts libs/yemaya/case-assets/src/quality-selection.test.ts libs/yemaya/case-assets/src/orchestrate.ts libs/yemaya/case-assets/src/orchestrate.test.ts libs/yemaya/case-assets/src/realize.ts libs/yemaya/case-assets/src/realize-best-of-n.test.ts apps/v8/loom-service/src/asset-evaluation-provider.ts apps/v8/loom-service/src/asset-evaluation-provider.test.ts apps/v8/loom-service/src/asset-provenance-verifier.ts apps/v8/loom-service/src/asset-provenance-verifier.test.ts apps/oshun/bff/src/generation/video-probe.ts apps/oshun/bff/src/generation/video-probe.test.ts apps/oshun/bff/src/generation/video-provider-env.ts apps/oshun/bff/src/generation/video-provider-env.test.ts apps/oshun/bff/src/generation/video-executor.ts apps/oshun/bff/src/generation/video-executor.test.ts apps/oshun/bff/src/agentic/autonomy-bindings/media-critic.ts apps/oshun/bff/src/agentic/autonomy-bindings/media-critic.test.ts libs/oshun/creative-orchestrator/src/critique-revise.ts libs/oshun/creative-orchestrator/src/critique-revise.test.ts --no-ignore --max-warnings=0
The current value suites therefore passed all 386 executed assertions across
40 files, with three database-dependent Loom cases explicitly skipped. Seven
owning source/spec TypeScript configurations and exact lint over 24 relevant
files passed. The most recent published BFF app-owned ratchet, on source commit
24ae90ca8d896c5e1829c19f654c6fabb02979b0, reported zero new errors and zero
known backlog; the intervening commit through this refresh source changes only
AUTH-002 evidence, not BFF source or configuration.
The one current code drift in the six primary MEDIA-001 artifacts strengthens the boundary: V8 candidate selection now also parses and evaluates an exact artifact-rights proof bundle and retains it with every candidate record. The current 65-test case-assets suite covers that integration. The updated hash is recorded below rather than silently retaining the source-publication identity.
Historical implementation verification#
The integrated source-publication record passed 4,351/4,351 non-duplicated package and focused tests:
| Surface | Result |
|---|---|
| Canonical contracts full package, including all migration/rollback cases | 4,002/4,002 |
| V8 case-contracts full package | 17/17 |
| V8 case-assets full package | 64/64 |
| Creative orchestrator full package | 132/132 |
| V8 Loom service full package | 28/28 |
| V1 BFF affected media/autonomy paths | 108/108 |
Post-lint regression reruns passed 25/25 case-assets tests, 36/36 BFF tests, 27/27 real video-probe tests, and 11/11 critique/revise tests. After integrating the incoming V10 commits, an additional 9/9 canonical media tests and 5/5 V8 selection/realization tests passed.
Typechecks passed for contracts, case-contracts library/specs, case-assets
library/specs, creative-orchestrator, and Loom. The resource-supervised BFF
ratchet reported zero new and zero known app-owned errors. Explicit ESLint over
all changed TypeScript files passed with zero warnings; Prettier and
git diff --check passed. A direct production BFF esbuild bundle completed
successfully (dist/server.js, 30.0 MB; source map, 71.9 MB).
Representative commands:
pnpm --dir <affected-package> exec vitest run
pnpm exec tsc --noEmit -p <affected-tsconfig>
node tools/typecheck/ratchet.mjs --app apps/oshun/bff --tsconfig tsconfig.typecheck.json
pnpm exec eslint <all changed TypeScript files> --no-ignore --max-warnings=0
pnpm exec prettier --check <all changed non-lock files>
pnpm --dir apps/oshun/bff exec esbuild src/server.ts --bundle --platform=node --format=esm --target=node22 --outfile=dist/server.js --sourcemap
git diff --check
Baselines and bounded verification#
Current refresh notes:
- Two initial Vitest invocations passed package-root-relative include patterns from the repository root and therefore found no files: case-contracts at 2026-07-29T12:26:09.333Z–12:26:09.963Z and Loom at 2026-07-29T12:26:53.383Z–12:26:54.007Z. Both exited 1. The admitted commands run from their owning package roots and passed 18/18 and 41 passing with three explicit skips. These were command-location errors, not product failures.
- Loom emitted existing missing-source-map warnings for generated JavaScript in
libs/shared/database; all discovered tests completed and the process exited 0. - The host reported 15 GiB RAM, 31 GiB swap, 12 GiB initially available memory, and 16 GiB free disk. An unrelated checkout's long-running Vitest worker and this checkout's Nx daemon were active. Current tests and typechecks ran serially with one Vitest worker; no full-workspace or broad Nx matrix was started.
Historical source-publication notes:
- The first full contracts run passed 3,994 tests and exposed eight Prisma migration-diff failures because a filtered pnpm operation had pruned the ignored root Prisma/dotenv resolution links. Both packages remained installed offline. Restoring those local-only links made the exact Prisma command pass; rerunning all nine migration/rollback cases passed, completing 4,002/4,002.
- One initial case-assets invocation used workspace-relative paths against a package-root Vitest include and discovered no tests. The corrected package-root invocation passed the four focused files and then the full 64-test package.
- The first BFF ratchet found one new excess-property error in an old aggregate fixture after the aggregate input was intentionally narrowed. The obsolete fields were removed; its 27 tests and the rerun ratchet then passed.
- Loom's passing run emitted existing missing-source-map warnings for generated
JavaScript under
libs/shared/http-client; all five files and 28 tests passed. - The source commit used the pre-commit hook's documented
SKIP_TYPECHECK=1after all owning package typechecks and the BFF ratchet had passed. The hook incorrectly expanded common-contract ownership to 145 projects at--parallel=4; that unsafe broad run was interrupted under the small-host policy. ESLint, formatting, V2 asset locking, stub scanning, and V5 documentation validation still passed in the successful commit. - The host had 15 GiB RAM and 31 GiB swap. Tests and package checks ran sequentially. The BFF ratchet peaked near 4.9 GiB RSS while at least 5.9 GiB remained available, above the stop threshold.
- The filtered offline lock update produced existing peer/deprecation warnings. Only the two required workspace importer links were committed. The unrelated user lock blob was preserved byte-for-byte as an unstaged modification across the V10 rebase.
- No customer UI, PWA, admin, mobile, Unreal, or other visual surface changed. Frontend skill, Playwright, mobile automation, screenshots, and Unreal verification were inapplicable.
Artifact identity#
| Artifact | SHA-256 |
|---|---|
| Canonical media evaluation | b2c623195d73cb50c20f1904b81865e7b2b16073a7aa22974946bee2a8f1f800 |
| V8 quality selection | 606addc864823c4de625c2ef1bf74992671f609d5f06effead975a756baddeb4 |
| V8 seeded orchestration | 9a4f7f47b6199f04ad1c815c3a0776c593a3083287999f91fa345c882b4fb1fa |
| Loom evaluation service adapter | 13e9a20b23ba4adadce18b48749a236f3baa5f2091638e09d3b171637cbaf7f6 |
| Real video frame probe | b4784c4ab41892062842bd431d755e44fa4f6b0375f0b92d48788fecb1e34c31 |
| Typed media critic hold | 38b6d5226b8a8c680540daad60915cdb2f937f3196d02cce3db76f67285e0af9 |
Scope and remaining seams#
- Repository publication of the contract and evidence pack requires no fabricated human signoff. A passing production human-comparison record does require real blinded, randomized participant results with cohort, instruction, locale, vote, and evidence metadata. Test fixtures exercise the schema and fail-closed policy but are not approvals or live study results.
- Production image/video/audio/3D/gameplay probe implementations and human study operations remain deployment services behind the canonical executable runner/HTTPS adapter. Their absence is now explicit and blocks production V8 composition or candidate selection; no live study result is claimed.
- V8 currently mints image, video, audio, and 3D assets. Gameplay evaluation is canonical and executable for gameplay artifacts but is not falsely attached to a non-gameplay V8 asset.
- EVAL-005's separate education battery is outside MEDIA-001. Its repository contract is complete; production education study evidence remains an external-state requirement recorded by that pack.
- External benchmark ingestion remains diagnostic by contract. This package does not claim live population superiority, a production evaluator deployment, or completed human studies.
Publication identity#
- Source commit:
f8046ba3a573f082aa24e95573cd30de8a5c31d6 - Historical declared closure evidence commit:
573cce758a2d33cb5caaee8c0e824da09a0eb966 - Equivalent rebased closure identity on the current origin/main graph:
089e875030dbace657765680ce67a0d506ca5865(same tree221cd068d5af8608d086abaa5ce0deb97eef90b6) - Current admitted verification source:
8354d0897b597796fc4898f6c2d462427d552158 - Current evidence-refresh commit: recorded by the task-scoped commit that contains this file.