# EVAL-005 evidence pack — domain-appropriate experiential batteries

> **Status: repository-complete / closed.** Canonical, executable batteries now
> cover every modality and education requirement in EVAL-005, bind evidence to
> exact artifacts, require the applicable blinded human or controlled study, and
> fail closed at production release boundaries. This closure does not claim that
> a live education study or evaluator deployment has occurred; absent real
> evidence blocks release.

## Work-package charter

- **Task ID:** `EVAL-005`
- **Owner:** canonical evaluation contracts, V1/V8 multimodal production, and V9
  lesson-generation/release maintainers; implementation owner: Elikem Agudogo
- **Accountable product:** shared V1–V9 autonomous-content evaluation plane,
  with V9 education as the invoked release integration and MEDIA-001 as the
  inherited multimodal battery
- **Priority:** P0-G/P1
- **Target milestone:** 2026-07-15 SOTA gap-closure program
- **Risk tier:** high education-efficacy, learner-safety, false-release, and
  evidence-substitution risk
- **Dependencies:** `MEDIA-001` multimodal batteries; canonical evaluation,
  semantic-grounding, artifact-rights, signing, and V9 lesson contracts; V9
  lesson gates, lesson forge, governance, and Prometheus production pipeline;
  real qualified-teacher and target-learner study operations for any production
  release
- **Affected schemas and content slots:** education evaluation schema version 1,
  V9 lesson artifact schema `v9.lesson.1`, G2 semantic-grounding and G3
  teachability/education gates, lesson-forge provider boundary, Prometheus
  compilation/signature surface, and the V9 education lesson content slot. The
  image, video, speech/music, and 3D/game slots are inherited from MEDIA-001 and
  were not changed by the EVAL-005 education source commit.
- **Linked task IDs:** `EVAL-005`, `MEDIA-001`, `PROOF-002`, and `PROOF-003`
- **Audit finding:** MEDIA-001 closed the image, video, audio, 3D, and gameplay
  battery requirements, but the education battery remained absent and V9's
  teachability gate did not require exact-artifact outcome evidence.
- **Repository objective:** complete the education battery, make every required
  outcome non-compensatory, and bind the result through both V9 generation paths
  and signed artifact provenance without changing the seven-gate model.
- **Rollout strategy:** additive strict contracts and explicit test fixtures,
  followed by fail-closed provider injection and G3 publication enforcement.
- **Rollback strategy:** revert source commit
  `5ebde0db9f81c8d3010e9ea0a0185d6cc7311f43`; this removes the education
  contract and restores the prior V9 G3 behavior.
- **Evidence-pack location:** this file.

## Closure matrix

| Ledger requirement  | Closure proof                                                                                                                                                                                                                                                                               |
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Image               | MEDIA-001's canonical `MediaEvaluationBundle` requires composition, prompt/source alignment, identity, typography, aesthetics, cultural/brand correctness, artifact detection, accessibility, and blinded side-randomized preference for the exact artifact.                                |
| Video               | MEDIA-001 requires temporal continuity, direction, edit rhythm, lip sync, story comprehension, flicker/artifacts, captioning, accessibility, and blinded viewer preference.                                                                                                                 |
| Speech/music        | MEDIA-001 requires intelligibility, WER where applicable, speaker/consent match, timing, loudness, silence/artifacts, musical structure, originality/fingerprint safety, accessibility, and blinded expert/listener preference.                                                             |
| 3D/game             | MEDIA-001 requires topology, UV, rigging, collision, LOD/Nanite, cook, performance, compatibility, novelty, accessibility, playability, fun, player comprehension, and blinded human comparison as applicable.                                                                              |
| Education           | The new canonical `EducationEvaluationBundle` requires source faithfulness, misconception repair, near transfer, far transfer, delayed retention, calibration, qualified-teacher judgment, learner preference, and a preregistered controlled efficacy study for the exact lesson artifact. |
| External benchmarks | Both media and education contracts retain external benchmarks as diagnostic records only. They cannot satisfy missing probes, human comparisons, controlled-study requirements, or a release threshold.                                                                                     |

The first four rows and the shared benchmark rule inherit their already
published source and closure proof from MEDIA-001. EVAL-005 adds the previously
missing education implementation and integrates it into V9's actual production
and release paths.

## Canonical education battery

`EducationEvaluationBundle` binds one versioned battery to an exact education
subject and SHA-256 artifact digest. Its eight required dimensions are:

- source faithfulness;
- misconception repair;
- near transfer;
- far transfer;
- delayed retention;
- calibration;
- teacher judgment; and
- learner preference.

Every dimension has an exact population type, score, threshold, derived status,
unique release probe, and evidence references. Claims, learners, and qualified
teachers cannot be substituted for one another. A declared `passed` status with
a sub-threshold score is rejected instead of trusted.

The controlled target-learner comparison requires a distinct baseline, frozen
preregistration, randomized assignment, learner blindness, teacher-review
blindness, consent evidence, locales, sample and vote totals, qualified teacher
reviewers, and evidence for all eight outcome families. Delayed retention must
be measured at least seven days after the intervention. The candidate must win
the blinded learner comparison; high machine or benchmark scores cannot
compensate for a missing or losing study.

`runEducationEvaluationBattery` is executable rather than a shape-only schema.
It requires all eight dimension runners and the controlled-study runner, parses
their outputs, and evaluates complete coverage. Missing runners, omitted or
duplicated probes, inflated statuses, wrong subjects/digests, insufficient
retention delay, invalid vote totals, and absent study evidence fail loud or
produce an explicit release hold.

## V9 production and release binding

The existing V9 seven-gate governance model is preserved. G3 remains one gate,
but now requires both teachability and a complete education evaluation for the
exact assembled lesson artifact. Missing, incomplete, or substituted evidence
blocks publication. G2 was also hardened so a declared Sophia source pin must
appear in the semantic-grounding citations rather than merely having the right
shape.

The lesson forge accepts explicit semantic-grounding and education-evaluation
providers. Neither provider has a production pass-through default: absence fails
closed. Test-only passing fixtures live behind explicit `test-support` exports
and are not used by production composition.

The Prometheus production pipeline independently constructs the canonical
education artifact from ground truth, narrative, media, explorables, assessment,
and schedule; derives its canonical subject and digest; obtains evaluation
evidence; and supplies that evidence to G3. Artifact compilation retains the
evaluation bundle and includes it in the signed surface hash. Publishability
requires complete evidence for the exact canonical subject and digest.
Provenance verification recomputes the skeleton and surface hashes and verifies
the Ed25519 signature, so replacing a study after signing is rejected.

## Adversarial fixture matrix

| Fixture                                                     | Result                                                                 |
| ----------------------------------------------------------- | ---------------------------------------------------------------------- |
| A required dimension runner is absent                       | Battery execution fails before a partial bundle can authorize release. |
| A required probe is missing or duplicated                   | Coverage fails; unrelated scores cannot compensate.                    |
| A caller marks a sub-threshold score as passed              | Schema/evaluator rejects the inflated status.                          |
| Evidence names another subject or digest                    | Exact-artifact coverage and V9 G3 fail.                                |
| Retention is measured before seven days                     | Controlled efficacy study is invalid.                                  |
| Vote totals disagree with the declared sample               | Controlled study is invalid.                                           |
| Baseline wins learner preference                            | Education coverage fails.                                              |
| External benchmark is high but a release probe is absent    | Benchmark remains diagnostic; release stays held.                      |
| V9 has no education provider                                | Forge/Prometheus fail closed and produce no publishable artifact.      |
| Semantic evidence declares but does not cite the source pin | G2 fails.                                                              |
| Signed Prometheus artifact receives a substituted study     | Recomputed surface hash/signature verification fails.                  |

## Automated verification

### Current admitted refresh

The current refresh ran against repository commit
`a2373f54fa9930fa65cabb0a376d75f42ef8bfa8` with lockfile SHA-256
`08b055786e9b8a6104d5313601d7eef3205dedcf395066fd9a8264c9859d690f` on host
`app-dev-server`, Ubuntu 24.04.4 LTS, Linux `6.8.0-134-generic` x86_64. The
relevant toolchain was Node.js `v24.15.0`, pnpm `10.25.0`, TypeScript `6.0.3`,
and Vitest `4.1.7`.

No live model/provider, educator panel, learner cohort, customer data,
credential, or deployed V9 service was used. The test-support education bundles
are explicitly versioned fixtures; they prove fail-closed composition and exact
artifact binding, never a real efficacy outcome.

| UTC interval                                      | Exit | Exact command and result                                                                                                                                                                                                                                                      |
| ------------------------------------------------- | ---: | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 2026-07-29T11:41:57.632Z–2026-07-29T11:41:59.244Z |    0 | `pnpm exec vitest run --config libs/contracts/vitest.config.ts libs/contracts/src/common/education-evaluation.test.ts libs/contracts/src/v9/__tests__/v9-contracts.spec.ts libs/contracts/src/common/contract-prisma-alignment.test.ts --maxWorkers=1` — 3 files, 30/30 tests |
| 2026-07-29T11:42:03.998Z–2026-07-29T11:42:08.906Z |    0 | `pnpm exec vitest run --config libs/v9/lesson-gates/vitest.config.ts --maxWorkers=1` — 9/9 tests                                                                                                                                                                              |
| 2026-07-29T11:42:16.071Z–2026-07-29T11:42:23.198Z |    0 | `pnpm exec vitest run --config libs/v9/lesson-explorables/vitest.config.ts --maxWorkers=1` — 5 files, 22/22 tests                                                                                                                                                             |
| 2026-07-29T11:42:29.488Z–2026-07-29T11:42:34.678Z |    0 | `pnpm exec vitest run --config libs/v9/governance/vitest.config.ts --maxWorkers=1` — 10/10 tests                                                                                                                                                                              |
| 2026-07-29T11:42:42.722Z–2026-07-29T11:42:51.105Z |    0 | `pnpm exec vitest run --config libs/v9/prometheus/vitest.config.ts --maxWorkers=1` — 8/8 tests                                                                                                                                                                                |
| 2026-07-29T11:43:03.040Z–2026-07-29T11:43:17.572Z |    0 | `pnpm exec tsc -p libs/contracts/tsconfig.lib.json --noEmit`                                                                                                                                                                                                                  |
| 2026-07-29T11:43:17.574Z–2026-07-29T11:43:28.683Z |    0 | `pnpm exec tsc -p libs/v9/lesson-gates/tsconfig.json --noEmit`                                                                                                                                                                                                                |
| 2026-07-29T11:43:28.684Z–2026-07-29T11:43:40.815Z |    0 | `pnpm exec tsc -p libs/v9/lesson-explorables/tsconfig.json --noEmit`                                                                                                                                                                                                          |
| 2026-07-29T11:43:40.817Z–2026-07-29T11:43:57.780Z |    0 | `pnpm exec tsc -p libs/v9/governance/tsconfig.json --noEmit`                                                                                                                                                                                                                  |
| 2026-07-29T11:43:57.781Z–2026-07-29T11:44:15.559Z |    0 | `pnpm exec tsc -p libs/v9/prometheus/tsconfig.json --noEmit`                                                                                                                                                                                                                  |

The admitted current suites passed **79/79** value assertions and all five
owning typechecks. They cover the applicable boundaries as follows:

- **Unit/schema/contract:** exact dimension coverage, derived thresholds,
  population types, artifact digests, study constraints, V9 lesson
  publishability, and contract-to-persistence alignment.
- **Integration:** lesson-gate, lesson-forge, seven-gate governance, and
  complete Prometheus generation/compilation/provenance paths consume the
  canonical bundle instead of private pass-through shapes.
- **Adversarial/negative:** missing runners or dimensions, high diagnostic
  benchmark substitution, wrong artifact bytes, duplicate probes, inflated pass
  status, early retention measurement, invalid votes, losing learner preference,
  missing providers, uncited source pins, and signed-study substitution all fail
  for their intended reasons.
- **Repository E2E:** the Prometheus pipeline reaches signed artifact
  compilation and re-verification with explicit injected evaluation evidence.
  This is not represented as a deployed provider, educator/learner study, or
  production release.

UI, accessibility automation, screenshot/media probes, mobile/Unreal automation,
and production performance/load tests are inapplicable to this evidence-only
refresh and the underlying contract/runtime change. No rendered surface, media
byte, mobile flow, Unreal asset, or deployed latency boundary was changed.
MEDIA-001 retains the separately published media-probe evidence for the
inherited image/video/audio/3D/game batteries.

### Historical implementation verification

At source publication, the integrated source passed **4,133/4,133**
non-duplicated owning-package tests:

| Surface                                      |      Result |
| -------------------------------------------- | ----------: |
| Canonical contracts full package             | 4,085/4,085 |
| V9 lesson gates                              |         9/9 |
| V9 lesson explorables/forge                  |       22/22 |
| V9 seven-gate governance                     |       10/10 |
| V9 Prometheus production pipeline/provenance |         7/7 |

The final exact-source focused reruns passed 30/30 canonical education/V9
contract/alignment tests, 9/9 lesson-gate tests, and 7/7 Prometheus tests. These
focused runs overlap the totals above and are not double-counted. MEDIA-001's
separately published multimodal implementation passed **4,351/4,351** tests at
its closure.

Typechecks passed for contracts, lesson-gates, lesson-explorables, V9
governance, and Prometheus. The canonical contracts emitted TypeScript build and
Nx build passed. Explicit ESLint over every changed production TypeScript file
passed; ignored test files were then checked with `--no-ignore` and passed with
zero errors or warnings. Prettier and `git diff --check` passed.

Representative commands:

```text
pnpm --dir libs/contracts exec vitest run --testTimeout=120000
pnpm --dir libs/contracts exec vitest run <education/V9/alignment files>
pnpm --dir libs/v9/lesson-gates exec vitest run --config vitest.config.ts
pnpm --dir libs/v9/lesson-explorables exec vitest run --config vitest.config.ts
pnpm --dir libs/v9/governance exec vitest run --config vitest.config.ts
pnpm --dir libs/v9/prometheus exec vitest run --config vitest.config.ts
pnpm exec tsc --noEmit -p <each owning package tsconfig>
pnpm exec tsc -p libs/contracts/tsconfig.lib.json
NX_DAEMON=false NX_TUI=false pnpm nx run @oshun/contracts:build --parallel=1 --skip-nx-cache
pnpm exec eslint <changed production TypeScript>
pnpm exec eslint --no-ignore <changed test TypeScript>
pnpm exec prettier --check <all changed non-lock files>
git diff --check
```

## Bounded verification notes

- Before the 2026-07-29 focused refresh, the host had 15 GiB RAM, 31 GiB swap,
  11 GiB available memory, and 16 GiB free disk. An unrelated checkout's
  long-running Vitest worker and this checkout's Nx daemon were active, so no
  broad contracts/workspace matrix or production build was started; the five
  small owning suites and five typechecks ran serially.
- The host had 15 GiB RAM and 31 GiB swap with other long-lived agent/Nx
  processes present. Broad workspace gates were therefore inapplicable under the
  repository's safety policy; all owning-package checks ran sequentially.
- The full contracts run was supervised and passed 4,085 tests in 178 files.
  Later canonical digest integration was covered by the 30-test focused rerun,
  contracts typecheck, and emitting contracts build.
- The source rebase integrated four unrelated V10 commits. No EVAL-005 source
  file overlapped; the only shared path was the unrelated binary lockfile.
- The pre-existing `pnpm-lock.yaml` modification was preserved byte-for-byte
  through hooks and rebase (Git blob
  `a37431842e330b3a06b030947c25bebdc72bc2b1`). Generated Cargo locks and
  `libs/urania/` were not staged.
- The source commit used the hook's documented `SKIP_TYPECHECK=1` only after all
  five owning-package typechecks and the contracts builds had passed. Hook
  ESLint, Prettier, asset locking, and documentation validation passed.
- No customer UI, PWA, admin, mobile, Unreal, or visual behavior changed.
  Frontend skill, Playwright, mobile automation, screenshots, and Unreal
  verification were inapplicable.

## Artifact identity

| Artifact                       | SHA-256                                                            |
| ------------------------------ | ------------------------------------------------------------------ |
| Canonical education evaluation | `b379e87e259306579eb0e80c1aa46ab135ed72a3cdf90efe9cc63be7523aaee1` |
| Canonical V9 lesson binding    | `0fc16d84017ea55cd5b8b3ff6653dfb31b43ef0f7392103fa9d023b4794a1c50` |
| V9 G2/G3 release gates         | `696b0f5ef5503664f810ce77b4e4b8ffae25444fbaf3169d4d38eb8b5d365b23` |
| V9 fail-closed lesson forge    | `51857787a41c933f915cb97393f9bc753dfc7eb941adf726b955572cb571002c` |
| Prometheus education pipeline  | `52d20e2dc066880532ecabf6405cd15985c7b5e1da3429e14dfe661a6b5e134a` |
| Prometheus release gates       | `db94ae3639b4d1754fa20df76c04d1e13800d15f6628642a5f22f9e7e53a1be7` |
| Prometheus signed compilation  | `7e9dd3bffaa2a2b8cbd3afe5b98303a7d7cd67790ba121659c3c3d741a19c654` |

These hashes identify current source at the admitted refresh commit. Later V9
privacy and retrieval-proof work invalidated the original Prometheus pipeline
and compilation hashes; the current 79 tests and five typechecks cover the
integrated source. Governing versions are education evaluation schema 1 and V9
lesson schema `v9.lesson.1`. Test fixtures use probe/methodology/rater/policy
references ending in `v1` or `2026.07.1`; those are test identities, not
production rubric, policy, human-study, provider, or model versions.

## Release, human-authority, blocker, and generated-status disposition

- The repository verdict is `repository-complete / closed` because the contracts
  and invoked V9 boundaries exist and fail closed. Production release remains
  `blocked` unless the exact lesson carries real probe evidence and a valid
  preregistered controlled study from the target learner population.
- The contract requires qualified teachers, consent evidence, learner and
  teacher blindness, and target-learner outcomes for release. No such human
  decision is needed to publish the repository contract itself, and no reviewer
  identity, signature, teacher judgment, learner preference, or efficacy result
  is fabricated in this pack.
- Remaining external dependencies are real scorer services, a frozen
  preregistration, consented target learners, qualified teachers, delayed
  retention collection, provider/runtime deployment, and exact-artifact release
  evidence. Fixtures and high external benchmark scores cannot discharge them.
- No generated capability/status artifact claims live educational efficacy or a
  production release. The completed-work-package admission verifier derives only
  this checked task's evidence-pack presence and identity; no generated green
  outcome was hand-edited for this refresh.

## Scope and remaining seams

- Real education dimension scorers, controlled-study operations, qualified
  teacher panels, and learner cohorts remain deployment services behind the
  canonical runner/provider interfaces. Their absence blocks publication; this
  package does not present fixtures as human evidence.
- The battery establishes release authority and exact-artifact integrity. It
  does not claim population superiority, a completed field trial, or a live
  provider deployment.
- External benchmark records remain useful for diagnosis and regression triage,
  but never grant release authority.

## Publication identity

- **MEDIA-001 source commit:** `f8046ba3a573f082aa24e95573cd30de8a5c31d6`
- **MEDIA-001 closure evidence commit:**
  `573cce758a2d33cb5caaee8c0e824da09a0eb966`
- **EVAL-005 source commit:** `5ebde0db9f81c8d3010e9ea0a0185d6cc7311f43`
- **EVAL-005 source publication tip:**
  `5ebde0db9f81c8d3010e9ea0a0185d6cc7311f43`
- **EVAL-005 closure evidence commit:**
  `9a1e240288e9aeb7778c205faf1b172c004191e9`
- **Current refresh source commit:** `a2373f54fa9930fa65cabb0a376d75f42ef8bfa8`
- **Current refresh evidence commit:** this evidence-only publication commit;
  the immutable identity is the Git commit containing this file.
