Eve library

AUD-002 evidence pack — generated-evidence freshness enforcement

Evidence

Revision 1

On this page

Work-package charter#

  • Task IDs: AUD-002
  • Owner: Oshun portfolio audit-integrity maintainer; implementation owner: Codex
  • Accountable product: V1–V9 autonomous-content portfolio
  • Priority: P0
  • Target milestone: 2026-07-15 SOTA gap-closure program
  • Risk tier: Critical release-evidence integrity
  • Dependencies: Generated gate producers and consumers, git source history, V3 launch-readiness generator, provider/engine capable hosts
  • Affected schemas and content slots: New shared generated-evidence metadata schema; no content slot or shipped artifact changes in the foundation slice
  • Rollout strategy: Land and prove the shared metadata/freshness contract, migrate the V3 launch gate as the first end-to-end consumer, then inventory and migrate each remaining traceable generated gate family without pretending hand-authored records were generated.
  • Rollback strategy: Revert the task-scoped migration commit for a producer and consumer together. Never roll back a consumer alone to accept metadata it can no longer authenticate.
  • Evidence-pack location: This file

Planned slices#

  1. Complete. Shared metadata builder, canonical digest, per-class freshness SLOs, and fail-closed evaluator with stale, future, wrong-commit, wrong-contract, missing-field, lifecycle, and tamper coverage.
  2. Complete. V3 launch-readiness generator and consumer migration, including explicit executed/written/published lifecycle states.
  3. Complete. Clean capable-host V3 run through the supported --write-evidence path, bounded log, and committed fresh artifact.
  4. Complete. Mechanical producer/consumer inventory and migration of every remaining traceable generated gate artifact.

All four slices and every AUD-002 ledger requirement are complete. Product readiness remains independent: current red gate verdicts stay red and unpublished.

Slice 1 closure evidence#

  • Implementation commit: e14c6423ca1ae26fac554dac0c0c297c47a07a1d
  • Verified at: 2026-07-15T12:24:26.371Z
  • Verification host: dev-server; Linux 6.8.0-110-generic x86_64
  • Toolchain: Node.js v24.14.1, pnpm 10.25.0, Git 2.43.0
  • pnpm verify:evidence:metadata — pass, 12/12 tests
  • pnpm exec eslint --no-ignore scripts/evidence/generated-evidence-metadata.mjs scripts/evidence/generated-evidence-metadata.test.mjs — pass
  • pnpm exec prettier --write package.json evidence/v1-v9/AUD-002.md scripts/evidence/generated-evidence-metadata.mjs scripts/evidence/generated-evidence-metadata.test.mjs — pass, unchanged after formatting
  • git diff --check — pass

The foundation defines six evidence classes with separate SLOs, binds evidence to a full source commit, versioned generator, exact command, host, contract digest, and canonical payload digest, and records verifier completion, evidence write, and publication as independent lifecycle states. Its consumer is fail-closed for stale, future-dated, unresolved-commit, wrong-contract, tampered, incomplete, and causally invalid evidence. At slice 1 closure, no product gate migration or fresh capable-host artifact was claimed; slices 2–3 below now close that work for V3, while the portfolio inventory remains open in slice 4.

Slices 2–3 closure evidence — V3 launch readiness#

  • Committed generator source: a25d8f0ad74fe887b3538c3200a5131620c5b40d
  • Artifact and consumer coverage commit: 954c78fe10a8e256222c2896348d7069619b784b
  • Verified through: 2026-07-15T12:36:22.235Z
  • Verification host: dev-server; Linux 6.8.0-110-generic x86_64

Bounded capable-host run log#

Observation Recorded result
Command node scripts/v3/verify-v3-launch-readiness.mjs --write-evidence
Verifier interval 2026-07-15T12:30:23.482Z through 2026-07-15T12:30:30.172Z
Process result Exit 1, expected because the measured release gate remains red
Command coverage All 41 distinct constituent commands executed; 30 passed and 11 failed
Named-gate coverage 24/39 passed; 15 failed
Artifact write V3/launch/launch-readiness-release-gate.json written successfully
Structural/metadata result No structural drift / dishonest evidence section; the freshly written metadata was accepted by the consumer in the same run
Honest readiness result gaReady: false; 39 waived external tasks, missing §1–§74 completion, 15 failing named gates, and 11 failing sub-gates reported
Follow-up consumer command node scripts/v3/verify-v3-launch-readiness.mjs at 2026-07-15T12:33:28Z
Follow-up consumer result Exit 1 for the same readiness-only blockers; no freshness, contract, source, payload, lifecycle, generator, command, or other structural failure

The nonzero result is the required honest launch decision, not a failed verification run. The writer completed, wrote current evidence, immediately consumed that evidence without structural failure, and refused to report GA readiness while the measured product gates remained red.

Artifact metadata#

  • Evidence schema: oshun.generated-evidence-metadata.v1
  • Evidence class and SLO: release-readiness, 24 hours
  • Generator: scripts/v3/verify-v3-launch-readiness.mjs version 2.0.0
  • Exact write command ID: verify:v3:launch-readiness:write-evidence
  • Source commit recorded in the artifact: a25d8f0ad74fe887b3538c3200a5131620c5b40d
  • Contract ID and digest: v3.launch-readiness-release-gate.contract.v1, 1999044f214edccdc6aeee50aaccb6778d2894c01b567939e7c09cd9de98b044
  • Canonical payload digest: 38cc60cfa46682e65b4ffbd79bdf944c0def5f65abe74bdbe28cd4d82fa27da1
  • Lifecycle: verifier completed, evidence written, release publication unpublished. The publication state is intentionally not promoted while gaReady is false.

Automated verification#

  • pnpm verify:v3:launch-readiness-evidence — pass, 6/6 tests, including current-artifact authentication plus tampered-payload, wrong-contract, unavailable-commit, stale-evidence, and missing-lifecycle rejection
  • pnpm verify:evidence:metadata — pass, 12/12 shared contract tests
  • pnpm exec vitest run --config libs/oshun/analytics/vitest.config.ts libs/oshun/analytics/src/__tests__/v3-launch-readiness.test.ts --maxWorkers=1 — pass, 7/7 tests
  • pnpm exec eslint --no-ignore scripts/v3/verify-v3-launch-readiness.mjs scripts/v3/verify-v3-launch-readiness-evidence.test.mjs libs/oshun/analytics/src/__tests__/v3-launch-readiness.test.ts — pass
  • pnpm exec tsc -p libs/oshun/analytics/tsconfig.lib.json --noEmit — pass
  • Commit-hook affected typechecks — pass for @oshun/analytics, @oshun/mobile, @oshun/web, and @oshun/bff
  • git diff --check — pass

Integrity digests#

File SHA-256
V3/launch/launch-readiness-release-gate.json 6302314c5979611ed5e1ea6d4479673debe5f5a63097f23b6bd3385cdf6d08fe
scripts/v3/verify-v3-launch-readiness.mjs c0ed0cfd6cadd59d98bc45217f5a1f7c95c929c1cf8fc77ff381ad4e264f2389
scripts/v3/verify-v3-launch-readiness-evidence.test.mjs 385728053eb7241344beb341fa9f970688a39d580cfc4ec59a3a3d1aa8622bba
libs/oshun/analytics/src/__tests__/v3-launch-readiness.test.ts 33f1ca71e66de349373485f6376b6ec2ffa32b406ee70c1c8b0dd569faa743fb
libs/oshun/analytics/src/v3-launch-readiness.ts c19f221a79e501079c546e877c0bc21a958e8687456d7343997e4c09c3a55982

Slice 4 closure evidence — portfolio generated-gate inventory#

  • Producer/runtime implementation: 0d1b90b065da410c8536cf60b03cd1d6afc0581e
  • Sequential regeneration command: c776c653d928dd5e8d3c032700027fedd336cab0
  • Dashboard/validator contract correction: a7c32f96440941b9bd1713069683a02aec11d8f4
  • Final generated-artifact portfolio: d0f3dabf7ee7af0b94bcd6a4ad1d048efccded5b
  • Verified through: 2026-07-15T13:23:16Z
  • Verification host: dev-server; Linux 6.8.0-110-generic x86_64
  • Toolchain: Node.js v24.14.1, pnpm 10.25.0, Git 2.43.0
  • Affected content slots: Release/audit evidence only; no shipped content, placement target, customer flow, media, or provider output changed.

Mechanical inventory and classification#

Inventory class Count Enforcement
Committed ISIS V2 generated gate artifacts 416 One executable producer and one legacy validator per artifact; metadata authenticated and fresh
Direct ISIS producer scripts 400 Instrumented at the JSON write boundary
ISIS wrappers using the shared multi-control producer 16 Instrumented inside the shared producer
Committed V3 generated launch gate 1 Migrated in slices 2–3
Dynamic RunPod generated-gate producer 1 Provider-probe metadata attached to every emitted report
Dated, hand-authored V3 gate records with no executable producer 5 Explicitly classified as non-generated; no fabricated provenance
Unclassified dated gate records 0 Inventory consumer fails if this becomes nonzero

The five non-generated records are V3/launch/ga-inventory-release-gate.json, V3/qa/testing-qa-accessibility-release-gate.json, and the three V3 security validation records named in scripts/evidence/verify-generated-gate-inventory.mjs. The dynamic RunPod pattern is tmp/isis-release-gate/release-gate-report-*.json; no current real provider publication is claimed.

Final artifact state and integrity#

  • All 416 ISIS artifacts use schema oshun.generated-evidence-metadata.v1, evidence class audit-verification, generator version 1.0.0, host dev-server, and source commit a7c32f96440941b9bd1713069683a02aec11d8f4.
  • Generation ran sequentially from 2026-07-15T13:15:23.867Z through 2026-07-15T13:16:15.184Z; evidence writes completed by 2026-07-15T13:16:15.185Z.
  • The canonical contract digest for each producer covers that executable and its recursive local-import closure. The payload digest excludes metadata and detects post-generation artifact mutation.
  • Aggregate SHA-256 over the sorted 416 sha256sum records: 7cb673d2d599cabf73f7fef77bbac1205de706baf33aebfc00a61aabf3f6c033.
  • Current product verdicts are 358 pass and 58 fail. All 416 publication states are deliberately unpublished; freshness enforcement does not promote a release decision.
  • A stale dashboard contract was exposed during regeneration: it listed only failing controls while 191 downstream certificates required addressable rows for controls that had improved to passing. The dashboard now retains all 38 benchmarked controls, and a regression test proves exact ID/verdict/required parity. No green claim was restored by hand.

Class-specific SLO policy#

Evidence class Freshness SLO
provider-probe 6 hours
release-readiness 1 day
security-validation 1 day
audit-verification 7 days
engine-execution 7 days
human-review 30 days

Execution completion, evidence write, and publication are separate lifecycle records with causal timestamp validation. Consumers may require publication independently; the portfolio freshness gate requires authentic current written evidence without misreporting unpublished artifacts as released.

Exact verification record#

Command Exit/result
pnpm generate:evidence:generated-gates 0; regenerated 416/416 sequentially
pnpm verify:evidence:metadata 0; 21/21 metadata, dynamic-producer, contract, inventory, dashboard, and verdict-agreement tests passed
pnpm verify:evidence:generated-gate-producers 0; 400 direct producers, 16 shared wrappers, and one dynamic producer verified
pnpm verify:evidence:generated-gates 0; 416 ISIS artifacts, one V3 artifact, one dynamic producer, five explicit non-generated records, zero unclassified records
pnpm verify:evidence:generated-gate-validators 0; 358 green artifacts accepted and 58 red artifacts rejected as recorded across 416/416 legacy validators
Targeted ESLint over all shared evidence scripts, tests, the RunPod producer, and dashboard generator 0; no errors or warnings
Node syntax checks and git diff --check 0
pnpm verify:v3:launch-readiness Expected 1, 2026-07-15T13:23:09Z–13:23:16Z; readiness-only blockers, with no freshness/contract/source/payload/lifecycle failure

The scheduled and change-triggered .github/workflows/generated-evidence-freshness.yml checks out full history, verifies producer instrumentation, runs all negative metadata fixtures, authenticates freshness and contracts, and requires legacy-validator agreement with every recorded artifact verdict.

Load-bearing negative evidence#

Automated fixtures fail for stale evidence, future timestamps beyond clock skew, unresolved/wrong source commits, wrong contracts, tampered payloads, missing generator/command/host/lifecycle fields, incomplete or causally reversed execution/write lifecycle, and invalid builder inputs. Additional tests prove a green artifact fails verification when its legacy validator rejects it, a red artifact fails verification when its validator accepts it, and a red validator must reject the recorded releaseGate.status rather than merely crash.

Honest remaining product state#

AUD-002 closes evidence freshness, not product readiness. The 58 current ISIS exposure gates remain red for measured runtime/API/SDK/UI/E2E gaps, and the V3 launch gate remains 30/41 required commands and 24/39 named gates with gaReady: false. No provider, engine, media, placement, screenshot, or human review proof applies to this metadata-control work package. No policy in scope requires a signed human release decision, and none is claimed.

Key implementation digests#

File SHA-256
scripts/evidence/generated-evidence-metadata.mjs 29023a2ee29778f1be52490c562df3ba4ec0f6065360462c768a74cfa89aa94a
scripts/evidence/generated-gate-runtime.mjs 579fcafd1e4482089b4f19b7846617f7e0fe5c402ade0dd3c90aa0b6c8a7bae0
scripts/evidence/instrument-generated-gate-producers.mjs 482b72de58154df981cdd70c9ed7150e3f7ec08bed90dc4dad6700a0d066cb29
scripts/evidence/regenerate-generated-gates.mjs 6ad05232f87b65d328c57be7442725029e182d796cb614ea9b1692a1638c1bf0
scripts/evidence/verify-generated-gate-inventory.mjs 7e769af164b99c37ec7bf4ddfb731eb4dc0e023accbc25ad0ec3e9b7af497c51
.github/workflows/generated-evidence-freshness.yml 6b2a9c1e6caa4dc56270aa953ada723d60ba59e356c3dce5cf2492781bae94a5