Work-package charter#
- Task IDs:
AUD-002 - Owner: Oshun portfolio audit-integrity maintainer; implementation owner: Codex
- Accountable product: V1–V9 autonomous-content portfolio
- Priority: P0
- Target milestone: 2026-07-15 SOTA gap-closure program
- Risk tier: Critical release-evidence integrity
- Dependencies: Generated gate producers and consumers, git source history, V3 launch-readiness generator, provider/engine capable hosts
- Affected schemas and content slots: New shared generated-evidence metadata schema; no content slot or shipped artifact changes in the foundation slice
- Rollout strategy: Land and prove the shared metadata/freshness contract, migrate the V3 launch gate as the first end-to-end consumer, then inventory and migrate each remaining traceable generated gate family without pretending hand-authored records were generated.
- Rollback strategy: Revert the task-scoped migration commit for a producer and consumer together. Never roll back a consumer alone to accept metadata it can no longer authenticate.
- Evidence-pack location: This file
Planned slices#
- Complete. Shared metadata builder, canonical digest, per-class freshness SLOs, and fail-closed evaluator with stale, future, wrong-commit, wrong-contract, missing-field, lifecycle, and tamper coverage.
- Complete. V3 launch-readiness generator and consumer migration, including explicit executed/written/published lifecycle states.
- Complete. Clean capable-host V3 run through the supported
--write-evidencepath, bounded log, and committed fresh artifact. - Complete. Mechanical producer/consumer inventory and migration of every remaining traceable generated gate artifact.
All four slices and every AUD-002 ledger requirement are complete. Product readiness remains independent: current red gate verdicts stay red and unpublished.
Slice 1 closure evidence#
- Implementation commit:
e14c6423ca1ae26fac554dac0c0c297c47a07a1d - Verified at:
2026-07-15T12:24:26.371Z - Verification host:
dev-server; Linux6.8.0-110-genericx86_64 - Toolchain: Node.js
v24.14.1, pnpm10.25.0, Git2.43.0 pnpm verify:evidence:metadata— pass, 12/12 testspnpm exec eslint --no-ignore scripts/evidence/generated-evidence-metadata.mjs scripts/evidence/generated-evidence-metadata.test.mjs— passpnpm exec prettier --write package.json evidence/v1-v9/AUD-002.md scripts/evidence/generated-evidence-metadata.mjs scripts/evidence/generated-evidence-metadata.test.mjs— pass, unchanged after formattinggit diff --check— pass
The foundation defines six evidence classes with separate SLOs, binds evidence to a full source commit, versioned generator, exact command, host, contract digest, and canonical payload digest, and records verifier completion, evidence write, and publication as independent lifecycle states. Its consumer is fail-closed for stale, future-dated, unresolved-commit, wrong-contract, tampered, incomplete, and causally invalid evidence. At slice 1 closure, no product gate migration or fresh capable-host artifact was claimed; slices 2–3 below now close that work for V3, while the portfolio inventory remains open in slice 4.
Slices 2–3 closure evidence — V3 launch readiness#
- Committed generator source:
a25d8f0ad74fe887b3538c3200a5131620c5b40d - Artifact and consumer coverage commit:
954c78fe10a8e256222c2896348d7069619b784b - Verified through:
2026-07-15T12:36:22.235Z - Verification host:
dev-server; Linux6.8.0-110-genericx86_64
Bounded capable-host run log#
| Observation | Recorded result |
|---|---|
| Command | node scripts/v3/verify-v3-launch-readiness.mjs --write-evidence |
| Verifier interval | 2026-07-15T12:30:23.482Z through 2026-07-15T12:30:30.172Z |
| Process result | Exit 1, expected because the measured release gate remains red |
| Command coverage | All 41 distinct constituent commands executed; 30 passed and 11 failed |
| Named-gate coverage | 24/39 passed; 15 failed |
| Artifact write | V3/launch/launch-readiness-release-gate.json written successfully |
| Structural/metadata result | No structural drift / dishonest evidence section; the freshly written metadata was accepted by the consumer in the same run |
| Honest readiness result | gaReady: false; 39 waived external tasks, missing §1–§74 completion, 15 failing named gates, and 11 failing sub-gates reported |
| Follow-up consumer command | node scripts/v3/verify-v3-launch-readiness.mjs at 2026-07-15T12:33:28Z |
| Follow-up consumer result | Exit 1 for the same readiness-only blockers; no freshness, contract, source, payload, lifecycle, generator, command, or other structural failure |
The nonzero result is the required honest launch decision, not a failed verification run. The writer completed, wrote current evidence, immediately consumed that evidence without structural failure, and refused to report GA readiness while the measured product gates remained red.
Artifact metadata#
- Evidence schema:
oshun.generated-evidence-metadata.v1 - Evidence class and SLO:
release-readiness, 24 hours - Generator:
scripts/v3/verify-v3-launch-readiness.mjsversion2.0.0 - Exact write command ID:
verify:v3:launch-readiness:write-evidence - Source commit recorded in the artifact:
a25d8f0ad74fe887b3538c3200a5131620c5b40d - Contract ID and digest:
v3.launch-readiness-release-gate.contract.v1,1999044f214edccdc6aeee50aaccb6778d2894c01b567939e7c09cd9de98b044 - Canonical payload digest:
38cc60cfa46682e65b4ffbd79bdf944c0def5f65abe74bdbe28cd4d82fa27da1 - Lifecycle: verifier
completed, evidencewritten, release publicationunpublished. The publication state is intentionally not promoted whilegaReadyis false.
Automated verification#
pnpm verify:v3:launch-readiness-evidence— pass, 6/6 tests, including current-artifact authentication plus tampered-payload, wrong-contract, unavailable-commit, stale-evidence, and missing-lifecycle rejectionpnpm verify:evidence:metadata— pass, 12/12 shared contract testspnpm exec vitest run --config libs/oshun/analytics/vitest.config.ts libs/oshun/analytics/src/__tests__/v3-launch-readiness.test.ts --maxWorkers=1— pass, 7/7 testspnpm exec eslint --no-ignore scripts/v3/verify-v3-launch-readiness.mjs scripts/v3/verify-v3-launch-readiness-evidence.test.mjs libs/oshun/analytics/src/__tests__/v3-launch-readiness.test.ts— passpnpm exec tsc -p libs/oshun/analytics/tsconfig.lib.json --noEmit— pass- Commit-hook affected typechecks — pass for
@oshun/analytics,@oshun/mobile,@oshun/web, and@oshun/bff git diff --check— pass
Integrity digests#
| File | SHA-256 |
|---|---|
V3/launch/launch-readiness-release-gate.json |
6302314c5979611ed5e1ea6d4479673debe5f5a63097f23b6bd3385cdf6d08fe |
scripts/v3/verify-v3-launch-readiness.mjs |
c0ed0cfd6cadd59d98bc45217f5a1f7c95c929c1cf8fc77ff381ad4e264f2389 |
scripts/v3/verify-v3-launch-readiness-evidence.test.mjs |
385728053eb7241344beb341fa9f970688a39d580cfc4ec59a3a3d1aa8622bba |
libs/oshun/analytics/src/__tests__/v3-launch-readiness.test.ts |
33f1ca71e66de349373485f6376b6ec2ffa32b406ee70c1c8b0dd569faa743fb |
libs/oshun/analytics/src/v3-launch-readiness.ts |
c19f221a79e501079c546e877c0bc21a958e8687456d7343997e4c09c3a55982 |
Slice 4 closure evidence — portfolio generated-gate inventory#
- Producer/runtime implementation:
0d1b90b065da410c8536cf60b03cd1d6afc0581e - Sequential regeneration command:
c776c653d928dd5e8d3c032700027fedd336cab0 - Dashboard/validator contract correction:
a7c32f96440941b9bd1713069683a02aec11d8f4 - Final generated-artifact portfolio:
d0f3dabf7ee7af0b94bcd6a4ad1d048efccded5b - Verified through:
2026-07-15T13:23:16Z - Verification host:
dev-server; Linux6.8.0-110-genericx86_64 - Toolchain: Node.js
v24.14.1, pnpm10.25.0, Git2.43.0 - Affected content slots: Release/audit evidence only; no shipped content, placement target, customer flow, media, or provider output changed.
Mechanical inventory and classification#
| Inventory class | Count | Enforcement |
|---|---|---|
| Committed ISIS V2 generated gate artifacts | 416 | One executable producer and one legacy validator per artifact; metadata authenticated and fresh |
| Direct ISIS producer scripts | 400 | Instrumented at the JSON write boundary |
| ISIS wrappers using the shared multi-control producer | 16 | Instrumented inside the shared producer |
| Committed V3 generated launch gate | 1 | Migrated in slices 2–3 |
| Dynamic RunPod generated-gate producer | 1 | Provider-probe metadata attached to every emitted report |
| Dated, hand-authored V3 gate records with no executable producer | 5 | Explicitly classified as non-generated; no fabricated provenance |
| Unclassified dated gate records | 0 | Inventory consumer fails if this becomes nonzero |
The five non-generated records are V3/launch/ga-inventory-release-gate.json,
V3/qa/testing-qa-accessibility-release-gate.json, and the three V3 security
validation records named in
scripts/evidence/verify-generated-gate-inventory.mjs. The dynamic RunPod
pattern is tmp/isis-release-gate/release-gate-report-*.json; no current real
provider publication is claimed.
Final artifact state and integrity#
- All 416 ISIS artifacts use schema
oshun.generated-evidence-metadata.v1, evidence classaudit-verification, generator version1.0.0, hostdev-server, and source commita7c32f96440941b9bd1713069683a02aec11d8f4. - Generation ran sequentially from
2026-07-15T13:15:23.867Zthrough2026-07-15T13:16:15.184Z; evidence writes completed by2026-07-15T13:16:15.185Z. - The canonical contract digest for each producer covers that executable and its recursive local-import closure. The payload digest excludes metadata and detects post-generation artifact mutation.
- Aggregate SHA-256 over the sorted 416
sha256sumrecords:7cb673d2d599cabf73f7fef77bbac1205de706baf33aebfc00a61aabf3f6c033. - Current product verdicts are 358 pass and 58 fail. All 416 publication states
are deliberately
unpublished; freshness enforcement does not promote a release decision. - A stale dashboard contract was exposed during regeneration: it listed only failing controls while 191 downstream certificates required addressable rows for controls that had improved to passing. The dashboard now retains all 38 benchmarked controls, and a regression test proves exact ID/verdict/required parity. No green claim was restored by hand.
Class-specific SLO policy#
| Evidence class | Freshness SLO |
|---|---|
provider-probe |
6 hours |
release-readiness |
1 day |
security-validation |
1 day |
audit-verification |
7 days |
engine-execution |
7 days |
human-review |
30 days |
Execution completion, evidence write, and publication are separate lifecycle records with causal timestamp validation. Consumers may require publication independently; the portfolio freshness gate requires authentic current written evidence without misreporting unpublished artifacts as released.
Exact verification record#
| Command | Exit/result |
|---|---|
pnpm generate:evidence:generated-gates |
0; regenerated 416/416 sequentially |
pnpm verify:evidence:metadata |
0; 21/21 metadata, dynamic-producer, contract, inventory, dashboard, and verdict-agreement tests passed |
pnpm verify:evidence:generated-gate-producers |
0; 400 direct producers, 16 shared wrappers, and one dynamic producer verified |
pnpm verify:evidence:generated-gates |
0; 416 ISIS artifacts, one V3 artifact, one dynamic producer, five explicit non-generated records, zero unclassified records |
pnpm verify:evidence:generated-gate-validators |
0; 358 green artifacts accepted and 58 red artifacts rejected as recorded across 416/416 legacy validators |
| Targeted ESLint over all shared evidence scripts, tests, the RunPod producer, and dashboard generator | 0; no errors or warnings |
Node syntax checks and git diff --check |
0 |
pnpm verify:v3:launch-readiness |
Expected 1, 2026-07-15T13:23:09Z–13:23:16Z; readiness-only blockers, with no freshness/contract/source/payload/lifecycle failure |
The scheduled and change-triggered
.github/workflows/generated-evidence-freshness.yml checks out full history,
verifies producer instrumentation, runs all negative metadata fixtures,
authenticates freshness and contracts, and requires legacy-validator agreement
with every recorded artifact verdict.
Load-bearing negative evidence#
Automated fixtures fail for stale evidence, future timestamps beyond clock skew,
unresolved/wrong source commits, wrong contracts, tampered payloads, missing
generator/command/host/lifecycle fields, incomplete or causally reversed
execution/write lifecycle, and invalid builder inputs. Additional tests prove a
green artifact fails verification when its legacy validator rejects it, a red
artifact fails verification when its validator accepts it, and a red validator
must reject the recorded releaseGate.status rather than merely crash.
Honest remaining product state#
AUD-002 closes evidence freshness, not product readiness. The 58 current ISIS
exposure gates remain red for measured runtime/API/SDK/UI/E2E gaps, and the V3
launch gate remains 30/41 required commands and 24/39 named gates with
gaReady: false. No provider, engine, media, placement, screenshot, or human
review proof applies to this metadata-control work package. No policy in scope
requires a signed human release decision, and none is claimed.
Key implementation digests#
| File | SHA-256 |
|---|---|
scripts/evidence/generated-evidence-metadata.mjs |
29023a2ee29778f1be52490c562df3ba4ec0f6065360462c768a74cfa89aa94a |
scripts/evidence/generated-gate-runtime.mjs |
579fcafd1e4482089b4f19b7846617f7e0fe5c402ade0dd3c90aa0b6c8a7bae0 |
scripts/evidence/instrument-generated-gate-producers.mjs |
482b72de58154df981cdd70c9ed7150e3f7ec08bed90dc4dad6700a0d066cb29 |
scripts/evidence/regenerate-generated-gates.mjs |
6ad05232f87b65d328c57be7442725029e182d796cb614ea9b1692a1638c1bf0 |
scripts/evidence/verify-generated-gate-inventory.mjs |
7e769af164b99c37ec7bf4ddfb731eb4dc0e023accbc25ad0ec3e9b7af497c51 |
.github/workflows/generated-evidence-freshness.yml |
6b2a9c1e6caa4dc56270aa953ada723d60ba59e356c3dce5cf2492781bae94a5 |