Disciplines · Launch

Internal Dogfood Drill — V1 Launch Gate

before the targeted GA date.

8sections5 minread

On this page

Gate owner: Release Captain. Source-of-truth: this file. Status: template + procedure. Each drill cycle produces a dated evidence snapshot at docs/releases/v1/verification/dogfood/YYYY-MM-DD/.

The dogfood drill is the 30-day pre-GA window during which the OSHUN core team exercises every launch-critical customer and admin journey on production-grade infrastructure with production-shaped data. The drill is the last gate before private beta in §33.1.

1. Scope and duration#

  • Duration: ≥ 30 consecutive calendar days ending no later than 14 days before the targeted GA date.
  • Build cadence: nightly canary build promoted from main after CI green; release-candidate build cut every Monday.
  • Infrastructure: production-equivalent cluster (oshun-stage-prod-mirror) with production-shaped fixtures from testing/fixtures/ and tests/integration/fixtures/, plus the per-domain seed-data inventory in docs/content/v1-bundles.md#seed-data-inventory.
  • Cohort: every named OSHUN engineer, PM, designer, content lead, support operator, moderator, reviewer, persona operator, model operator, privacy operator, compliance operator (~80 participants).

A drill that runs for less than 30 days, or skips any required journey, does not satisfy this gate and the release captain must re-open it.

2. Required journey coverage#

Every participant runs at least one customer journey and one admin journey per day, logged via the dogfood telemetry channel (#dogfood-evidence in Slack, mirrored to dated journal files under docs/releases/v1/verification/dogfood/<date>/journal.md). Coverage matrix:

2.1 Customer journeys (every room this release ships + cross-domain)#

V1.0 scope. Rows for Veritas and Metis are struck: those rooms are deferred to V1.2, so nobody can run their journeys and a required-hit target against them would only ever be met by fabricated evidence. They return to this matrix in the release that opens them. Mobile rows are V1.1.

Journey Required hits / 30 days Owner
Tara: ritual start → pause → resume → complete → reflection ≥ 200 Tara PM
Arete: daily check-in → habit complete → weekly review ≥ 200 Arete PM
Veritas: briefing → claim inspect → source drill → counterclaim (V1.2) Veritas PM
Nyx: tonight view → save event → calendar sync → reminder fires ≥ 100 Nyx PM
Nisaba: passage open → annotate → notebook capture → resume in the PWA ≥ 150 Nisaba PM
Metis: BYOM ingest → grounded course → tutor session → assessment (V1.2) Metis PM
Assistant: persona switch → grounded answer → memory update → privacy pause ≥ 200 Iris PM
Search + library + activity + memory + privacy + billing + offline ≥ 100 each Shell PM
PWA install + update + offline ≥ 50 each Shell PM
Mobile push (V1.1, with the native apps) Mobile PM

2.2 Admin journeys (all critical operator workflows)#

Journey Required hits / 30 days Owner
Queue triage → review approve → publish gate → audit trail ≥ 100 Trust & Safety lead
Kill-switch fire → fallback recovery ≥ 10 SRE lead
Persona promotion + rollback ≥ 5 Persona ops lead
Model promotion + rollback ≥ 5 Model ops lead
Incident open → mitigate → post-incident review ≥ 10 On-call captain
Privacy: DSAR end-to-end (export + delete + restore-blocked + audit) ≥ 10 Privacy ops lead
Exports, policy edits, research-integrity reviews, dense-data workspaces ≥ 20 each Research ops
Tenant: SSO + SCIM + OneRoster + custom role + audit + export ≥ 5 each Tenant ops

Hits are counted by structured journal entries containing journey:, outcome: (pass / defect / blocked), defect_id: (if applicable), and evidence_url:. The evidence_url MUST resolve to a screenshot, HAR, trace, or recording stored under docs/releases/v1/verification/dogfood/.

3. Daily cadence#

  • 08:00 PT — release captain posts the day's focus journeys.
  • Continuous — participants log journal entries as journeys complete.
  • 17:00 PT — release captain triages new defects, assigns owners.
  • 17:30 PT — auto-report posted to #dogfood-evidence summarising:
    • Hits per journey vs. plan.
    • New defects opened by severity.
    • Defects resolved by severity.
    • SLO compliance for production-mirror dashboards (see §5).

4. Defect handling#

Severity definitions match docs/runbooks/oshun-v1.md §4. Required SLAs:

Severity Triage SLA Fix SLA Beta gate
S0 (crash on launch, data loss, security, cross-tenant leak) 30 min 24 h Zero open S0
S1 (core journey broken, grounding gate failure, kill-switch failure) 4 h 5 d Zero open S1
S2 (degraded feature, performance regression > 20 %) 1 day 14 d ≤ 3 open S2
S3 (cosmetic, minor friction) 3 days Best effort Tracked, not gating

Every S0 / S1 defect closure requires:

  • Repro evidence (HAR / trace / recording).
  • Fix commit linked.
  • Regression test added (Playwright / Vitest / contract).
  • Re-run of the failing journey by a different participant.

5. SLO mirror#

For the 30-day window the production-mirror cluster runs the same post-deploy monitoring stack as production:

Dashboard Threshold Source
dashboards-service-health.ts 99.9 % API uptime, p95 < 400 ms libs/oshun/analytics/src/dashboards-service-health.ts
dashboards-customer-kpi.ts D1 retention ≥ 60 %, crash-free ≥ 99.5 % libs/oshun/analytics/src/dashboards-customer-kpi.ts
dashboards-assistant.ts Grounded-answer rate ≥ 95 % libs/oshun/analytics/src/dashboards-assistant.ts
dashboards-queue-health.ts Review queue p95 dwell ≤ 4 h libs/oshun/analytics/src/dashboards-queue-health.ts
dashboards-nisaba.ts Nisaba grounding-source bind rate ≥ 90 % libs/oshun/analytics/src/dashboards-nisaba.ts
dashboards-satisfaction.ts Internal NPS ≥ 40 over rolling 7 d libs/oshun/analytics/src/dashboards-satisfaction.ts

Any dashboard outside threshold for > 2 consecutive days triggers a drill freeze: no new feature merges until the regression is closed.

6. Exit criteria (must all be true to advance to private beta)#

  • 30 calendar days elapsed without an unrecovered drill freeze.
  • Coverage matrix in §2 met or exceeded.
  • Zero open S0, zero open S1, ≤ 3 open S2.
  • All dashboards in §5 within threshold for last 7 consecutive days.
  • Privacy DSAR drill (export, delete, restore-blocked) completed end-to-end with audit-log evidence.
  • Kill-switch fire + fallback recovery drill completed at least twice.
  • Backup-restore drill completed at least once with RTO ≤ 1 h, RPO ≤ 15 min.
  • Release captain, security lead, privacy lead, product lead, support lead, and engineering lead each post lgtm in #dogfood-evidence with a link to the evidence snapshot.

7. Evidence snapshot#

At drill close, release captain commits:

text
docs/releases/v1/verification/dogfood/YYYY-MM-DD/
├── coverage-matrix.csv          # one row per journey, hit count, owner
├── defect-ledger.csv            # all defects with severity, status, fix commit
├── slo-report.md                # weekly dashboard rollup
├── kill-switch-drill.md         # narrative of fire + recovery
├── dsar-drill.md                # narrative of export+delete+restore-blocked
├── backup-restore-drill.md      # narrative of RTO/RPO measurement
└── signoffs.md                  # six lgtm comments with timestamps

Without this snapshot present in the repo at drill close, §33.1 dogfood remains unsatisfied regardless of any other state.

8. Cross-references#

  • Beta program (week 31-day private-beta gate): docs/launch/private-beta.md.
  • Safety review pass (Lilith + T&S + Privacy board): docs/launch/safety-review.md.
  • Quality review pass (content + AI eval thresholds): docs/launch/quality-review.md.
  • Beta-blocking-issue remediation: docs/launch/beta-blocking-remediation.md.
  • Go/no-go: docs/launch/go-no-go.md.
  • Rollback plan: docs/reference/rollback-procedures.md.