Gate owner: Release Captain + Product Lead. Operational runbook:
docs/releases/v1/runbooks/beta-program.md. Required for §33.1.
The private beta is the first external validation of OSHUN V1 with real
users outside the dogfood cohort. It runs after — never before — the 30-day
dogfood drill (docs/launch/dogfood-drill.md), safety review
(docs/launch/safety-review.md), and quality review
(docs/launch/quality-review.md) have produced passing signoffs.
The beta-program runbook covers cohort selection, test scripts, recruitment, feedback collection, and TestFlight / Play Console distribution. This file adds the measurable target metrics the beta must hit before GA.
1. Duration#
- Calendar minimum: 21 consecutive days.
- Calendar maximum: 42 consecutive days before the gate auto-extends via Quality Council re-review.
2. Cohort sizes (floor)#
| Cohort | Floor | Reason |
|---|---|---|
| Internal (engineering + PM + design + QA + leadership) | ≥ 23 | Statistical power for crash + retention |
| External — domain power users | ≥ 20 | Stress core journeys per domain |
| External — new users | ≥ 10 | Validate onboarding flow |
| External — accessibility (assistive tech) | ≥ 5 | Validate accessibility gate |
| External — low-connectivity | ≥ 5 | Validate offline + retry paths |
A cohort below floor invalidates that cohort's measurements. Refilling restarts the trailing-7-day window for affected metrics.
3. Measurable target metrics (gate floors)#
All targets are gate floors. The beta does not pass §33.1 if any single metric is below floor over the trailing 7 consecutive days at gate close.
3.1 Reliability#
| Metric | Floor |
|---|---|
| Crash-free sessions (iOS) | ≥ 99.5 % |
| Crash-free sessions (Android) | ≥ 99.5 % |
| ANR rate (Android) | ≤ 0.2 % |
| Web JS-error rate (per session) | ≤ 0.5 % |
| Service-health API uptime (mirror) | ≥ 99.9 % |
| API p95 latency | ≤ 400 ms |
3.2 Engagement and retention#
| Metric | Floor |
|---|---|
| Onboarding completion rate | ≥ 85 % |
| D1 retention (returning the next day) | ≥ 55 % |
| D7 retention | ≥ 40 % |
| Median sessions / week / active user | ≥ 3 |
| % of users touching ≥ 2 domains in week 1 | ≥ 50 % |
| % of users completing a Tara ritual in week 1 | ≥ 60 % |
3.3 Satisfaction#
| Metric | Floor |
|---|---|
| External cohort NPS (Day-14 survey) | ≥ 30 |
| In-app rating prompt CSAT mean | ≥ 4.2 / 5 |
| Support contact rate per active user / week | ≤ 0.04 |
3.4 Quality and grounding#
| Metric | Floor |
|---|---|
| Assistant grounded-answer rate | ≥ 95 % |
| Veritas claim-source bind rate | ≥ 99 % |
| Metis grounded-course pass-through | 100 % |
| Sophia evidence available wherever cited | 100 % |
| Persona-tone drift | ≤ 1 σ from envelope |
| Crisis-flow safe-route hit rate | 1.00 |
3.5 Safety, privacy, integrity#
| Metric | Floor |
|---|---|
| Open S0 / S1 defects at gate close | 0 |
| DSAR export round-trip success | 100 % |
| DSAR delete + restore-blocked verification | 100 % |
| Cross-tenant data leakage incidents | 0 |
| Moderation queue p95 dwell | ≤ 4 h |
| Persona-rollback drill success | 100 % |
| Watermark offline-verify rate | ≥ 99.9 % |
3.6 Performance and resource#
| Metric | Floor |
|---|---|
| Tara cold launch (cold tap → playable) | ≤ 3.0 s p95 |
| Domain switch (warm) | ≤ 700 ms p95 |
| Mobile RSS at 30 min steady-state | ≤ 300 MB |
| 30-min battery drain | ≤ 5 % |
| Lighthouse Performance (all customer routes) | ≥ 85 |
| Lighthouse Accessibility | ≥ 95 |
| Lighthouse Best Practices | ≥ 95 |
| Lighthouse SEO (public routes) | ≥ 90 |
4. Critical-journey participation gate#
Each external cohort participant must complete at least one full pass of
these journeys before gate close. Tracked via journal entries
(docs/releases/v1/verification/beta/journeys/<participant-id>.md):
- Tara ritual (start → complete → reflection).
- Arete daily check-in.
- Veritas briefing read with at least one source drill.
- Nyx tonight view.
- Nisaba passage read with one annotation.
- Metis BYOM ingest of one personal artifact + one grounded course session.
- Assistant grounded answer + memory update + privacy pause toggle.
Participation < 80 % invalidates the journey-coverage axis of the gate.
5. Telemetry and observability#
The beta cluster ships the same dashboards as production. Per-cohort slices required:
- iOS vs. Android vs. Web.
- Internal vs. External.
- New users vs. power users.
- Accessibility cohort vs. baseline.
- Low-connectivity cohort vs. baseline.
Any sustained (≥ 48 h) regression beyond the floors in §3 triggers a beta
freeze — see docs/launch/beta-blocking-remediation.md.
6. Exit criteria#
Beta clears §33.1 when all are true:
- 21+ days elapsed.
- Every cohort meets floor in §2.
- Every metric in §3 within floor on trailing 7 days.
- Critical-journey participation (§4) ≥ 80 % per external cohort.
- Zero open S0, zero open S1 defects.
- Beta-blocking-issue remediation
(
docs/launch/beta-blocking-remediation.md) reportsclear. - Release captain, product lead, support lead, security lead, privacy lead,
and engineering lead post a written signoff to
docs/releases/v1/verification/beta/signoffs/<role>-<date>.md.
7. Re-opening#
The private beta is re-opened automatically if any of the following occur between beta close and GA:
- New S0 / S1 defect.
- Trailing 7-day rolling mean of any §3 metric drops below floor.
- Crisis-flow incident in production-mirror.
- Cross-tenant data exposure detected.
A re-open extends the gate by 14 days minimum.
8. Cross-references#
- Operational runbook:
docs/releases/v1/runbooks/beta-program.md. - Dogfood drill:
docs/launch/dogfood-drill.md. - Safety review:
docs/launch/safety-review.md. - Quality review:
docs/launch/quality-review.md. - Beta-blocking remediation:
docs/launch/beta-blocking-remediation.md. - Go/no-go:
docs/launch/go-no-go.md.