V1 Capacity Model — Living Scenes, Psyche, BFF

Status: planning baseline, created 2026-06-12 to close V1_V7_PLAN_SET_AUDIT_2026-06-12.md §6.2 (V1: "no capacity/load numbers for GPU-bound Living Scenes"). All spec budgets cited below are the product's own committed numbers; everything labeled planning assumption adopted 2026-06-12 is replaced by measurement during beta (§8).

1. Spec inputs this model is derived from#

2. Launch concurrency assumptions (explicit)#

All rows are planning assumptions adopted 2026-06-12 with derivations; they define the design point the fleet is sized for (≈ GA+90 days), plus the smaller GA-week point used for initial procurement.

3. Living Scenes GPU fleet sizing#

3.1 SKU selection — the spec budget forces it#

The segment budget (p50 ≤ 6 s for a 4–12 s segment) requires a distilled few-step Hunyuan/WAN-class video model at 480–720p. Relative throughput (planning assumption adopted 2026-06-12; FP8/TensorRT-class inference, H100 ≈ 2–2.5× A100 on video-diffusion workloads):

This also serves the determinism release gate (V1/features.md:4193): one SKU plus one driver/CUDA/container digest per workflow class (RISK_REGISTER R-08).

3.2 Renders-per-GPU-hour math (from the spec's own budget)#

• Segment renders/GPU-hour at p50 budget: 3600 s ÷ 6 s = 600 renders/GPU-hr.
• Sustainable effective throughput at 75 % target utilization (scheduling gaps, model load, p95 tail): 600 × 0.75 = 450 renders/GPU-hr.
• Mean segment duration: 9 s (mid-band of the 4–12 s spec range; planning assumption adopted 2026-06-12).
• Per-session GPU occupancy = gen time ÷ playback time = 6 ÷ 9 = 0.67 GPU steady-state at p50 (p95 worst case 12 ÷ 9 = 1.33 — absorbed by the ≥ 2-segment pre-warm lookahead, not by fleet overprovisioning).

3.3 Cache and replay offset#

30 % of session-seconds need no live generation (planning assumption adopted 2026-06-12). Two sources: Nyx briefings materialize once per location-bucket (V1/features.md:4072-4074) — ≈ 20 % of sessions × 80 % cache hit — and replays of kept artifacts are served from the 30-day MP4 cache (V1/features.md:4196-4199) — ≈ 15 % of sessions fully cached.

3.4 Fleet formula and numbers#

fleet = sessions × occupancy × (1 − cache_offset) ÷ utilization + warm_pool

Warm pool / first-frame budget: The p50 ≤ 400 ms first-frame budget cannot be met by cold generation. Mechanism: (a) the opening segment of every template variant is pre-rendered and cached (Tara breath-cycle anchors, Nyx celestial anchors, etc., V1/features.md:3797-3802), so first frame is a cache read + stream start; (b) a warm idle-worker pool of max(8, 5 % of fleet) absorbs session-start bursts so the second (first personalized) segment begins generation immediately.

Procurement split (feeds RISK_REGISTER R-01):

• 60 % RunPod Secure Cloud committed (78 of 130 at design point),
• 40 % RunPod on-demand,
• plus 20 % of peak (26 H100) warm at AWS EC2 P5 as the cross-provider failover reserve, same image digest, registered as drained in the Conductor worker pool.

3.5 Scaling policy#

Diurnal: scale-in floor = 30 % of peak fleet (model-load latency makes cold-adds slow); scale-out signal = queue depth (§6) and projected wait; predictive pre-scale 30 min ahead of the historical evening peak per region.

4. Psyche voice/avatar capacity#

4.1 Per-node session capacity#

Psyche runtime node = c7i.2xlarge-class (8 vCPU / 16 GB) — planning assumption adopted 2026-06-12. Per active voice session the node carries Opus encode/decode, VAD, event fan-out, and transcript sync ≈ 25 ms CPU per second of audio ≈ 2.5 % vCPU (planning assumption; derived from public Opus/WebRTC SFU benchmarks at 48 kHz mono). At a 60 % CPU target:

• Voice sessions/node: 8 vCPU × 60 % ÷ 0.025 ≈ 190 → plan 150/node (jitter + reconnect-storm headroom).
• Avatar sessions/node: viseme + expression streams roughly double per-session event throughput (V1/features.md:2202-2215) → 75/node.

4.2 Node and vendor sizing at design point#

4.3 Budget decomposition constraint#

The voice budget only closes with a fully streamed pipeline: ASR final ≤ 150 ms (V1/features.md:2225) + LLM first token ≤ 350 ms (:2223) already sums to 500 ms — equal to the entire p50 end-to-end budget (:2220). Therefore: ASR finalization must overlap LLM prefill, and TTS synthesis must start on the first clause with a flash-tier voice model (first-chunk ≈ 75–100 ms, planning assumption adopted 2026-06-12). Any provider or config that serializes these stages breaches p50 by construction — load tests assert overlap (§7 LT-1).

5. Unit economics (feeds RISK_REGISTER R-04)#

GPU price: H100 80GB ≈ $2.99/GPU-hr RunPod Secure Cloud (planning assumption adopted 2026-06-12; mid-point of recent published secure-cloud rates; AWS P5 failover capacity runs 2–3× this, which is why it is reserve, not primary).

Kill criterion and per-user daily budgets are defined in RISK_REGISTER R-04.

6. BFF capacity and queue-depth thresholds#

6.1 BFF (Fastify, apps/oshun/bff, single BFF per V1/ARCHITECTURE.md:292)#

Per-client rate limiting at the BFF is already spec'd (V1/ARCHITECTURE.md:1069-1071); overload shedding order in RISK_REGISTER R-07.

6.2 Segment-render queue thresholds#

Let C = current live fleet size (one in-flight segment per GPU). Projected wait = depth × p50 gen time ÷ C.

Crisis frames are never shed or queued in any state — the crisis path (V1/features.md:3701-3704) has reserved capacity (2 GPUs held out of the admission pool).

6.3 Voice admission threshold#

If ElevenLabs concurrency utilization > 85 % of contract, new sessions start in text mode with a voice-upgrade offer (matches the spec's recovery behavior, V1/features.md:2245-2246) rather than risking mid-session budget breaches.

7. Load-test plan#

Tooling: k6 (HTTP + WebSocket scenarios) for BFF and Psyche envelope; Conductor replay harness driving the per-template fixture Scores (≥ 30 per template, V1/features.md:4466-4469) at controlled arrival rates against the real GPU fleet; Prometheus/Grafana (observability profile, V1/DEPENDENCIES.md §21) for measurement; fault injection at the provider adapters for chaos legs. Test types map to the named V1 quality matrix — load, spike, soak, memory-leak, performance, queue-throughput, failure, recovery, chaos, disaster-recovery (V1/features.md:6171-6173).

Gate linkage: LT-1 through LT-6 passing is an entry criterion for private beta wave 3 and for the canary ramp (LAUNCH_TIMELINE Phases 2–3). LT-7 is a GA go/no-go input.

8. Re-forecast cadence#

• Weekly during beta: observed attach rates, session lengths, cache-hit rates, and measured renders/GPU-hr replace the §2/§3 assumptions; fleet plan re-derived from the same formulas.
• At every canary promotion: capacity review signed by SRE Lead (the promotion criteria in LAUNCH_TIMELINE require headroom ≥ 2× the current step's traffic).
• Owners: SRE Lead (model), Generation Platform Lead (GPU inputs), Realtime Lead (Psyche inputs), Finance Lead (unit economics).