# V3 Product Promise, Tiered Client Stack, and High-Level Architecture

V3 — "Lilith" — is Oshun's embodied, multi-user surface: one 3D space that
carries V1's identity, memory, grounding, safety, provenance, and rights
guarantees into a real-time world where people practice yoga together (Tara
Studio), attend concerts by persistent AI artists (Saraswati Stage), and gather
in shared civic rooms (Lilith Commons). The defining architectural choice is a
**tiered client stack** with Unreal Engine 5.5 as the canonical runtime and a
reduced-fidelity browser client as the floor, so that no device is locked out at
the door. That choice is not aspirational prose — it is wired through the
repository: the UE project at `V3/ue/V3.uproject` pins
`"EngineAssociation": "5.5"` and `"DisableEnginePluginsByDefault": true`,
declares **17 C++ modules** and a dedicated `V3PixelStreamingWorker.Target.cs`
server-render target, and enables PixelStreaming, GameplayAbilities, Niagara,
Metasound, Mover/PoseSearch, the Iris replication system, OpenXR, and EOS; the
Tier-2 client is a real Next.js app at `apps/v3/lilith-web-fallback/` rendering
through `FallbackThreeCanvas.tsx`; and three authoritative Rust services
(`apps/v3/lilith-world-server`, `lilith-realtime-gateway`,
`lilith-pxstream-relay`) share a single cross-language wire protocol. This page
is the orientation hub for how those pieces fit; the section index is
[../V3_ARCHITECTURE.md](../V3_ARCHITECTURE.md).

The reason the stack is built this way is economic and physical, not stylistic.
The premium experience — Lumen GI, Nanite geometry, MetaHumans, Sequencer-driven
concerts — can only run inside UE5, so a single UE project drives every premium
surface (native desktop/VR/console/mobile **and** the browser via
server-rendered Pixel Streaming over WebRTC). But server-rendered video needs a
GPU worker per session and a nearby POP, which is not always available; so a
second, locally-rendered three.js/WebGPU client exists for constrained devices,
constrained bandwidth, and regions without Pixel Streaming capacity. Both
clients speak the _same_ multiplayer wire protocol to the _same_ world server
and reuse the _same_ V1 BFF — the only thing that degrades across tiers is
rendered fidelity, never identity, presence, or safety. The launch decision that
routes a visitor to one of those surfaces is itself a typed contract
(`LilithLaunchDecisionSchema` in `libs/contracts/src/v3/lilith.ts`), with four
possible surfaces and twenty-one reason codes (a single decision carries up to
sixteen).

## What ships, honestly

The **wire protocol, the contract model, the three Rust services, and the UE C++
module skeleton are real and tested.** One Protobuf schema
(`libs/v3/multiplayer-protocol/proto/oshun/v3/multiplayer/v1/multiplayer.proto`)
is implemented three times — a ts-proto/TypeScript codec, a prost/Rust codec,
and a _hand-written_ UE C++ varint/zig-zag codec in
`V3/ue/Source/V3Net/Private/V3NetProtocol.cpp` — and a cross-language golden-hex
test (`SNAPSHOT_DELTA_GOLDEN_HEX` in
`libs/v3/multiplayer-protocol/rust/src/lib.rs`) proves the three encoders agree
byte-for-byte on the canonical fixture. The world server, gateway, and Pixel
Streaming relay are substantial Rust (consistent-hash sharding, an rstar R-tree
interest manager, rapier3d physics validation, JWT refresh-token rotation with
reuse detection, a KEDA/Karpenter-emitting autoscaler) under a
`unsafe_code = "forbid"` workspace, each with its own `main.rs` and an axum
`/healthz`. `libs/contracts/src/v3` ships **30 registered Zod contracts**
(`V3_CONTRACT_REGISTRY` in `registry.ts`) with validated fixtures.

Some surfaces are **modeled, not live.** The fleet-load and reconnect
"validation" reports in the Rust services compute deterministic phase-timing
_models_ (base latency plus a seeded jitter term — see
`simulate_transient_reconnect_validation` in the gateway), even though the
underlying reconnect/resume and shard-pin logic they exercise is real. The Pixel
Streaming relay is a genuine matchmaker, admission gate, and Epic-signaller
**relay** — `build_epic_signaller_answer_sdp` synthesizes a valid WebRTC answer
SDP with HMAC-derived ICE credentials — but the actual UE-rendered video frames
require a deployed GPU worker fleet that this repository plans for but does not
run. Finally, the heavy creative payload is **vendor/art-gated:** the
`V3.uproject` lists MetaHumanRuntime, MetaHumanSDK, LiveLinkFaceImporter, VRM4U,
ResonanceAudio, and SteamAudio as `"Optional": true` plugins, Wwise as disabled,
and the 129 tracked `.uasset` binaries are overwhelmingly _generated_
design-token UI assets (64 Styles, 40 Brushes, 3 Fonts under
`Content/UI/DesignTokens/Generated/`) plus the 21 `GameFeatureData.uasset`
plugin descriptors — there are no hand-authored MetaHuman, venue, or concert art
assets in-tree. Where a claim depends on those, this page says so.

## The product promise, concretely

Three tenants ride one substrate. The fixed vocabulary is enumerated in
`V3TenantSchema` (`libs/contracts/src/v3/primitives.ts`) as exactly four tenant
ids — `lilith-platform`, `lilith-commons`, `tara-studio`, `saraswati-stage` —
and each maps to a TypeScript adapter package (`@oshun/tenant-tara-studio`,
`@oshun/tenant-saraswati-stage`, `@oshun/tenant-lilith-commons`) and a set of UE
Game Feature plugins. The promise is that the substrate, not each tenant,
provides embodiment: a V1 Oshun account binds to an avatar
(`AvatarBindingSchema`, with a `swapCooldownUntil` enforcing the 24-hour swap
cooldown defined as `AVATAR_SWAP_COOLDOWN_MS` in the world server), presence is
authoritative and visibility-banded, and rights/provenance travel with every
captured artifact (`ProvenanceBundle3DSchema` extends the V1 provenance bundle
with `performanceId`, `venueId`, and a 64-hex `musicSyncDigest`).

## The tiered client stack

### Tier 1 — the canonical UE5 client (`V3/ue/`)

A single UE5.5 project targets every premium surface. Its 17 modules
(`V3.uproject`) split responsibilities cleanly: `V3Core` (subsystems, save-game,
the V1-account bridge), `V3World` (room replication, scene streaming, interest
management — it depends on `Json` and `V3UI` per
`V3/ue/Source/V3World/V3World.Build.cs`), `V3Net` (the wire-format codec),
`V3Avatar`/`V3Animation` (VRM + MetaHuman, motion matching via the Mover and
PoseSearch plugins), `V3Voice`/`V3Audio` (SFU client + MetaSounds), `V3VFX`
(Niagara), `V3Cinematics` (Sequencer), `V3OnlineServices` (EOS), plus `V3Editor`
(editor-only) and `V3Tests` (a DeveloperTool module for Gauntlet automation).
Across `V3/ue/Source` there are **73 `.cpp` and 40 `.h`** files. The project
enables the full premium plugin set — PixelStreaming(+Player), CommonUI +
ModelViewViewModel, GameplayAbilities, MotionWarping/AnimationWarping,
ChaosClothAsset, ReplicationGraph + NetworkPrediction + Iris,
OpenXR(+HandTracking), Gauntlet, and MovieRenderPipeline — and gates the
art/audio middleware behind `"Optional": true`. Crucially,
`V3PixelStreamingWorker.Target.cs` is a _separate build target_: the same
project compiles both the player-facing client and the headless server-render
worker that streams frames to browsers.

Modularity is delivered through **21 Game Feature plugins** under
`V3/ue/Plugins/GameFeatures/` — 3 tenant plugins (`V3Tenant_TaraStudio`,
`V3Tenant_SaraswatiStage`, `V3Tenant_LilithCommons`) and 18 mode plugins
(`V3Mode_TaraLiveClass`, `V3Mode_SaraswatiConcert`, `V3Mode_CommonsObservatory`,
…). Each ships a real `GameFeatureData.uasset` so it can be hot-activated per
room. See
[./subsystem-glossary-and-layout.md](./subsystem-glossary-and-layout.md) for the
full module/plugin map and [./tier1-ue5-client.md](./tier1-ue5-client.md) for
the client architecture.

### Tier 2 — the lightweight web fallback (`apps/v3/lilith-web-fallback/`)

The fallback is a real Next.js app, not a placeholder. Its root
`src/app/page.tsx` is a one-line `redirect('/commons')`; the actual experience
lives at `src/app/[tenant]/page.tsx`, which renders `FallbackExperience.tsx`
over a WebGPU/WebGL2 canvas (`FallbackThreeCanvas.tsx`). It renders locally at
reduced fidelity (selectable `webgpu`/`webgl2` renderer with per-renderer FPS
budgets from `lilithFallbackRendererBudgets`), connects to the same gateway via
`connectLilithGatewayWithFallback` (degrading WebTransport → WebRTC → WebSocket
by network profile), runs a rapier-based prop-pickup _prediction_ check, and
drives spatial audio through `@oshun/spatial-audio` (HRTF when available, flat
stereo when toggled off). Tenant features degrade gracefully rather than vanish:
Saraswati stadium concerts surface as a degraded preview tier
(`LilithSaraswatiFallbackTier` of `'hall'` etc.) with a Yemaya proxy URL, while
Tara classes and Commons rooms keep full interactivity. The same component also
composes the launch-accessibility reports (reduced motion, color-vision
palettes, photosensitive-safe mode, single-switch navigation, launch
localization) from `@oshun/tenant-lilith-commons`. See
[./tier2-fallback-web-client.md](./tier2-fallback-web-client.md).

### The browser-via-Pixel-Streaming entry (`apps/v3/lilith-web/`)

Between the two tiers sits the thin browser shell that _prefers_ Tier 1 by
streaming server-rendered UE frames. `LilithLanding.tsx` offers three launch
modes (Pixel Streaming, native deep-link, fallback WebGL), and on "Enter Lilith"
it `POST`s to `/api/v3/pxstream/match` and drives an
`@oshun/lilith-web-pxstream` client over `createEpicPixelStreamingAdapter()`,
reporting first-frame latency, mic permission, and reconnect state. This is the
"web via Pixel Streaming" surface — the browser sees full UE fidelity because a
GPU worker is rendering it remotely. Tier selection across native/pxstream/
fallback/static is the dedicated subject of
[./tier-routing-and-pixel-streaming.md](./tier-routing-and-pixel-streaming.md).

### One honest wrinkle: "tiers" mean different things at different layers

The word _tier_ is overloaded across the codebase, and the docs-center should be
precise about it. The **contract** `V3CapacityTierSchema` defines four venue
sizes — `class`, `salon`, `theater`, `stadium`. The **wire protocol**
`CapacityTier` enum collapses to two — `CLASS` and `STADIUM` — and the only
per-tier bandwidth budgets that exist are `CLASS_TIER_MAX_BPS = 32_000` and
`STADIUM_TIER_MAX_BPS = 256_000` (`multiplayer-protocol` crate). The **gateway**
introduces a third taxonomy, `VoiceRealmTier { Class, Hall, Stadium }`, for
voice policy. And the world server's interest manager caps visible entities at
`CLASS_TIER_VISIBLE_ENTITY_CAP = 32` vs `STADIUM_TIER_VISIBLE_ENTITY_CAP = 256`.
None of these contradict each other, but they are _not_ the same axis: contract
tiers describe a venue's authored size; wire/interest tiers describe the
bandwidth-and-LOD regime; voice tiers describe crowd-audio policy. Treat "tier"
as layer-scoped.

## The shared wire protocol — the spine that makes the stack coherent

Everything above only works because all clients speak one protocol. The schema
(`multiplayer.proto`, package `oshun.v3.multiplayer.v1`) quantizes everything to
integers for compactness and determinism: positions are `Vector3Mm` (signed
millimeters, zig-zag varint), rotations are `RotationMilliDegrees`, expression
intensity is basis points. It defines snapshot and delta packets, presence,
voice control, interaction, gameplay-action, and operator-control messages, plus
`ClientEnvelope`/`ServerEnvelope` with `oneof` payloads and a
`VersionNegotiation` handshake.

The protocol is implemented in all three runtimes from that one schema:

- **TypeScript** (`libs/v3/multiplayer-protocol/src/index.ts`) re-exports the
  generated codec and adds `applySnapshotDelta`, `predictSnapshot`
  (dead-reckoning clamped to 250 ms), `negotiateProtocolVersion`, and a
  `runDecoderBenchmark`.
- **Rust** (`.../rust/src/lib.rs`) uses prost (`wire` module) plus
  `encode_snapshot_delta`/`apply_snapshot_delta`, `validate_bandwidth_budget`,
  and `negotiate_protocol_version`.
- **UE C++** (`V3NetProtocol.cpp`) hand-rolls the varint/zig-zag wire encoding —
  `FV3NetMultiplayerProtocolCodec::EncodeSnapshotDeltaPacket` and the matching
  decoders — and `BuildInEngineProtocolExchangeReport()` round-trips the full
  packet set in-engine.

The interop guarantee is a _golden hex string_ asserted in the Rust tests:
`protobuf_snapshot_delta_matches_cross_language_golden_wire` checks the
canonical delta (room `lilith-commons-atrium`, base sequence 6, sequence 7, a
`calm-smile` expression at 7,300 bp) encodes to a fixed 131-byte hex blob — the
same fixture the TS `canonicalSnapshotDeltaFixture()` and the C++
`CanonicalSnapshotDeltaFixture()` build. Protocol negotiation selects the
highest mutually-supported version and otherwise returns `unsupported_version`.
See [./netcode-protocol-and-physics.md](./netcode-protocol-and-physics.md).

## The authoritative service plane (Rust)

Three services form a Cargo workspace (`apps/v3/Cargo.toml`, resolver 2, Rust
1.82, `unsafe_code = "forbid"`), each with a `SERVICE_DESCRIPTOR`, capability
list, and `main.rs`.

**Lilith World Server** (port 43101, `lilith-world-server/src/lib.rs`) is the
authority. It holds rooms in a `RoomRegistry`, routes them to shards with a
`ConsistentHashRing` (virtual nodes + stable hashing) and gossips cross-shard
chat over a `CrossShardEventBus`. Interest management is a real spatial query —
an `rstar` R-tree over participant positions, returning the nearest _N_ entities
capped per tier. Movement is validated against a
`TELEPORT_SPEED_LIMIT_MPS = 12.0` ceiling, and interactive objects use
server-validated pickup/place (`InteractionAuthority`). Presence visibility is a
five-band model (`VisibilityBand::{Public, Tenant, Cohort, Invited, Invisible}`)
computed against viewer profiles. The fixed-tick loop (`TickLoopConfig`)
defaults to 50 Hz simulation, 20 Hz transform broadcast, 60 Hz expression
interpolation, under a 5 ms p99 budget, and durable state splits across a
Postgres durable store and a Redis hot-state stream (`durable_persistence`
module). Packet processing emits OTel/Jaeger spans
(`process_join_packet_with_tracing`).

**Lilith Realtime Gateway** (port 43102, `lilith-realtime-gateway/src/lib.rs`)
is the edge. It terminates three transports with graceful fallback
(`WebClientNetworkProfile`: UDP → TURN-TCP → WebSocket), authenticates
handshakes with HS256 JWTs, and runs a full **refresh-token rotation store**
with reuse detection and family revocation (`rotate_refresh_token` revokes the
whole token family on replay; 15-minute access, 30-day refresh). It pins
sessions to shards (`SessionPinDirectory`, 60 s TTL) and resumes transient drops
while preserving shard pin and server sequence (`GatewayReconnectDirectory`).
Its Voice SFU is LiveKit-compatible, Opus 24 kbps mono / 20 ms frames, with HRTF
listener positioning and operator mute/kick; per-tenant voice policy is concrete
— `VoiceChatPolicyDirectory::lilith_defaults()` gives Tara direct voice with
instructor mute at every tier, and Saraswati disabled at class/hall but a
crowd-bed proxy at stadium.

**Lilith Pixel Streaming Relay** (port 43103,
`lilith-pxstream-relay/src/lib.rs`) is the browser-fidelity broker. It matches a
request to a POP by RTT, utilization, codec (H.264/AV1), and residency; issues a
short-lived session JWT (HMAC, from `V3_PXSTREAM_JWT_HMAC_SECRET`); brokers the
Epic signaller SDP exchange; and plans GPU capacity with a
`plan_pxstream_worker_scale` that emits real KEDA `ScaledObject` and Karpenter
`NodePool` YAML, plus scheduled-concert prewarm (15-minute asset lead, 30-minute
worker lead) and multi-cloud pools (AWS/Azure/GCP/CoreWeave/Lambda). Its
admission gate is a domain model, not a gate-stub: a `PxStreamAdmissionDecision`
with seven actions (allow/deny/fallback/
prompt-idle/disconnect-idle/refer-safety/ban) backed by an input-entropy bot
classifier (precision floor 90%), free-tier minutes (120/day), and per-user (2),
per-network (8), and institutional (64) session caps. See
[./world-server-and-gateway.md](./world-server-and-gateway.md).

## Data, contracts, and tenancy

The V3 data plane is contract-first. `libs/contracts/src/v3` defines primitives
(`V3UuidSchema`, `V3Vector3Schema`, `V3RoyaltySplitSchema` that must sum to
10,000 bps, `V3CitationRefSchema`) and four domain files (`lilith.ts`,
`tara.ts`, `saraswati.ts`, `commons.ts`) feeding a single `V3_CONTRACT_REGISTRY`
of **30 contracts** — 8 lilith-platform, 8 tara, 8 saraswati, 5 commons, 1
cross-cutting `EmbodiedConsent`. Each descriptor carries a Zod schema, a
validated fixture, a route segment, and a Prisma model name (`V3<ContractName>`)
pointing at one of three tenant Prisma schemas. Tenancy is enforced both in data
(databases are domain-isolated) and at the body — presence is filtered by
visibility band in the world server before broadcast, so a Commons "solitary
cell" participant is structurally invisible, not merely hidden in UI. Residency
and DSAR ride the V1 substrate. See
[./data-tenancy-and-residency.md](./data-tenancy-and-residency.md) and
[./tara-classes-aja-and-commons.md](./tara-classes-aja-and-commons.md).

## A worked example: cold-open to first frame

The launch decision is modeled as a contract (`LilithLaunchDecisionSchema`), and
the four-surface routing (`native`, `pxstream`, `fallback`, `static`) plays out
across the BFF, the relay, and the world server:

```mermaid
sequenceDiagram
    participant U as Browser
    participant BFF as V1 BFF
    participant R as PxStream Relay (43103)
    participant W as UE Worker (server-render)
    participant G as Realtime Gateway (43102)
    participant WS as World Server (43101)

    U->>BFF: GET catalog / identity / entitlement (HTTPS)
    Note over U: probe webRtc / codecs / webGpu / downlink
    U->>R: POST /api/v3/pxstream/match (region, codec, residency)
    R->>R: admission gate + POP scoring + session JWT
    R-->>U: match decision (POP, signaller WS, first-frame budget)
    U->>R: POST /signalling/exchange (SDP offer + ICE)
    R-->>U: synthesized answer SDP (HMAC-derived ICE creds)
    U-->>W: WebRTC media (server-rendered UE frames)
    U->>G: handshake (Bearer access JWT, room id)
    G->>G: verify JWT, pin session to shard (60s TTL)
    G->>WS: join_room(participant)
    WS-->>G: snapshot + visibility-filtered presence
    G-->>U: snapshot delta stream (20 Hz), voice SFU (Opus)
```

If the browser lacks WebRTC or no POP is within the latency budget, the same
decision contract routes to the `fallback-browser` target (WebGPU/WebGL2,
reduced fidelity); if even local rendering is unsupported, it routes to
`static-landing` with reason `local-rendering-unsupported`. Native clients skip
the relay and go straight to the gateway. Failure modes are first-class: the
relay returns typed errors (`PXSTREAM_NO_POP_AVAILABLE` → 503,
`PXSTREAM_SESSION_JWT_INVALID` → 401), the gateway rejects replayed refresh
tokens by revoking the token family, and the world server rejects teleport-speed
jumps and out-of-band interaction requests.

## How it connects to neighbouring systems

V3 layers on V1 without forking it: clients hit the V1 BFF over HTTPS for
catalog, identity, billing, and scheduling, while the world plane handles
real-time state. Cross-domain extensions reuse V1 packages — `@oshun/aja-pose`
for in-world pose coaching, `@oshun/sophia-saraswati-grounding` for artist
backstory citations, Aje for tickets and royalties, Iris for spatial memory
scopes. The neighbouring docs-center pages cover each seam in depth:
[./avatar-animation-and-audio.md](./avatar-animation-and-audio.md),
[./saraswati-stage-pipeline.md](./saraswati-stage-pipeline.md),
[./authoring-and-content-pipeline.md](./authoring-and-content-pipeline.md),
[./v1-integration-and-identity-bridge.md](./v1-integration-and-identity-bridge.md),
[./persona-policy-provenance-and-rights.md](./persona-policy-provenance-and-rights.md),
[./commerce-and-royalties.md](./commerce-and-royalties.md), and
[./observability-performance-security-and-launch.md](./observability-performance-security-and-launch.md).

## Related

- Hub: [../V3_ARCHITECTURE.md](../V3_ARCHITECTURE.md)
- [Subsystem Glossary and Layout](./subsystem-glossary-and-layout.md) — the full
  module, plugin, and package map
- [Tier Routing and Pixel Streaming](./tier-routing-and-pixel-streaming.md) —
  the four-surface launch decision in detail
- [Tier-1 UE5 Client](./tier1-ue5-client.md) and
  [Tier-2 Fallback Web Client](./tier2-fallback-web-client.md)
- [World Server and Gateway](./world-server-and-gateway.md) and
  [Netcode, Protocol, and Physics](./netcode-protocol-and-physics.md)