# Lilith World Server and Realtime Gateway

The world server and the realtime gateway are the two halves of V3's
authoritative multiplayer backbone: the server that _owns_ the truth of every
room (who is where, what they touched, who may hear whom) and the edge that
_terminates_ client connections and routes their packets to the right server
shard. They exist because V3 deliberately refuses to let Unreal Engine's own
replication stack own room state — a single Rust authority has to speak to four
very different client surfaces (native UE5, a Pixel Streaming worker, a fallback
WebGPU browser client, and an operator console) over one identical wire format,
so the truth can never fork per-tier. The world server lives at
`apps/v3/lilith-world-server/src/lib.rs` (an ~215 KB axum/Tokio crate with 28
`#[test]`/`#[tokio::test]` cases plus a live-infra integration test); the
gateway lives at `apps/v3/lilith-realtime-gateway/src/lib.rs` (~123 KB, 24
tests, real `quinn` QUIC). Both consume the same Protobuf contract published as
`@oshun/multiplayer-protocol` (`libs/v3/multiplayer-protocol/`), and the Unreal
client re-implements that exact wire format byte-for-byte in C++.

This page is the deep dive behind the "Realtime Backbone" summary in the
orientation hub [../V3_ARCHITECTURE.md](../V3_ARCHITECTURE.md). It is the
networking companion to
[Netcode Protocol and Physics](./netcode-protocol-and-physics.md) (which goes
deeper on prediction and reconciliation), and it sits directly downstream of
[Tier Routing and Pixel Streaming](./tier-routing-and-pixel-streaming.md) — once
the router has decided _which_ surface a visitor gets, the gateway is the door
they actually knock on. Read it alongside
[Tier 1 UE5 Client](./tier1-ue5-client.md) and
[Tier 2 Fallback Web Client](./tier2-fallback-web-client.md), which are the two
protocol _peers_ the backbone serves.

## What ships, honestly

The **wire protocol, the world-server simulation library, and the gateway's
transport/auth/voice library are real and tested** — but the daemons that wrap
them are not yet the full real-time servers the monolith describes, and the
Unreal integration is not what the monolith says it is. Three honest layers:

- **Real and exercised by tests.** The three-language Protobuf wire format
  (Rust, TypeScript, and hand-rolled UE C++) is asserted _byte-identical_
  against a shared golden hex fixture. The world server's authoritative core —
  the fixed-tick snapshot loop, `rstar` R-tree interest management, `rapier3d`
  physics anti-cheat, asana-lock and physical-adjustment consent gating,
  consistent-hash sharding, 60 s reconnect retention, and **real Postgres +
  Redis durable persistence with a live crash-recovery round-trip** — are all
  genuine, deterministic, and covered. The gateway's **real QUIC echo
  round-trip** (`quinn`), HS256 JWT verification with one-time refresh rotation,
  the WebTransport→WebRTC→WebSocket fallback ladder, and the LiveKit-equivalent
  SFU fan-out/HRTF model are all real code with real tests.
- **Scaffolded, not wired into a running server.** Critically, neither binary
  actually runs the simulation in production form. `run_service()` for the world
  server (`lib.rs:4564`) binds an axum router that exposes only `/healthz`,
  `/readyz`, and `/metrics` (`build_router`, `lib.rs:4516`); the gateway's
  `run_service()` (`lib.rs:2277`) serves _only_ a health body. There is no
  accept loop feeding live client packets into a ticking `RoomRegistry`, no QUIC
  listener bound to the session router, and no live world-server↔gateway socket
  path. The rich logic is library-grade and test-driven; the assembled,
  client-serving daemon is not here yet.
- **Aspirational in the monolith / corrected here.** The monolith's
  `V3_ARCHITECTURE.md` describes a `UV3NetDriver` (a `UNetDriver` subclass), a
  `cxx`-bridge `V3NetTransport`, and a `UV3WorldSubsystem` owning a local mirror
  table. **None of those exist in `V3/ue/Source`.** What exists is a real
  protobuf _codec_ (`FV3NetMultiplayerProtocolCodec`) and a deterministic voice
  _probe_ — described honestly below. The monolith's "zstd-1 on the delta
  packet" is also not implemented (there is no `zstd` anywhere in the V3 Rust
  tree); compression is delta-encoding plus Protobuf varints only. And
  "cross-shard chat via the V1 event bus" is, in code, an in-process `Vec`.

## One packet shape for every tier

Everything rests on a single Protobuf schema:
`libs/v3/multiplayer-protocol/proto/oshun/v3/multiplayer/v1/multiplayer.proto`.
It is genuinely engine-agnostic. Positions are integer millimetres (`Vector3Mm`
with zig-zag `sint32` fields), rotations are integer milli-degrees
(`RotationMilliDegrees`), and a per-avatar `AvatarExpression` carries an
expression id plus an intensity in basis points — so the same bytes describe a
meditator's micro-smile whether they are rendered by Lumen or by three.js.
Client→server traffic is a `ClientEnvelope` `oneof` (`negotiation_request`,
`presence`, `voice_control`, `interaction`, `gameplay_action`,
`operator_control`); server→client is a `ServerEnvelope` `oneof` that adds
`snapshot` and `snapshot_delta`. Capacity is a first-class enum
(`CAPACITY_TIER_CLASS`, `CAPACITY_TIER_STADIUM`).

Delta encoding is real and shared. `encode_snapshot_delta`
(`libs/v3/multiplayer-protocol/rust/src/lib.rs:85`) diffs two snapshots through
a `BTreeMap`/`BTreeSet`, emitting only `upserted_transforms` (entities that are
new or whose transform changed) and `removed_entity_ids`; `apply_snapshot_delta`
(`:130`) reconstructs the full snapshot on the receiver. Bandwidth is then gated
against a per-tier budget: `validate_bandwidth_budget` (`:169`) measures average
bits-per-second against `CLASS_TIER_MAX_BPS = 32_000` or
`STADIUM_TIER_MAX_BPS = 256_000` (`:10`–`:11`). The crate's own test
`class_tier_snapshot_deltas_stay_under_32_kbps_for_256_participants` (`:461`)
proves 256 moving avatars fit the class budget on deltas alone — which is _why_
zstd was never needed in practice.

The contract has three coherent implementations, and a cross-language golden
proves they agree:

| Layer      | Where                                          | Role                                                                                                                                 |
| ---------- | ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| Rust       | `rust/src/lib.rs` (prost)                      | World server + gateway + durable store                                                                                               |
| TypeScript | `src/index.ts` (`@oshun/multiplayer-protocol`) | Tier-2 fallback client; `encodeSnapshotDelta`, `applySnapshotDelta`, `predictSnapshot` (`:193`), `negotiateProtocolVersion` (`:206`) |
| UE C++     | `V3/ue/Source/V3Net/Private/V3NetProtocol.cpp` | Tier-1 codec, hand-written varint/zig-zag                                                                                            |

The Rust test `protobuf_snapshot_delta_matches_cross_language_golden_wire`
(`rust/src/lib.rs:417`) and the UE automation test `V3NetProtocolTests.cpp` both
assert equality against the _same_ hex string (`0a0a0803220676332e302e30…`). The
UE test is explicit about its intent — its case is titled _"Snapshot delta
matches Rust and Tier-2 golden wire."_ That shared fixture is the contract's
real teeth: a drift in any one language's encoder breaks a committed test.

## Lilith World Server — the authority

### The authoritative tick

The simulation clock is `FixedTickLoop` (`lib.rs:2532`). Its `tick()` (`:2558`)
runs an accumulator: each tick it adds `transform_broadcast_hz` and, when the
accumulator crosses `tick_hz`, increments the snapshot sequence and produces a
`SnapshotBroadcastPlan`. With the canonical config (50 Hz internal tick, 20 Hz
transform broadcast) the test
`fixed_tick_loop_emits_twenty_transform_snapshots_per_second` (`:5607`) confirms
exactly twenty snapshots per simulated second. `snapshot_broadcast_plan`
(`:2577`) walks every active room, builds a `SnapshotPacket` from each
participant's `EntityTransform`, and accounts encoded bytes through the same
`encode_wire_packet` the wire crate uses — so the bandwidth figure the server
plans against is the literal on-wire size, not an estimate. Expression is
carried separately at a higher interpolation rate
(`expression_interpolation_hz`) for lip-sync continuity between transform
frames.

### Interest management with an R-tree

A viewer never receives every entity. `InterestManager::visible_entities`
(`lib.rs:1310`) bulk-loads all other participants' world-space points into an
`rstar::RTree`, then takes the `cap` nearest neighbours of the viewer, where
`cap` comes from `visible_entity_cap` (`:2523`):
`CLASS_TIER_VISIBLE_ENTITY_CAP = 32` or `STADIUM_TIER_VISIBLE_ENTITY_CAP = 256`
(`:506`–`:507`). The result is distance-sorted with a stable id tiebreak. Two
load tests pin the caps:
`interest_manager_caps_class_visibility_at_32_nearest_entities` (`:5734`) and
`stadium_interest_load_test_confirms_256_entity_cap` (`:5783`). This is a
genuine spatial index, not a "return the first N" stub — the nearest-N guarantee
is what keeps a stadium-tier viewer's snapshot bounded while the room holds
thousands.

### Physics authority and anti-cheat

`PhysicsAuthority` (`lib.rs:2414`) holds a real `rapier3d` `RigidBodySet` and
`ColliderSet`; each participant is registered as a dynamic body with a humanoid
capsule collider (`capsule_y(0.9, 0.25)`). The anti-cheat gate is
`validate_state_transition` (`:2483`): it computes metres-per-second from the
positional delta over `elapsed_ms` and rejects anything above
`TELEPORT_SPEED_LIMIT_MPS = 12.0` (`:1353`) with a
`PhysicsValidationError::TeleportJump` carrying the offending speed. Two further
authorities enforce V3's bodily-safety promises rather than mere movement:
`AsanaLockAuthority` (`:1795`) refuses to lock an avatar into a guided posture
without consent, and `PhysicalAdjustmentConsentAuthority` (`:2161`) gates
teacher-initiated adjustments behind an explicit, audited consent dialog
(`PhysicalAdjustmentConsentDialog`, with an audit-event trail). These are the
contractual backbone behind Tara live classes — see
[Tara, Classes, Aja, and the Commons](./tara-classes-aja-and-commons.md).

### Sharding, reconnect, and cross-shard chat

Rooms above a tier are distributed by a `ConsistentHashRing` (`lib.rs:1078`)
over `ShardNode`s with virtual replicas; `ShardedRoomDirectory` (`:1160`) routes
a room to a shard and `spawn_routed_room` places it. Reconnect tolerance is
real: `retain_session_for_reconnect` (`:3819`) parks a departing participant for
`RECONNECT_SESSION_RETENTION_MS = 60_000` (`:508`), `resume_retained_session`
(`:3839`) rejoins them if they return inside the window, and
`purge_expired_retained_sessions` (`:3877`) reaps the rest. One honest
correction: `CrossShardEventBus` (`:1121`) is an **in-process** structure (a
`Vec<CrossShardChatEvent>` filtered by target room), not an integration with
V1's event bus as the monolith implies — see
[V1 Integration and Identity Bridge](./v1-integration-and-identity-bridge.md)
for what the real V1 bridge does and does not cover.

### Durable persistence

This is the strongest "really runs against infrastructure" piece. The hot path
stays in memory (`RoomRegistry`), but
`apps/v3/lilith-world-server/src/durable_persistence.rs` adds two real backends.
`PostgresDurableSessionStore` (`durable_persistence.rs:164`) uses `sqlx` to
create and write the `v3_session_boundary_writes` table, persisting a JSON
checkpoint envelope at each session boundary
(`SessionBoundaryKind::Start`/`Checkpoint`/`End`). `RedisHotStateStream`
(`:281`) appends each hot-state checkpoint to a real Redis Stream via `XADD`
under the `v3:hot-state` prefix and reads the latest back with
`XREVRANGE COUNT 1`. Crucially, the checkpoint's participant transforms are
**prost-encoded with the canonical wire bytes** before being wrapped in JSON
(`CheckpointRecord`, `:83`–`:144`), so the durable representation can never
drift from the on-wire one. On crash, `recover_room_from_redis_hot_state`
(`:362`) replays the latest stream entry into a fresh
`RoomRegistry::restore_checkpoint`. The integration test
`durable_persistence_recovers_room_from_real_postgres_and_redis` exercises a
real Postgres + Redis round trip — this is not a mock.

### Hot-reload of scenes and assets

`HotReloadPoller` (`lib.rs:787`) polls a published scene-graph and asset
manifest at `HOT_RELOAD_POLL_INTERVAL_MS = 30_000` (`:637`), tracking
`SceneRevision`s through a `SceneRevisionCatalog` and applying new revisions on
instance-restart boundaries; `FileAssetManifestSource` (`:760`) reads manifests
from disk with typed `AssetManifestError`s. This is how a re-published Lilith
Commons scene reaches a live shard without a redeploy — the authoring side is
[Authoring and Content Pipeline](./authoring-and-content-pipeline.md).

## Lilith Realtime Gateway — the edge

### The transport-selection ladder

`web_client_echo_with_transport_fallback` (`lib.rs:1814`) encodes the monolith's
preference order as real branching against a `WebClientNetworkProfile`: if UDP
is allowed it returns **WebTransport over QUIC**
(`route_note: "webtransport-quic"`); else if TURN/TCP is allowed it returns a
**WebRTC data channel** relayed over TURN/TCP; else if WebSockets are allowed it
falls to a **WebSocket last resort** (`/v3/realtime/ws`, subprotocol
`oshun.v3.realtime+json`); else it errors with "no realtime transport
available." The QUIC path is not a placeholder:
`spawn_quinn_webtransport_echo_server` (`:1873`) and
`quinn_webtransport_echo_round_trip` (`:1898`) open a real `quinn` endpoint with
a self-signed cert and complete a bidirectional-stream echo, decoding a real
WebTransport `CONNECT`/draft-02 frame.

### Auth handshake

`authenticate_gateway_handshake` (`lib.rs:434`) parses a `Bearer` token and
calls `verify_auth_primitives_hs256_jwt` (`:2476`), which does a genuine HS256
HMAC verification: it base64url-decodes header/payload/signature, rejects any
header whose `alg`/`typ` is not `HS256`/`JWT`, recomputes the signature over the
signing input, then validates `exp`/`nbf`, requires `token_type == "access"`,
and enforces an optional required scope. The gateway also implements full
**one-time refresh-token rotation** (`JwtRefreshRotationStore`, `:166`), so a
rotated refresh token is single-use and an explicitly revoked family is rejected
(tests
`jwt_refresh_rotation_issues_one_time_pair_and_authenticates_rotated_access_token`
and `jwt_refresh_rotation_rejects_explicitly_revoked_refresh_token`). These JWTs
are the V1 identity service's auth-primitive tokens — the bridge is documented
in
[V1 Integration and Identity Bridge](./v1-integration-and-identity-bridge.md).

### Session pinning and reconnect

A session is pinned to a shard by `GatewayConsistentHashRing::route_session`
(`lib.rs:662`) and held in a `SessionPinDirectory` (`:686`) for
`SESSION_PIN_TTL_SECONDS = 60` (`:675`). Transient drops survive via
`GatewayReconnectDirectory` (`:806`) for
`GATEWAY_RECONNECT_RETENTION_SECONDS = 60` (`:676`), with a
`RECONNECT_RESUME_BUDGET_MS = 3_000` (`:1068`) resume target — the same 60 s
window the world server honours, so a reconnecting client lands back on the
shard that still holds its retained participant state.

### Voice SFU and the HRTF model

Voice routes through `VoiceSfuModule` (`lib.rs:1474`), whose compatibility mode
is `LiveKitServerSdkEquivalent`. `route_gateway_packet` (`:1504`) validates an
Opus 24 kbps mono payload, refuses muted senders, fans the packet out to every
_other_ peer in the room, and attaches per-recipient HRTF metadata. The
mouth-to-ear latency is modelled explicitly:

```
latency_ms = OPUS_FRAME_DURATION_MS(20) + gateway_to_sfu_ms
           + sfu_fanout_ms + (recipient_regional_rtt_ms / 2)
```

checked against `SFU_MOUTH_TO_EAR_LATENCY_BUDGET_MS = 80` and a per-region
`REGIONAL_VOICE_LATENCY_BUDGET_MS = 50` (`:1061`–`:1062`), with a packet-loss
budget of `VOICE_LOSS_RATE_BUDGET = 0.005`. Operator mute/kick commands route
through `apply_operator_command` (`:1569`) within an
`OPERATOR_SFU_COMMAND_BUDGET_MS = 200` deadline. Per-tenant voice policy (who
may speak, push-to-talk vs open mic) is resolved through
`VoiceChatPolicyDirectory` (`:1346`). The audio rendering itself is on the
client — see [Avatar, Animation, and Audio](./avatar-animation-and-audio.md).

## The Unreal side: a codec and a probe, not a NetDriver

The monolith's network-integration section is the largest gap between
description and code, so this section states what is actually in `V3/ue/Source`.

**V3Net is a wire codec, not a `UNetDriver`.** `V3NetProtocol.cpp` implements
`FV3NetMultiplayerProtocolCodec` — a hand-written Protobuf encoder/decoder
(zig-zag `WriteSInt32`, length-delimited `WriteMessage`, a `ReadVarint` state
machine) producing bytes identical to the Rust/TS encoders.
`BuildInEngineProtocolExchangeReport` (`V3NetProtocol.cpp:958`) round-trips the
full packet set (snapshot-delta, presence, voice-control, interaction, plus
client/server envelopes) entirely in-engine and reports `bDecodedFullPacketSet`.
The module's `V3Net.Build.cs` depends only on `Core`/`CoreUObject`/`Engine`/
`GameplayTags`/`V3Core` — there is **no `NetworkCore` dependency, no `cxx`
bridge, and no `quinn`**. The C++ structs are plain `FV3Net*` value types
(`V3Net.h`), not `UObject` actors with replication flags. This is a real,
valuable, test-covered protocol bridge — it just is not the engine-integrated
net driver the monolith claims.

**V3Voice is a deterministic spatialization probe.**
`FV3VoiceRealtimeGatewayClient` (`V3VoiceRealtimeGateway.cpp`) contains genuine
positional audio math: `BuildPositionalMetadata` computes distance, azimuth, and
elevation from millimetre relative positions, and
`ApplyResonanceAudioSpatialization` applies a real equal-power pan law plus
distance attenuation. Notably its mouth-to-ear formula (`ReceiveGatewayFrame`,
`:523`) is _the same_ as the Rust gateway's —
`frame + gateway_to_sfu + sfu_fanout + rtt/2` — so the two stacks agree on the
latency model by construction. But `ConnectToRoom` (`:397`) validates config and
participants and checks `IsResonanceAudioRuntimeAvailable()` via
`FModuleManager`; it does **not** open a WebRTC socket.
`RunSixteenParticipantLatencyProbe` and `RunRegionalLatencyValidationProbe` are
deterministic in-engine probes over test fixtures, not live SFU sessions.

**V3World is a scene/launch validator, not a `UV3WorldSubsystem`.**
`FV3AtriumPlaceholderSceneBuilder` (`V3World.cpp`) builds and validates a
placeholder Atrium scene spec, and `BuildColdJoinValidationReport` produces a
join-latency percentile report from _deterministic phase fixtures with seeded
jitter_ — a synthetic budget check (96 samples against a 5 s budget), not a
measurement of real joins. `ConnectLocalWorldServerStack` validates a service
name and port (`lilith-world-server:43101`); it is configuration validation, not
a live connection. These modules are covered by UE automation
(`V3NetProtocolTests.cpp`, `V3VoiceRealtimeGatewayTests.cpp`,
`V3AtriumPlaceholderSceneTests.cpp`), and the codec test's golden-hex assertion
is genuinely strong — but treat the "connects to the world server" language as a
fixture, per [Tier 1 UE5 Client](./tier1-ue5-client.md).

## A join, end to end

The intended flow — assembled from the real functions above, even though the
serving daemon does not yet wire them together — looks like this:

```mermaid
sequenceDiagram
    participant C as Client (UE / Web / PS worker)
    participant G as Realtime Gateway
    participant W as World Server (RoomRegistry)
    participant P as Postgres + Redis
    C->>G: handshake(Bearer JWT, room_id, transport)
    G->>G: verify_auth_primitives_hs256_jwt (HS256)
    G->>G: route_session -> shard pin (TTL 60s)
    G->>G: pick transport (QUIC -> WebRTC -> WebSocket)
    C->>G: ClientEnvelope{ negotiation_request }
    G->>W: forward to pinned shard
    W-->>C: VersionNegotiationResponse (v3.0.0)
    C->>W: ClientEnvelope{ presence }
    W->>W: validate_state_transition (<= 12 m/s)
    W->>W: visible_entities (rstar nearest-N, cap 32/256)
    loop 20 Hz transform / 60 Hz expression
        W->>W: FixedTickLoop.tick -> SnapshotBroadcastPlan
        W-->>C: ServerEnvelope{ snapshot_delta }
    end
    W->>P: session-boundary checkpoint + hot-state XADD
    Note over C,W: on transient drop, retain 60s; resume on the same shard
```

## Capacity math: Pixel Streaming workers are clients

The capacity model that makes V3 affordable is that **a Pixel Streaming worker
is itself a full multiplayer client of the world server**, so fan-out scales
with `native_ue + px_stream_workers + tier2_fallback`, not with unique humans. A
4,096-attendee stadium concert produces only ~1,025 connected world-server
clients, because the crowd band consumes one multicast video stream (a single
master capture worker is the client) rather than 3,072 per-attendee
subscriptions. For a Tara class-tier room (≤ 64 attendees) the math is flat —
every attendee is a client. This is an architectural model in the monolith and
the relay's autoscaler rather than something the (health-only) world-server
daemon measures today; the relay that schedules those workers is detailed in
[Tier Routing and Pixel Streaming](./tier-routing-and-pixel-streaming.md).

## Related

- [Netcode Protocol and Physics](./netcode-protocol-and-physics.md) —
  prediction, reconciliation, and the physics contract in depth
- [Tier Routing and Pixel Streaming](./tier-routing-and-pixel-streaming.md) —
  the router and GPU edge that sit in front of the gateway
- [Tier 1 UE5 Client](./tier1-ue5-client.md) and
  [Tier 2 Fallback Web Client](./tier2-fallback-web-client.md) — the two
  protocol peers the backbone serves
- [Avatar, Animation, and Audio](./avatar-animation-and-audio.md) — client-side
  voice rendering and expression interpolation
- [Tara, Classes, Aja, and the Commons](./tara-classes-aja-and-commons.md) —
  asana-lock and physical-adjustment consent in practice
- [V1 Integration and Identity Bridge](./v1-integration-and-identity-bridge.md)
  — the JWTs the gateway verifies and the V1 services behind them
- [Saraswati Stage Pipeline](./saraswati-stage-pipeline.md),
  [Authoring and Content Pipeline](./authoring-and-content-pipeline.md) — what
  the hot-reload poller consumes
- [Subsystem Glossary and Layout](./subsystem-glossary-and-layout.md) and
  [Product Promise and Architecture](./product-promise-and-architecture.md) —
  the orientation pages, and the hub
  [../V3_ARCHITECTURE.md](../V3_ARCHITECTURE.md)
- [Data Tenancy and Residency](./data-tenancy-and-residency.md),
  [Persona, Policy, Provenance, and Rights](./persona-policy-provenance-and-rights.md),
  [Commerce and Royalties](./commerce-and-royalties.md), and
  [Observability, Performance, Security, and Launch](./observability-performance-security-and-launch.md)
  — the cross-cutting concerns the backbone participates in