Egbe's premium surface is a single UE5.5 project — Lumen, Nanite, MetaHuman agents, an interest-managed crowd of LLM-driven minds. That is wonderful on a gaming PC and impossible on a Chromebook in a library. The edge is the set of systems that make the world reachable anyway, across the whole device spectrum, without ever forking the simulation. It has three planes, and they are deliberately layered so that fidelity can fall while identity, presence, and the authoritative world never do. The Egbe Realtime Gateway is the door: it terminates whatever transport a client can actually open (QUIC if the network allows, WebRTC or a plain WebSocket if it does not), pins the session to the world-server shard holding its instance, and hosts the voice SFU that carries conversation and squad comms. Pixel Streaming is the premium browser path: a pooled UE worker renders an Orun ground server-side and ships H.264 frames over WebRTC to a thin browser shell, so a tab gets the same world a native client does. The Tier-2 web fallback is the floor: a three.js/WebGPU client that renders a reduced Orun locally on constrained devices, speaking the exact same wire protocol to the exact same world server, capping how many agents it draws but never which world it joins.
The load-bearing idea is that all three planes converge on one authority. A
native UE client, a Pixel Streaming worker, and a Tier-2 fallback are, from the
world server's point of view, three kinds of the same thing: protocol peers that
receive interest-managed agent state and send actions. The wire protocol encodes
this directly — its AuthorityKind enum names UE_CLIENT, TIER2_FALLBACK,
and REALTIME_GATEWAY as distinct, first-class participants. This page is the
deep dive on those three planes and the two browser libraries that implement
them; it is a companion to the orientation map in
../V6_ARCHITECTURE.md.
What ships, honestly#
Real and on-disk. The wire protocol is genuine and the densest of the three
target libraries: libs/v6/egbe-protocol carries a 373-line proto3 schema
(proto/oshun/v6/egbe/v1/egbe.proto), generated TypeScript codecs
(src/generated/...), a Rust crate (rust/src/lib.rs), and UE C++ bindings
(ue/generated/.../egbe.pb.cc + .pb.h) so the same messages cross the
TypeScript fallback, the Rust services, and the UE client. Its src/index.ts
implements interest-managed delta replication for real
(projectAgentStateSnapshot, encodeAgentStateDeltaFromSnapshots,
applyAgentStateDelta), a version/capability handshake
(negotiateProtocolHandshake), and a bandwidth budget validator
(validateReplicationBandwidth against GROUND_REPLICATION_MAX_BPS = 256_000).
The browser Pixel Streaming client (libs/v6/egbe-web-pxstream) is a real state
machine wrapping Epic's published frontend SDK
(@epicgames-ps/lib-pixelstreamingfrontend-ue5.5, a real dependency in
package.json), with a first-frame SLA timer and telemetry. Off-engine, the two
Rust edge services exist and are substantial: the realtime gateway is 3,945
lines (apps/v6/egbe-realtime-gateway, 23 in-crate tests) and the Pixel
Streaming relay 1,290 lines (apps/v6/egbe-pxstream-relay, 9 in-crate
tests).
Policy-as-data, not a renderer — and labeled as such. The most important
honest caveat is libs/v6/egbe-engine-web-fallback. Despite its name and its
three dependency, this library contains no rendering code: its index.ts
never imports three.js. It is the budget and density-policy layer for the
fallback — it computes deterministic renderer budgets, the Tier-2 agent-density
cap, and a reduced-scene manifest. The actual WebGPU/WebGL2 renderer lives in
the apps/v6/egbe-web-fallback application; this library is the honest,
testable contract it renders against. That is a real and useful seam, but the
page calls it policy, not pixels.
The relay's match is a fail-honest env-seam, and simpler than V3's. V3's
matchmaker was a ~4,800-line affair with POP geo-scoring, HMAC session JWTs, and
SDP construction. V6's relay is deliberately narrower:
match_pxstream_session resolves a worker, region, signalling URL, and codec by
reading V6_PXSTREAM_* environment variables with local-dev defaults (region
→ "local-orun", codec → "H264") — there is no POP geo-scoring in
source (a grep for select_pop/haversine/nearest_pop returns zero hits).
The relay's real logic is admission control: abuse classification,
per-user/per-network concurrency caps, free-tier backpressure, and idle
lifecycle. So "matchmaking" here means "lease a worker and enforce the gates,"
with the geographic fleet picker left as an injectable seam rather than a fake.
Shared across versions. The gateway crate is shared infrastructure that has
accreted features from later products: alongside the V6 transport and SFU
capabilities it carries v7.pheme.* proximity-voice extensions
(PHEME_WHISPER_RANGE_M, occlusion attenuation, parental controls). That is not
fabrication, but it means the crate is broader than "the V6 gateway"; this page
describes the V6-relevant surface and flags the rest as cross-version reuse.
The shape of the edge#
The tier router that picks which of these a visitor gets is a route inside the V1 BFF, not part of the edge services themselves; see ../../platform/bff-gateway.html for that front door. Everything below is what happens after the door.
The Egbe Realtime Gateway#
The gateway (apps/v6/egbe-realtime-gateway/src/lib.rs, Rust) is the single
junction every live client reaches. Its SERVICE_DESCRIPTOR advertises the V6
capabilities precisely: webtransport-edge, webrtc-fallback,
websocket-fallback, world-shard-routing, presence-transform-20hz,
voice-sfu, and squad-voice-routing.
Transport negotiation and the fallback order#
A browser on a corporate network may have UDP/QUIC blocked; a mobile client on a
flaky link may only manage a WebSocket. The gateway models this as a typed
ladder. RealtimeTransportMode is an enum of WebTransportQuic,
WebRtcDataChannel, and WebSocket, each with a stable protocol_label() and
an expected_path() (/v6/realtime/webtransport, /v6/realtime/webrtc,
/v6/realtime/websocket). The canonical preference is
RealtimeTransportMode::fallback_order() →
[WebTransportQuic, WebRtcDataChannel, WebSocket], QUIC-primary with two
documented fallbacks.
Clients declare what they can open through RealtimeClientNetworkProfile, which
ships three named constructors that read like the real-world cases they
represent: quic_primary() (everything available), quic_blocked_with_webrtc()
(no QUIC, WebRTC + WS), and websocket_only() (the floor).
route_client_with_best_transport walks the fallback order and selects the
first mode the profile supports(...); if none is available the gateway fails
loudly with RealtimeGatewayError::NoAvailableTransport, carrying the
client_id — it does not pretend a connection it could not make. The negotiated
ClientShardRoutingDecision also carries the full fallback_chain so the
client knows what it can renegotiate to if its link degrades mid-session.
World-shard routing#
Transport is only half the door; the other half is which shard. The gateway
holds a WorldShardDirectory keyed by instance id (route_for_instance,
upsert_route). route_client first terminates the transport, then looks up
the WorldShardRoute for the requested instance, and fails with
InstanceShardNotFound if no shard claims it. A successful decision binds
client_id, instance_id, the terminated transport, and the target shard into
one ClientShardRoutingDecision. This is the mechanism behind "session shard
pinning": a reconnecting client returns to the same shard within the retention
window (GATEWAY_RECONNECT_RETENTION_SECONDS = 60,
SESSION_PIN_TTL_SECONDS = 60), so a dropped frame of network does not scatter
a player across the continuum. The shard model itself — Solo homesteads, Co-op,
and the Commons fleet — is described in
./world-server-and-shard-continuum.md;
the gateway's job is only to find the right one and pin to it.
Presence flows over this junction at the protocol's ground cadence:
presence-transform-20hz, matching GROUND_REPLICATION_HZ = 20 from the shared
protocol crate — transforms at ≤ 20 Hz, activity-state changes event-driven.
The voice SFU#
Vac's live conversation and squad comms run as voice, and the gateway is the
selective forwarding unit. A VoiceSfuPeer models a participant richly:
session_id, entity_id, room_id, region, regional_rtt_ms, an optional
position_mm for positional audio, plus squad_ids,
muted_speaker_entity_ids, an accessibility profile, and parental controls.
Voice rides a real codec budget — OPUS_24KBPS_MONO_* constants fix 24 kbps
mono in 20 ms frames — against a regional latency target
(VAC_REGIONAL_VOICE_LATENCY_BUDGET_MS = 80) and a loss ceiling
(VAC_VOICE_LOSS_RATE_BUDGET = 0.005). This is the same SFU whether a squad is
together in Orun or split across an Aye incarnation, which is why the protocol's
squad-comms routing asserts realm parity (below). The proximity-voice ranges and
occlusion model carried here are the v7.pheme.* extensions noted above —
present in the shared crate, beyond V6's own scope.
Pixel Streaming: server-rendered Orun in a browser#
Pixel Streaming gives a browser the premium world: a UE worker renders an Orun ground headless and streams it. Two pieces implement it — the relay that leases a worker, and the browser library that plays the stream.
The relay: admission, abuse, idle — and the match seam#
apps/v6/egbe-pxstream-relay/src/lib.rs exposes match_pxstream_session, which
returns a PixelStreamingMatchResponse: a session_id, worker_id, region,
signalling_url, streamer_id, the first_frame_budget_ms
(DEFAULT_FIRST_FRAME_BUDGET_MS = 8_000), an expires_at_epoch_seconds
(+ DEFAULT_SESSION_TTL_SECONDS = 120), a fallback_url, and codec: "H264".
As flagged in What ships, honestly, the worker/region/signalling fields are
resolved from V6_PXSTREAM_* env vars with local defaults — a fail-honest seam
where a production fleet picker would plug in, not a simulated one.
The genuine logic is evaluate_v6_pxstream_admission, a real gate that runs in
order: first abuse — if an abuse_signal_window is present it calls
classify_pxstream_abuse and, on a non-Allow action, returns a review or ban
decision before any worker is leased (the comment is explicit that V6 reuses
the V3 abuse classifier, held to
MIN_ABUSE_CLASSIFIER_PRECISION_BASIS_POINTS = 9_000). Then concurrency:
PXSTREAM_MAX_CONCURRENT_SESSIONS_PER_USER = 2 and ..._PER_NETWORK = 8
(institutional networks get 64). Then free-tier backpressure: a free
entitlement under regional load (FREE_TIER_BACKPRESSURE_BASIS_POINTS = 8_500)
is steered to the fallback URL rather than a GPU.
evaluate_pxstream_idle_lifecycle reclaims an idle worker — prompt at
PXSTREAM_IDLE_PROMPT_AFTER_SECONDS = 90, disconnect and release-to-pool at
150. Every rejection carries a reason code and a fallback_url, so the
relay's answer to "no GPU for you" is always "here is the fallback," never a
dead end.
The browser client: the first-frame SLA#
libs/v6/egbe-web-pxstream/src/index.ts is the player.
EgbePixelStreamingClient is an observable state machine over the lifecycle
idle → matching → matched → signalling → streaming, with degraded as the
off-ramp. start() POSTs a session request to the matchmaker
(V6_PIXEL_STREAMING_MATCH_PATH = '/api/v6/pxstream/match'), validates the
response with parseEgbePixelStreamingSession (which rejects a non-matched
status or a non-positive budget — the client refuses a malformed lease), then
constructs the real Epic adapter.
EpicEgbePixelStreamingAdapter dynamically imports
@epicgames-ps/lib-pixelstreamingfrontend-ue5.5 and configures it for an Orun
session: the signalling URL and streamer id from the lease, PreferredCodec
from the match (H264 or AV1), WebRTCMaxBitrate of 12 Mbps, and keyboard /
mouse / touch / gamepad input on with the mic off. It maps Epic's events onto
the client's vocabulary — webRtcConnecting → signalling,
streamLoading → loading, videoInitialized → first-frame,
webRtcDisconnected → disconnected with the SDK's allowClickToReconnect flag
preserved.
The first-frame SLA is the contract that makes this honest about quality.
DEFAULT_FIRST_FRAME_BUDGET_MS = 8_000: when signalling begins the client arms
a timer for the budget; if first-frame arrives first it computes
evaluateFirstFrameTelemetry and moves to streaming only when
elapsedMs <= budgetMs, otherwise to degraded with the over-budget elapsed
time recorded. If the timer fires first, the session is declared degraded
without waiting. The adapter is an injected adapterFactory, so the unit spec
drives the whole machine with a fake adapter and asserts the real transitions —
including that an 8,999 ms first frame classifies within-budget and a 9,001 ms
one exceeded-budget. The state machine, not a mock, is what is under test.
The Tier-2 web fallback#
When Pixel Streaming is unavailable — no GPU near enough, free-tier
backpressure, a relay rejection — the visitor still gets Orun, rendered locally
at reduced fidelity. libs/v6/egbe-engine-web-fallback is the policy layer for
that client.
Budgets and the agent-density cap#
selectFallbackRendererBudget returns a deterministic budget per backend.
WebGPU is the preferred path: reduced quality, 45 fps target, a 22 ms
frame cost ceiling, 32 visible agents, agent LOD ≤ 2, procedural foliage on.
WebGL2 is the floor: low quality, 30 fps, a 30 ms ceiling, 18 visible
agents, LOD ≤ 1, no procedural foliage. selectTier2AgentDensityBudget then
applies the density rule that makes a constrained device believable rather than
empty: on a constrained profile the visible cap drops hard — WebGL2 to 6,
WebGPU to 10 — but protectsFocusedAgent stays true, the selection
strategy is 'focus-then-salience-distance', and culled agents are not deleted:
maxNarrativeBackfillAgents = max(visible * 3, 12) keeps them present as Clio
narrative summary. A small device sees fewer faces, not a smaller world. Each
budget emits a densitySignature string (e.g.
tier2-density:webgl2:constrained:visible6:backfill18) so a parity test can pin
the exact policy. createOrunFallbackSceneManifest rounds it out with the
reduced reference scene (orun-tier2-grove, district GroveOfBeginnings) and
an explicit omittedTier1Features list — lumen-global-illumination,
nanite-hero-geometry, chaos-cloth-crowds, niagara-high-density-particles —
so the fidelity gap from Tier-1 is documented data, not a surprise.
One protocol, two renderers#
The fallback's whole premise is that it is the same world, and that holds
because rendering is the only thing that changes — the wire is identical.
libs/v6/egbe-protocol is the shared contract, and its proto makes the
fallback a first-class peer: AuthorityKind enumerates UE_CLIENT and
TIER2_FALLBACK and REALTIME_GATEWAY as distinct authorities, and the
ClientEnvelope/ServerEnvelope oneofs carry the same PresencePacket,
AgentStateSnapshot/AgentStateDelta, PerceptionBatch, ActionBatch,
SquadCommsMessage, and WorldEvent to every client kind. The interest
management that keeps a constrained client cheap is real protocol code:
projectAgentStateSnapshot clips a snapshot to an InterestWindow (origin,
radius, GROUND_VISIBLE_AGENT_CAP = 32, GROUND_VISIBLE_PLAYER_CAP = 8),
encodeAgentStateDeltaFromSnapshots emits only upserts and removals against a
base sequence, and validateReplicationBandwidth proves the result fits
GROUND_REPLICATION_MAX_BPS = 256_000. negotiateProtocolHandshake lets a
weaker client advertise a smaller capability set and receive a DEGRADED
acceptance with the dropped features named — the same mechanism that lets the
fallback ask for fewer perception items or a lower agent LOD ceiling without
falling off the protocol. The UE client renders these messages with Nanite and
MetaHumans; the fallback renders them with three.js; the gateway forwards them;
none of them disagree about what is true in the world.
How the three planes compose#
Read top to bottom, a visitor's path is: the V1 BFF tier router picks a surface
(../../platform/bff-gateway.html); a premium
browser is leased a worker by the relay and plays it through
@oshun/egbe-web-pxstream under an 8-second first-frame SLA, degrading to the
fallback URL on any rejection; a constrained browser loads the Tier-2 client
under a focus-protecting density cap; and all of them — plus the native UE
client and the Pixel Streaming worker — open a transport through the Realtime
Gateway, pin to a world shard, and exchange egbe-protocol messages with the
authoritative world. Fidelity is the only thing that moves down the ladder.
Identity, presence, the shard, the SFU, and the wire are constant. The client
side of that wire — the UE5 module split, agent embodiment, and the density LOD
pipeline that the protocol's interest windows feed — is detailed in
./ue5-client-modules-and-embodiment.md,
and the authoritative world the gateway routes into is in
./world-server-and-shard-continuum.md.