A heist crew forming up in 1947 Urban, a posse queuing for a Frontier showdown,
a 5v5 Capital Raid in the Sci-Fi cell, and a lone detective syncing a Mind
Palace clue from the companion app on a train all touch the same backbone.
They sign in through one account, carry one Bureau XP ledger, route through one
skill-rated matchmaker, and land their scores, replays, and cosmetics in one set
of domain-isolated stores. That backbone is not a diagram on a slide: it is
sixteen deployable services under apps/v5, each a TypeScript/NestJS package
with a real Postgres/Redis domain behind it, fronted by a sixteen-endpoint
Unreal C++ client catalog in V5/ue/Source/V5OnlineServices that speaks to them
over genuine HTTP. This page is the product-and-code tour of that layer: the
services, what each really does, the request seam that connects client to
cloud, and the backend infrastructure — containers, clusters, regions, health,
and graceful degradation — that keeps it standing. The connection models, PvP,
ranked, and esports surfaces that ride this backbone live in
Modes & Multiplayer; the seasons, calendar,
workshop marketplace, and community programs are in
Live Service & Community; the engine-side
save, persistence, and cross-play machinery is the architecture companion,
Persistence, Online Services & Cross-Play.
For the full feature scope this slots into, start at the hub:
../V5_features.md.
What ships, honestly#
The split is clean, and stating it up front keeps the rest of the page honest. Two tiers are real code, and a small set of seams are labelled as plan or fixture rather than wired integration.
The client tier is real Unreal C++. V5/ue/Source/V5OnlineServices ships a
sixteen-endpoint service catalog
(UV5_Online_ServiceCatalog::BuildEndpointCatalog) fronted by a family of
UBlueprintFunctionLibrary request builders — one per service
(UV5_Online_AuthService, …MatchmakingService, …AntiCheatClient,
…WorkshopClient, and the rest) plus a cross-cell-world builder — over a typed
V5OnlineServicesTypes.h surface. Crucially the transport underneath is
genuine network I/O: V5OnlineServices.Build.cs declares the HTTP and
Json modules (the build comment marks this "the first FHttpModule usage in
V5 UE"), and FV5OnlineHttp::BuildHttpRequest turns a request value into a real
IHttpRequest from FHttpModule::Get(). Authentication is real cryptography:
FV5JwtCrypto is a self-contained FIPS 180-4 SHA-256, RFC 2104 HMAC-SHA256, RFC
7519 HS256 JWT, and RFC 7636 PKCE S256 implementation, unit-tested against
published vectors.
The service tier is real TypeScript + NestJS. apps/v5 holds sixteen
service packages — each with a contract.json, a Dockerfile, and k8s/
manifests — and every one delegates its handle() to handleRealRequest in
@v5/service-shared, which resolves live pg.Pool + ioredis clients and
dispatches to a domain that is not CRUD: a Glicko-2 rating pipeline, an OAuth
refresh-rotation flow with reuse detection, a hash-chained balance ledger, a
version-vector CRDT Mind Palace merge. Each carries Vitest cases that assert
computed answers, not shapes.
Three honest labels travel with the rest of the page:
- There are two backend code paths, and the live one is real. The pure
evaluateServiceSpecificRuleinruntime.tsstill returns the oldjwt.v5.<account>.oauthfixture string for golden contract tests — but the deployedservice.handle()callshandleRealRequest→executeRealServiceRequest, the Postgres/Redis-backed path. The fixture path validates contract shape; the real path is what runs in the container. - Kernel anti-cheat is a planned client tier. The client builds EAC signal
requests and
platformPolicynames the signed Windows driver (EasyAntiCheat_EOS.sys), but that kernel driver is named, not in-tree — and the server-side strike journal that does ship is real. - The design's service table is a subset, and "Voice" is design-named. The
feature brief lists ten representative rows (including a
Loginand aVoiceservice); the shipped backbone is sixteen packages whereLoginis theauthservice and there is no standalonevoicepackage inapps/v5— in-game voice is routed through party/presence, not a separate in-tree service.
The service catalog — sixteen services on one backbone#
Gameplay code never hand-rolls a URL. BuildEndpointCatalog enumerates all
sixteen services, each as an FV5OnlineEndpointDefinition carrying its verb,
path, bRequiresJwt, and bSupportsOfflineQueue flags; a UV5_Online_*
builder fills an FV5OnlineServiceRequest, and the catalog is the single source
of truth for ports and routing. The service catalog (service-catalog.json)
declares the same sixteen on the server, with ports assigned as
4210 + service index (auth = 4210 … faction-rep = 4225, identical in
FV5OnlineHttp::ServicePort and the catalog).
| Service | Client builder | Real domain behind it |
|---|---|---|
| auth | UV5_Online_AuthService |
AuthDomain — OAuth exchange, HS256 JWT, refresh rotation + reuse detection |
| friends | UV5_Online_FriendsService |
FriendsDomain — cross-platform graph + Redis presence heartbeats |
| parties | UV5_Online_PartiesService |
PartiesDomain — group/ready-check, cross-cell Bureau-HQ sessions |
| matchmaking | UV5_Online_MatchmakingService |
MatchmakingDomain — Glicko-2 + Redis ZSET ticket queue |
| leaderboards | UV5_Online_LeaderboardsService |
LeaderboardDomain — per-cell/per-season Redis ranked boards |
| replays | UV5_Online_ReplaysService |
ReplaysDomain — chunked upload, signed download grants, AI highlight reels |
| telemetry | UV5_Online_TelemetryClient |
TelemetryDomain — batched event ingest, ClickHouse routing |
| anti-cheat | UV5_Online_AntiCheatClient |
AntiCheatStore — eight detectors + persisted strike journal |
| crash-reporting | UV5_Online_CrashReportingClient |
CrashReportingDomain — dedupe + symbolication queue |
| companion-app-bridge | UV5_Online_CompanionAppBridge |
CompanionAppBridgeDomain — read-only home view, save export/import, AR/VTuber |
| mindpalace-cloud-sync | UV5_Online_MindPalaceCloudSyncClient |
MindPalaceCloudSyncDomain — version-vector CRDT graph merge |
| workshop | UV5_Online_WorkshopClient |
WorkshopDomain — publish/moderate, paid-mod marketplace, refunds |
| compliance-dsar | UV5_Online_ComplianceDSARClient |
ComplianceDsarStore — GDPR/CCPA/DSA/COPPA + data residency |
| live-service-calendar | UV5_Online_LiveServiceCalendarClient |
LiveServiceCalendarDomain — seasonal feeds with ETags |
| balance-ledger | UV5_Online_BalanceLedgerClient |
BalanceLedgerDomain — hash-chained append-only ledger |
| faction-rep | UV5_Online_FactionRepClient |
FactionRepDomain — reputation writes + weekly decay |
The HTTP seam and fail-loud transport#
FV5OnlineHttp::BuildHttpRequest constructs the real request: it sets the verb,
Content-Type/Accept: application/json, an Authorization: Bearer header
when a JWT is present, the routing headers X-V5-Account and X-V5-Region, and
the JSON body for non-GET methods. Base-URL resolution is fail-soft and never
fabricated — V5_ONLINE_BASE_URL is treated as an API gateway, else
V5_ONLINE_HOST plus the service's catalog port.
The latent action node UV5_Online_HttpRequest::Activate is where the fail-loud
discipline shows. A request whose endpoint bRequiresJwt but whose token is
empty short-circuits to a 401 before touching the network
(auth.jwt.required). A genuine transport failure resolves to
TransportFailureResponse (service.unreachable, HttpStatus = 0) and sets
bQueuedOffline from the endpoint's offline-queue flag. A 503 likewise flips
bQueuedOffline and parses the backend's structured errorCode/errorMessage.
No path fabricates a success it did not receive — the response booleans reflect
the wire, not a hope.
Identity: JWT, PKCE, and the single account#
Login is the gateway every other service trusts, so it is real on both sides of
the wire. On the client, UV5_Online_AuthService::IssueOAuthJwt mints a genuine
HS256 token: it assembles RFC 7519 registered claims (iss = v5.bureau.auth,
sub, aud = v5.client, provider, iat, nbf, exp with a 3,600-second
TTL, jti), derives the jti from SHA-256(account|provider|iat) so each
issuance is unique yet deterministic, signs with FV5JwtCrypto::IssueHs256, and
derives an opaque refresh token as
v5rt_ + base64url(HMAC(SHA-256(key.refresh), SHA-256(accessJwt))) — revocable
server-side and not user-forgeable. VerifyAccessJwt does the real inverse: a
constant-time signature comparison plus exp/nbf enforcement, and
VerifyPkceChallenge checks BASE64URL(SHA256(verifier)) == challenge.
Server-side, AuthDomain (domain/auth.ts) is Postgres-backed and stricter
still. resolveCanonicalAccount enforces oldest-verified-credential-wins:
the first verified (platform, externalSubject) pair binds a canonical
v5acct.… id, and later exchanges for the same pair return that id — one human
cannot fork into multiple V5 accounts per platform, which is exactly what
cross-progression needs to stay coherent across all nine platforms.
refreshAccessToken implements RFC 6749 §6 rotation with RFC 6819 reuse
detection: presenting an already-rotated token sets every row in the family to
revoked = TRUE and returns reuse_detected. Refresh tokens are stored only as
SHA-256 hashes, never cleartext. The canonical account this issues is the spine
the cross-play policy
builds on, and the client mirror BuildPlatformCredentialLinkRequest binds an
external subject to it.
Matchmaking and the skill model#
The client builders are deliberately thin — BuildQueueRequest and
BuildCrossplayQueueRequest carry mode, region, an integer SkillRating, ping,
and (for cross-play) Platform, ModeId, and bCrossplayOptOut to a per-cell
queue path. The authority is server-side and genuine. MatchmakingDomain
(domain/matchmaking.ts) runs a real Glicko-2 update (Glickman 2013):
glicko2Update computes the g(φ) and expected-score terms, then solves the
volatility step with an Illinois (regula falsi) root finder — pinned in
tests to the canonical worked example (a 1500/200/0.06 player against three
opponents converging to ~1464.06 / ~151.52 / ~0.05999). The queue is a Redis
ZSET banded by v5:mm:cell:region:mode:skillBand:pingBand (200-wide skill
bands, three ping bands), ordered by enqueue time; formMatch uses an atomic
ZPOPMIN so two formation passes never claim the same player, and re-enqueues
what it took if a rare race leaves it short. The per-cell ranked rules, season
ladder, and crossplay opt-out consequences are surfaced in
Modes & Multiplayer; this service is the fairness
engine they sit on.
Anti-cheat: defense in depth, labelled by tier#
Anti-cheat is defended in depth. The client UV5_Online_AntiCheatClient builds
eight distinct, dedicated-endpoint requests: an EAC session signal, aim
plausibility (impossible-angle turn rate), sub-tick aimbot snap, line-of-sight
wall-hack history, position-delta speed-hack, auto-fire interval pattern, a
weighted classifier carrying modelId: V5_AntiCheat_ML, and an appeal routed to
the companion-app support path. Each detector's scoring math is real and lives
in runtime.ts — the speed-delta detector, for instance, divides observed
distance by elapsed time and flags when the ratio exceeds the weapon/character
max speed by more than 15%.
Server-side, AntiCheatStore (domain/anti-cheat.ts) makes this stateful:
recordSignal journals each flagged detector to Postgres and recomputes the
account's running strike count over verdicts in ('strike', 'ban-review'),
driving the three-strike ladder (warning → ranked-suspension →
ban-review); fileAppeal queues a row on the companion-app path. The
governing posture matches the platform's: model output prioritises a review
queue — it never silently disciplines. The platformPolicy helper encodes the
integration plan honestly: Windows is a signed kernel driver
(EasyAntiCheat_EOS.sys, blockLaunchIfUnavailable), Mac/Linux/console/iOS are
userland modules — but, per the label above, that kernel driver is named, not
shipped in this tree. The strike journal is what ships real.
The ledgers and graphs the social layer rides#
Three domains turn "online" into durable, auditable state:
BalanceLedgerDomainis a genuine append-only, hash-chained public ledger. Every entry's hash binds the previous entry's hash, its sequence number, and the canonical JSON of its payload; appends serialise through a single head row takenFOR UPDATEfor a gap-free monotonic sequence under concurrent writers; andverifyChainre-walks the chain, recomputes every hash, and returns the firstbrokenAtSeqif a row was edited out from under it. That is what makes a designer's economy tune evidence-linked rather than trust-me.MindPalaceCloudSyncDomainis a version-vector CRDT for the deduction graph.mergeNoderesolves by causal dominance, falling back to a deterministic(updatedUnix, lastWriter)tiebreak only on genuinely concurrent edits, and accusations are an append-only union (ON CONFLICT DO NOTHING). The merge is commutative and idempotent, so two devices syncing in either order — or the same diff twice — converge to one graph. This is the service that lets a clue annotated on the companion app reach the console save without clobbering it.FriendsDomainandPartiesDomainback the cross-platform social graph: friends pairs and presence heartbeats in Postgres + Redis, party create/join/ready-check with cross-cell Bureau-HQ sessions (a 64-player dedicated-server hub where account-scoped profile, cosmetics, Bureau XP, Mind Palace, and codex are shared while per-cell campaign branches stay isolated).
The backend infrastructure#
Every service is the same shape — a contract, a container, a cluster entry — and the shape is what makes sixteen of them operable as one fleet.
This picture is wider than it can be shown here. Move across it, or read it as text.
Drawn from its Mermaid source.
As text
Groups: - V5 client · Unreal C++: UV5_Online_ServiceCatalog 16 endpoints · UV5_Online_* builders, FV5OnlineHttp / UV5_Online_HttpRequest real FHttpModule · Bearer JWT · X-V5-Account/Region Parts: - UV5_Online_ServiceCatalog 16 endpoints · UV5_Online_* builders, in V5 client · Unreal C++ - FV5OnlineHttp / UV5_Online_HttpRequest real FHttpModule · Bearer JWT · X-V5-Account/Region, in V5 client · Unreal C++ - API gateway / DNS failover - 16 × NestJS service handle() → handleRealRequest → executeRealServiceRequest - V5_POSTGRES_URL + V5_REDIS_URL set? (decision) - 503 outage · not_configured bQueuedOffline per endpoint - real domain AuthDomain · MatchmakingDomain · … - Postgres db-per-service (data) - Redis ZSET queues · presence (data) - telemetry · crash-reporting - ClickHouse (data) - MultiRegionFailover CRD iad·fra·sin / pdx·dublin·syd Connections: - UV5_Online_ServiceCatalog 16 endpoints · UV5_Online_* builders to FV5OnlineHttp / UV5_Online_HttpRequest real FHttpModule · Bearer JWT · X-V5-Account/Region - FV5OnlineHttp / UV5_Online_HttpRequest real FHttpModule · Bearer JWT · X-V5-Account/Region to API gateway / DNS failover: V5_ONLINE_BASE_URL · else host:4210+idx - API gateway / DNS failover to 16 × NestJS service handle() → handleRealRequest → executeRealServiceRequest - 16 × NestJS service handle() → handleRealRequest → executeRealServiceRequest to V5_POSTGRES_URL + V5_REDIS_URL set?: resolveBackingClients - V5_POSTGRES_URL + V5_REDIS_URL set? to 503 outage · not_configured bQueuedOffline per endpoint: no - V5_POSTGRES_URL + V5_REDIS_URL set? to real domain AuthDomain · MatchmakingDomain · …: yes - real domain AuthDomain · MatchmakingDomain · … to Postgres db-per-service - real domain AuthDomain · MatchmakingDomain · … to Redis ZSET queues · presence - telemetry · crash-reporting to ClickHouse (dotted): residency-pinned - 16 × NestJS service handle() → handleRealRequest → executeRealServiceRequest to MultiRegionFailover CRD iad·fra·sin / pdx·dublin·syd
Show source
flowchart TD
subgraph Client["V5 client · Unreal C++"]
CAT["UV5_Online_ServiceCatalog<br/>16 endpoints · UV5_Online_* builders"]
HTTP["FV5OnlineHttp / UV5_Online_HttpRequest<br/>real FHttpModule · Bearer JWT · X-V5-Account/Region"]
end
CAT --> HTTP
HTTP -->|"V5_ONLINE_BASE_URL · else host:4210+idx"| GW[API gateway / DNS failover]
GW --> SVC["16 × NestJS service<br/>handle() → handleRealRequest → executeRealServiceRequest"]
SVC -->|resolveBackingClients| CFG{"V5_POSTGRES_URL<br/>+ V5_REDIS_URL set?"}
CFG -->|no| OUT["503 outage · not_configured<br/>bQueuedOffline per endpoint"]
CFG -->|yes| DOM["real domain<br/>AuthDomain · MatchmakingDomain · …"]
DOM --> PG[("Postgres<br/>db-per-service")]
DOM --> RS[("Redis<br/>ZSET queues · presence")]
TEL["telemetry · crash-reporting"] -. residency-pinned .-> CH[(ClickHouse)]
SVC --> K8S["MultiRegionFailover CRD<br/>iad·fra·sin / pdx·dublin·syd"]
One shape per service: contract, container, cluster#
Each service's contract.json is its machine-readable spec — slug, package
name, port, primary and additional endpoints (with requiresJwt /
supportsOfflineQueue), declared storage (Postgres + Redis, optional
analytics), capabilities, contractCases, the environment variable names it
reads, and its docker/kubernetes block. The Dockerfile is a real
multi-stage Node 24 build: it installs the workspace with pnpm, builds
@v5/service-shared then the service, prunes to production deps, and runs
node dist/main.js on the service's catalog port. The k8s/ directory carries
a Deployment (two replicas, /healthz readiness and liveness probes,
CPU/memory requests and limits), a Service, and a per-service
MultiRegionFailover custom resource. Databases are domain-isolated — each
service owns its store rather than sharing one schema, the database-per-service
posture the architecture specifies.
Configuration, health, and graceful degradation#
The most important honesty seam is how a service behaves when its stores are
not there. resolveBackingClients hands a service live pg.Pool / ioredis
clients only when its catalog-declared V5_POSTGRES_URL and V5_REDIS_URL
resolve to non-empty values; otherwise it returns configured: false with null
clients, and executeRealServiceRequest serves the 503 outage contract —
carrying retryAfterSeconds, the failover mode, and fallback regions, with
queuedOffline set from the endpoint flag. buildHealth reports
status: not_configured in that case rather than the hardcoded
serviceHealthy: true that once made the outage branch dead code. Absence is
reported loudly, never faked — and that fail-loud health is what the readiness
probe and DNS failover key off.
Region and failover policy is checked-in data. The catalog declares primary
regions iad / fra / sin and secondary pdx / dublin / syd, with
compliance-dsar, telemetry, and crash-reporting flagged
dataResidencyPinned. The shared services/k8s/multi-region-failover.yaml
OnlineServiceFailoverPlan sets a 45-second health window, a 0.98 success-rate
threshold, a 180 ms regional-latency ceiling, automatic DNS failover, and a
per-service mode (global-active-passive for auth/friends,
regional-active-active for parties, queue-backed for the offline-tolerant
services). The result is the degradation promise the player feels: a service
outage drops single-player to offline-deterministic campaign play with local
saves that reconcile on reconnect; an unavailable matchmaker shows a clean retry
state, never a hang; and telemetry buffers client-side and flushes on recovery —
because every layer above is built to read a 503 as "retry or queue," not as a
reason to lie.
Where this connects#
- Sideways to the match: Modes & Multiplayer — the connection models (dedicated / listen / LAN / server browser), the PvP and horde modes that queue through this matchmaker, the ranked-season ladder, and the esports surfaces that run on this fleet.
- Sideways to live service: Live Service & Community — the seasonal calendar feeds, persistent world events, workshop marketplace, and faction-rep decay that ride these same services and ledgers.
- Down to the engine:
Persistence, Online Services & Cross-Play
— the encrypted save substrate, the
V5Persistenceledgers and replay determinism, the canonical-account cross-play policy, and the Postgres / Redis / object-storage substrate these services deploy onto. - The feature hub: ../V5_features.md </content>