Moremi: Realm Server & Netcode

Moremi is V7's answer to the hardest question a creator republic asks: who runs the authoritative server, and how do thousands of community-hosted worlds stay fair, deterministic, and cheat-resistant without the platform paying to host every one of them? The answer is a Rust realm server — moremi-realm-server — that owns a fixed-timestep authoritative simulation, validates every client-originated event before it touches state, runs untrusted creator logic only as capability-sandboxed WASM, and replicates the result to attested clients over an interest-managed, prediction-and-reconciliation netcode. It is the "Realm Backbone" of Mawu: the process that holds the truth of a realm, sitting behind the gateway and above the persistence layer.

This is the deep dive behind the "Moremi — Realm Server Architecture" and "Netcode and Replication" summaries in the orientation hub ../V7_ARCHITECTURE.md. It is the engineering companion to three sibling pages: the trust model that decides what a realm is allowed to own (./thesis-trust-boundary-and-topology.md), the meshing and persistence layers that let one realm span many sim nodes and survive a crash (./danu-meshing-and-nephthys-persistence.md), and the sandbox that makes community code safe to run at all (./ixchel-modding-runtime-and-wasm-sandbox.md). Read those for the planes Moremi plugs into; read this for Moremi itself.

What ships, honestly#

The realm runtime and the netcode are real, deterministic, and heavily test-driven Rust — but they ship today as verified libraries and eval gates, not as an assembled, client-serving daemon, and the Unreal side is a realm composition actor, not the net driver the monolith's project layout implies. Three honest layers:

• Real and test-covered. The cross-trust message envelope (HMAC-SHA256 security tokens + signed intents + a detached envelope signature), the server-authoritative state mutation path that validates before it writes, the deterministic replay core with a SHA-256 state hash, the dual-tick interest-managed scheduler with a per-node sim-budget gate, the WASM determinism profile on a real Wasmtime runtime, and the entire netcode eval suite — prediction/reconciliation, snapshot interpolation, ack-relative delta compression, lag compensation, delayed spectators, and a packet-loss gate — are all genuine code. There are 30 #[test] cases in libs/v7/realm-protocol/src/lib.rs, 69 in apps/v7/moremi-realm-server/src/lib.rs, and 8 in libs/v7/substrate-bridge/src/lib.rs, and they assert domain correctness (rubber-band distance is exactly zero, hostile events leave the state hash unchanged, a 4,096-entity realm encodes the same per-client bytes as a 512-entity one).
• Library-grade, not a running server. moremi-realm-server's run_service() (src/lib.rs:12462) calls service_contract::run_health_server(&DESCRIPTOR), which binds a TcpListener and answers exactly one route — GET /health — with the descriptor JSON (apps/v7/service_contract.rs:36, :88). There is no accept loop feeding live client packets into a ticking realm, and the crate has no tokio and no QUIC dependency despite the monolith's # Rust + tokio + QUIC layout note — its deps are wasmtime, pubgrub, semver, sha2, and the three V7 crates (Cargo.toml). The simulation is exercised by tests, not by a socket.
• Aspirational in the monolith / corrected here. libs/v7/realm-protocol defines the wire transports (WebTransport / WebRTC / WebSocket) but depends on only hmac and sha2 — there is no real transport here; conformance is proven against a simulated network schedule, not live I/O. And the monolith describes Source/MawuRealm as "client-side realm model, interest-set apply, prediction." In code, MawuRealm is AMawuComposedRealmActor — a Geometry Script lock-file compositor with no networking module at all (described honestly below). The prediction and interest logic live in the Rust eval harness, not in UE.

The Moremi realm server#

moremi-realm-server declares its contract in a single DESCRIPTOR (src/lib.rs:98): owner "Moremi", port 47201, and 121 capability strings spanning server-authoritative-state, dual-tick-loop, fixed-timestep-deterministic-sim, ixchel-resource-lifecycle, golden-replay-ci-gate, the Nàná economy/civic family, and the forge resolver/conflict/compositor. The realm's behaviour is built from resources (the FiveM unit, hardened): an Ixchel layer bundling assets, data, and WASM scripts, declaring its dependency set and required sandbox tier — covered in ./ixchel-modding-runtime-and-wasm-sandbox.md.

The deterministic simulation core#

Every higher-order V7 guarantee — rollback prediction, replay-based crash recovery, replayable creator plugins — rests on a deterministic sim. Moremi pins it: MOREMI_FIXED_TIMESTEP_HZ = 30 (:73) for persistent zones, MOREMI_COMBAT_TICK_HZ = 60 (:74) for combat/instanced zones. The core is run_moremi_deterministic_replay (:9117): it sorts an input log, applies each MoremiSimInput on its exact tick over a fixed tick count, then folds the result into a SHA-256 moremi_state_hash over actor positions and gathered resources. Determinism is not asserted by hope — run_moremi_determinism_smoke (:9151) runs the same log twice and compares hashes, and the golden-replay CI gate (run_moremi_golden_replay_ci_gate, :9307) reproduces a recorded state hash and fails an unrecorded-entropy probe (test golden_replay_ci_gate_fails_unrecorded_entropy_probe, :13030) — a replay that reads wall-clock or unseeded RNG would diverge and trip the gate.

The sandbox half of determinism is the WASM profile. MoremiWasmDeterministicProfile::gameplay_default() (:337) turns on NaN canonicalization, turns off relaxed-SIMD and threads, pre-grows linear memory, and meters gameplay scripts by fuel (MOREMI_IXCHEL_DETERMINISTIC_FUEL_PER_TICK = 8_000, :78) rather than epoch — so a plugin traps at the exact same instruction on every replay. This runs on a real wasmtime = "45.0.0" with the component-model and cranelift features (Cargo.toml), not a mock.

The dual-tick, interest-managed loop#

Moremi does not tick the whole world at one rate. run_moremi_dual_tick_fixture_case (:3622) schedules combat entities at the full 60 Hz and open-world entities at an interest-band rate: MoremiOpenWorldInterestBand::{Cold, Warm, Hot} map to 10 / 20 / 30 Hz via update_hz() (:3482), bounded by MOREMI_OPEN_WORLD_MIN_TICK_HZ = 10 and …MAX = 30 (:75–:76). A client's interest set decides each entity's band, so an avatar nobody is near is simulated at 10 Hz while a hot interaction runs at 30. The loop then sums each entity's simulated cost and checks it against a hard per-node budget — MOREMI_DUAL_TICK_NODE_BUDGET_US_PER_SECOND = 20_000 (:77) — raising MoremiDualTickViolation::SimBudgetExceeded if a node is asked to do more than it can in real time. The test dual_tick_fixture_uses_interest_managed_open_world_update_rates (:13857) pins the exact per-band rates (Hot 30, Warm 20, Cold 10), and dual_tick_fixture_flags_budget_overrun proves the budget gate fires when 700 combat entities are forced onto one node. (Overload spills to Danu's split/merge and time-dilation, not to dropped authority — see the meshing page.)

Cross-trust authority — validation before mutation#

The load-bearing security invariant is in libs/v7/realm-protocol: a client-originated event can never mutate authoritative state without server-side validation, and the sender is assumed hostile. A RealmEnvelope carries a RealmSecurityToken (a platform-issued, HMAC-SHA256-MAC'd binding of a sender netid to a realm audience, :1649), a RealmSignedIntent with issue/expiry stamps, and a detached envelope signature. validate_cross_trust (:1781) checks shape, intent freshness, token subject/audience/expiry/MAC, and the signature, in that order, returning a precise typed error (SecurityTokenMacMismatch, IntentExpired, SignatureMismatch, …) on the first failure.

Only then does apply_client_intent_to_authoritative_state (:1835) run: it re-validates cross-trust, rejects any non-ClientIntent kind, requires the realm ids to match, and requires the token to carry the realm.intent.submit scope (REALM_CLIENT_INTENT_REQUIRED_SCOPE, :82) before it parses the movement command and advances the actor by one deterministic cell. The whole chain is exercised adversarially by run_event_tamper_eval (:2624): it accepts one valid signed intent (state hash changes) and replays a corpus of hostile attempts (forged MAC, wrong audience, expired intent, missing scope), asserting that every hostile attempt leaves the authoritative state hash byte-for-byte unchanged (event_tamper_eval_blocks_hostile_client_events_before_mutation). Money, items, and position are server-owned; a tampered client cannot grant itself any of them.

Netcode and replication#

The client↔realm contract is server-authoritative with client-side prediction plus reconciliation, modelled on the Gambetta/Overwatch lineage. The whole contract lives in libs/v7/realm-protocol/src/lib.rs as a set of deterministic eval harnesses — each one a real algorithm plus a passed() gate that would fail on a stub. The baseline capability list (baseline_realm_capabilities(), :1196) enumerates the twenty guarantees the suite proves.

Prediction and reconciliation#

run_realm_prediction_eval (:1870) walks a timeline of local inputs and server acks. Each predictable input (RealmPredictionAbility::Move) advances the predicted position immediately; a ServerOnlyInteract ability explicitly opts out of local prediction. When an AuthoritativeAck arrives, the harness snaps to the server position, drops acked inputs, and re-applies every still-unacked predictable input — the Overwatch reconciliation step. The gate (RealmPredictionEvalReport::passed, :362) is strict: at the canonical REALM_PREDICTION_EVAL_RTT_MS = 80 (:31) the predicted and authoritative states must converge, max_rubber_band_cells must be exactly 0, there must be a recorded opt-out, and there must be a recorded re-application — a predictor that ignored unacked inputs, or one that rubber-banded, fails.

Snapshot interpolation#

Remote entities render on a delay buffer, not the latest packet. run_realm_interpolation_eval (:2041) interpolates position via Hermite (it consumes velocity, interpolate_available_snapshots) and orientation via quaternion SLERP (angular_distance_radians, :1331), rendering at server_tick − 0.5 behind the newest available snapshot. The buffer is jitter-adaptive: adaptive_buffer_frames = base + |max jitter| (:1370). The gate runs at REALM_INTERPOLATION_EVAL_LOSS_PERCENT = 5 (:37) and REALM_INTERPOLATION_EVAL_JITTER_FRAMES = 2 (:40), and fails if any rendered step exceeds the smoothness thresholds (0.75 units of position, 0.08 rad of orientation) — i.e. it proves the stream stays visually jitter-free under 5% loss and ±2 frames of jitter, or it reports ExcessivePositionJitter / MissingInterpolationBracket.

Ack-relative delta compression#

compress_ack_relative_delta (:2158) is the bandwidth engine. The server keeps a RealmSnapshotRing of exactly 32 snapshots (REALM_DELTA_SNAPSHOT_RING_SIZE, :58) and encodes each update against the last snapshot the client acknowledged, emitting only the dirty fields (Position / Orientation / Velocity) for entities in the client's interest set — entities outside it are never encoded. Payloads pre-fragment at REALM_DELTA_PREFRAGMENT_BYTES = 1_400 (:61, under the 1500 MTU). The default eval (run_default_realm_delta_compression_eval, :2415) proves the property that matters: it compresses a 4,096-entity realm and a 512-entity realm with the same 64-entity interest set and asserts the encoded bytes are identical — per-client bandwidth scales with visible density, not world population — and that the result lands inside the REALM_DELTA_MIN_BANDWIDTH_KBPS = 64 … MAX = 256 (:64–:67) budget. PopulationDependentBandwidth and BandwidthBudgetExceeded are explicit violations.

Lag compensation and delayed spectators#

evaluate_lag_compensated_shot (:2450) rewinds target history by RTT/2 + interpolation_buffer, capped at REALM_LAG_COMPENSATION_REWIND_CAP_MS = 250 (:76) over a one-second history window (…HISTORY_MS = 1_000, :73), then raycasts against the reconstructed position. The eval runs two shots: a legitimate one that must reconstruct a favor-the-shooter hit, and an exploit with absurd RTT that must be blocked by the cap (it falls to capped extrapolation and misses) — both pinned by lagcomp_eval_reconstructs_favor_the_shooter_hit_and_blocks_exploit (:4176).

Spectating is a separate, read-only path. build_delayed_spectator_delta_frame (:2204) reuses the same ack-relative compression but enforces a per-realm RealmSpectatorPolicy whose default is delayed read-only at REALM_SPECTATOR_DEFAULT_DELAY_MS = 30_000 (:49). The eval (run_realm_spectator_stream_eval, :2273) additionally fires a spectator intent at the authoritative state and asserts it is rejected with the state hash unchanged — a spectator can watch the broadcast but can never mutate the realm, and never sees an entity outside its policy-limited interest set.

The wire frame, conformance, and the loss gate#

The transport-neutral unit is RealmWireFrame (:980), carried over one of three transports and tagged with one of six message kinds. Each kind declares its channel and reliability: AuthoritativeDelta is unreliable-sequenced (a newer state supersedes a lost one), while Join, Leave, ClientIntent, InterestSetSubscription, and VoiceSignal are reliable-ordered (:1530). run_default_realm_wire_conformance (:2943) drives the full matrix — required_gateway_transports() (WebTransport, WebRTC, WebSocket; :3443) × required_wire_message_kinds() (six; :3451) — through a network schedule that reorders and drops frames, and asserts that reliable-ordered frames are reassembled in monotonic order with no loss while unreliable deltas may be dropped (realm_wire_conformance_passes_under_reordering_and_loss, :4323; the negative …fails_when_reliable_frame_is_lost, :4350). Capping it, run_default_realm_netcode_loss_gate (:2554) runs the whole suite at two loss levels and gates on the product promise: 5% loss must be playable at full quality, 20% loss must degrade gracefully — both within the bandwidth budget. Because there is no transport dependency in the crate, this is a deterministic simulation of loss, not a live UDP soak; it is honest about being a conformance harness rather than a wire benchmark.

The Unreal side: a composition actor, not a NetDriver#

V7/ue/Source/MawuRealm is not a UNetDriver and does not do prediction. It is AMawuComposedRealmActor (Public/MawuComposedRealmActor.h), whose ApplyLockFile (:22) takes an Ixchel FMawuRealmLockFile — an ordered set of FMawuRealmLayers, each with a Priority, a ContentHash, and a list of FMawuRealmPrimitives — and composes the realm's geometry on the client. Primitives with OperationKind == None render the engine BasicShape; every other kind drives a real UE5 Geometry Script operation (BuildProceduralPrimitive, voxel/CSG/sculpt/UV-project) on a runtime UDynamicMesh, folding triangle/vertex/watertight stats into an FMawuCompositionReport. The module's Build.cs depends only on Core/CoreUObject/Engine/GeometryFramework plus Json/GeometryScriptingCore/DynamicMesh — no networking module. So the lock-file compositor is the on-client realization of the Ixchel compositor's output (the same priority-layering and content-hash model the Rust forge produces).

What is realm-server-shaped on the UE side is the build target. MawuDedicatedRealmServer.Target.cs is a TargetType.Server build that compiles project modules with FPSemanticsMode.Precise (including disabled contraction under Clang), adds /fp:strict /fp:except- on Win64, and defines V7_MAWU_DEDICATED_REALM=1. That controlled-FP posture is the engine-side half of the determinism contract the Rust replay gate enforces; cross-machine reproducibility is admitted by replay hashes, not assumed from compiler flags. The integrated path — a Mawu client speaking the realm wire format to a live moremi-realm-server through the gateway — is not assembled in code yet; the gateway (apps/v7/mawu-gateway, port 47204) is itself a health-only daemon today, purpose "attested client ingress and V6 transport bridge."

Where state actually lives#

Moremi is the authority, not the store. The libs/v7/substrate-bridge crate is how a realm reaches the rest of the platform without re-implementing it: an V7IdentityFirewall projects a platform principal into an opaque, per-realm pseudonymous handle (HMAC over a deployment pepper, :343) so realm code never sees a platform account id; a trust-boundary eval proves platform secrets never reach realm or creator-web-view surfaces; and the EgbeOriFacade (:1276) writes Nàná character memories through the reused V6 Ori event store and reads them back through the V6 operator-read audit path. Durable realm and character state is event-sourced in Nephthys, and one realm becomes many seamless sim nodes through Danu — both detailed in ./danu-meshing-and-nephthys-persistence.md.

Related#

• Thesis, Trust Boundary, and Topology — what a realm is allowed to own, and why Moremi assumes a hostile operator
• Danu Meshing and Nephthys Persistence — many sim nodes as one world, atomic authority handoff, and the source of truth
• Ixchel Modding Runtime and the WASM Sandbox — the resource model and the capability-typed Wasmtime sandbox Moremi hosts
• The orientation hub ../V7_ARCHITECTURE.md