Tier-2 Fallback Web Client and Asset Baking

The Tier-2 fallback is V3's "no one gets locked out" path: a Next.js web app (apps/v3/lilith-web-fallback/) that renders the same Lilith Commons, Tara Studio, and Saraswati Stage rooms as the Tier-1 UE5 client — but entirely in the browser, with no native install and no server-side GPU. Where Tier-1 ships a cooked Unreal binary and Pixel Streaming hands a browser a remote UE frame (see ./tier-routing-and-pixel-streaming.md and ./tier1-ue5-client.md), Tier-2 rasterizes locally on the device's own GPU through three.js. It exists because the tier router needs a terminal fallback for devices that can neither install the native client nor sustain a Pixel Streaming WebRTC session: old laptops, locked-down work machines, cellular tablets. The engine library behind it is libs/v3/lilith-engine-web-fallback/ (renderer, physics, parity, steady-FPS, gateway), with audio in libs/v3/spatial-audio/ and the nightly asset transcode plan in tools/v3/fallback-bake/.

The design tension this page documents is honest degradation with verifiable parity. The fallback is not a second implementation of the world that can silently drift from Tier-1; it is a reduced-fidelity rasterizer whose scene geometry, frame budgets, and per-tenant feature envelope are pinned to deterministic signatures shared with the bake pipeline. A WebGL2 device sees fewer particles and baked light instead of Lumen, but it sees the same room, and a Playwright suite asserts the exact parity strings frame-by-frame. This page is the architecture-side companion to the monolith; the hub is ../V3_ARCHITECTURE.md.

What ships, honestly#

The rendering engine, client-side physics, spatial-audio runtime, tenant parity model, steady-FPS gate, realtime-transport negotiation, and offline service worker are real and exercised — by Vitest unit suites and by a multi-browser Playwright suite (apps/v3/lilith-web-fallback/e2e/) that drives the live React-three-fiber canvas and reads back rendered-pixel evidence (collectCanvasEvidence asserts non-trivial luma range and pixel buckets, not just DOM attributes). The bake planner and validator are real, deterministic, and CLI-runnable (tools/v3/fallback-bake/src/cli.ts).

Three claims in the source monolith are not backed by this codebase and are called out where they appear:

• VRM 1.0 avatar rasterization at "≤ 80 K triangles." There is no GLTFLoader, no VRM importer, and no .glb/.vrm load path anywhere in the fallback app or engine (grep returns zero). Avatars in the running scene are procedural primitive clusters — a capsule body plus a sphere head (createAvatarCluster, renderer.ts:506). The 80 K cap exists only as the constant maxAvatarTriangles inside the bake planner (tools/v3/fallback-bake/src/index.ts:135); nothing at runtime loads a baked avatar mesh today. Treat VRM-on-Tier-2 as planned.
• "Same @oshun/multiplayer-protocol Protobuf packets; full presence / voice / interaction parity." The protocol library exists (libs/v3/multiplayer-protocol), but the fallback does not import it (grep returns zero). The fallback's gateway negotiates a transport and returns a simulated echo handshake with a hard-coded RTT; there is no live packet exchange, presence fan-out, or voice in this tree. See Realtime gateway negotiation.
• The bake "produces" glTF/KTX2/Draco/Opus binaries. The bake is a deterministic plan + manifest + glTF-JSON generator. It does not transcode binary geometry, textures, or audio; non-glTF artifacts are emitted as byte-budgeted text placeholders. See What the bake does and does not do.

The web first-load budget is also projected, not measured: V3/WEB_FIRST_LOAD_BUDGET.md records the §66 task as [~] with hand-authored "observed" p95 figures pending a real-device run.

The rendering engine#

Backend negotiation: WebGPU first, WebGL2 floor#

selectLilithFallbackRendererBackend(mode, capabilities) (renderer.ts:176) is the whole policy in one function. WebGPU is preferred; WebGL2 is the hard floor. The three outcomes are explicit:

• requested webgpu and navigator.gpu present → backend webgpu;
• WebGPU requested but unavailable, WebGL2 present → backend webgl2-webgpu-fallback (a "bridge" that keeps the high-power preference but drops to the WebGL2 budget);
• WebGL2 absent → it throws (Lilith Tier-2 renderer requires WebGL2 when WebGPU is unavailable). The fallback fails loud rather than rendering nothing.

Capability detection (detectRendererCapabilities, renderer.ts:338) probes a throwaway <canvas> for a webgl2 context and checks 'gpu' in navigator. The actual WebGPU renderer is dynamically imported from three/webgpu so the WebGL2 path never pays for the WebGPU module; if that import or renderer.init() throws, createRenderer catches and returns the WebGL2 bridge (renderer.ts:354). three.js is catalog: ^0.184.0 (r184) — comfortably above the monolith's "≥ r170" floor — driven through @react-three/fiber ^9.6.1.

Frame budgets and the steady-FPS gate#

Two budget profiles are frozen in lilithFallbackRendererBudgets (renderer.ts:142):

The webgl2-webgpu-fallback bridge resolves to the webgl2 budget via effectiveBudgetMode (renderer.ts:331), so a WebGPU-requested device that quietly fell back is held to 30 fps, not 60 — the budget follows the real backend, not the request. The resolved pixel ratio is min(devicePixelRatio, budget.pixelRatioCap), capping a Retina display at 1.5× (WebGPU) or 1× (WebGL2) to keep fill rate inside budget.

These budgets feed an explicit acceptance gate in steadyFps.ts. validateLilithTier2SteadyFpsSample (steadyFps.ts:60) checks a one-hour sustained sample against eight conditions: minimum and average FPS at or above target, p95 and p99 frame time within the per-mode budget, dropped-frame rate ≤ 0.5 %, and no thermal throttling — each failure pushed as a named reason (steady_fps_under_target, p99_frame_time_over_budget, thermal_throttled). minSustainedMinutes is the literal 60, which is why the rendered canvas exposes data-sustained-minutes="60" and the Playwright suite asserts it.

The reference scene and its parity signature#

The scene each tenant renders is a deterministic, fully procedural composition: a 9×6 floor plane, an 8.4×3.2 horizon, three half-torus arches, two avatar clusters, a 36-point light-guide ring, and one Rapier-predicted prop, under a fixed three-light rig (ambient 0.72, key directional 1.45 at (-3,5,4), accent fill at (4,2,2)). The tenant only swaps an accent colour and scene name; the geometry and lighting are shared. That sharing is encoded as a string: lilithReferenceSceneSignature(tenant) (renderer.ts:164) joins tenant, scene name, geometry signature, lighting signature, object count (19), and material count (7).

This signature is the load-bearing parity contract. The bake pipeline computes the identical string from fallbackReferenceSceneSignature (tools/v3/fallback-bake/src/index.ts:424) — verified identical down to the literal tier2-reference-floor9x6-horizon8x3-arches3-avatars2-guides36-prop1 geometry token (renderer.ts:116 ≡ index.ts:433). Bake and runtime cannot drift without the signature diverging, and the Playwright suite hard-asserts the joined four-tenant parity signature (tara:aja-hud:pose9|commons:baked-lightmap3:niagara96|saraswati:hall:cue-particles384|saraswati:stadium:yemaya-llhls-crowd).

One honest wrinkle: the scene is defined twice. renderer.ts builds it imperatively in createThreeReferenceScene (used by the exported createLilithFallbackThreeRuntime), and the app re-declares the same scene in React-three-fiber as FallbackReferenceScene inside apps/v3/lilith-web-fallback/src/app/FallbackThreeCanvas.tsx. The shipping app mounts the R3F path (via createLilithFallbackRendererHost + a manual R3F createRoot), so the imperative runtime is the test/reference twin. Both are kept honest by the same signature and budgets.

Client-side physics: Rapier prediction#

Physics is real @dimforge/rapier3d-compat WASM, client-prediction only — server stays authoritative (the monolith's framing). simulateRapierPropPickupPrediction (physics.ts:57) runs two parallel Rapier worlds stepped in lockstep: a predicted world that grabs the prop at the local input frame, and an authoritative world that only grabs it once the (simulated) server frame arrives one tick later. Each frame it measures the Euclidean distance between the two prop translations and flags a visibleMispredicted frame when the gap exceeds visibleEpsilonMeters (default 0.025 m) before the authoritative correction lands, tracking the longest such run as maxVisibleMispredictionFrames. The prop is a kinematicPositionBased body on a fixed floor collider, and both worlds are free()d in a finally block so the WASM allocation never leaks (physics.ts:110).

This is a deterministic reconciliation scenario, not a live netcode loop: it proves the prediction-vs-authority math and feeds the canvas the real frame counts (data-rapier-max-misprediction-frames), which the UI surfaces as a "settled" pickup. The full wire protocol and rollback live in Tier-1 / the world server — see ./netcode-protocol-and-physics.md and ./world-server-and-gateway.md.

Spatial audio#

libs/v3/spatial-audio/ is a genuinely substantial DSP library, not a stub. The Tier-2 runtime (startTier2FallbackSpatialAudioRuntime, index.ts:1146) builds a real Web Audio graph through Resonance Audio JS: a ResonanceAudio scene at ambisonic order 1, per-source OscillatorNode → GainNode → ResonanceAudio.Source chains, room geometry and six-surface acoustic materials, and a master gain into AudioDestinationNode. Sources are placed from real HRTF panner state computed from world-space speaker/listener positions (createHrtfPannerStateFromWorldPositions, index.ts:650, with genuine atan2 azimuth/elevation and inverse-distance gain).

The library also ships real, testable algorithms used across tiers, not just the fallback: inverse-distance attenuation (evaluateDistanceAttenuation), a 5 Hz segment-vs-AABB occlusion raycast (evaluateSceneOcclusion), first-order ambisonic-to-stereo decode, an NTP-style music-sync clock estimator (estimateMusicSyncClock with smoothed server-offset), and a per-user mix bus with a ≤ 50 ms HUD-control budget. Two device-adaptive seams matter for the fallback specifically:

• Ambisonic stereo fallback for weak CPUs. selectSpatialRendererForDevice (index.ts:747) downgrades from HRTF convolution to a cheaper ambisonic-stereo path when hardwareConcurrency ≤ 2, or SIMD / AudioWorklet are missing — the low-end CPU path the monolith promises.
• Spatial-audio-off (flat stereo) accessibility mode. buildSpatialAudioOffModeReport (index.ts:971) proves every required client (pixel-streaming, tier-2-fallback, native-mobile, vr) is routed to a centered flat-stereo mix: pan = 0, channel delta ≤ 0.02, gain ≥ 0.85, and HRTF / ambisonics / distance / occlusion all bypassed — with named failure reasons when any client is non-compliant.

Note the codec split: voice is Opus 24 kbps mono (OPUS_24K_MONO_CODEC, benched to a 30 dB SNR target via evaluateOpusWebReceiverBench), while the baked music streams are 96 kbps stereo Opus (the bake's V3FallbackBakeAudioOutput). The concert-sync simulation (simulateMusicSyncConcert) is a documented simulation harness over the real clock math — createSimulated* helpers feed synthetic clock skew into the production estimator and assert a ≤ 25 ms p99 drift budget for 256 attendees over 60 minutes.

Tenant parity envelope#

parity.ts enumerates exactly four runtime profiles in lilithTier2TenantParityProfiles (parity.ts:72), selected by selectLilithTier2TenantParityProfile({ tenant, saraswatiTier }). Each profile declares its fidelity trade, cue events, baked-lightmap atlas budget, reduced Niagara particle budget, simplified-particle cue count, and stadium-preview mode, plus a telemetrySignature:

Only the Saraswati Stadium tier is degraded-preview; the other three are full parity at reduced fidelity, matching the monolith's promise that classes, venues, and hall concerts stay fully reachable while only stadium-scale concerts drop to a server-rendered Yemaya proxy (proxyUrl: 'https://stream.oshun.example/...m3u8' — a placeholder endpoint, not a live stream). The Tara profile is the only one with a pose-guide overlay (jointCount: 9), validated separately by validateAjaTier2CueUxReport (ajaCueValidation.ts:124), which gates the Aja cue on instructor + editorial sign-off, an 80 % cohort clarity rate, WCAG-AA contrast, 44 px touch targets, and no world-space anchoring (Tier-2 cues must be screen-space). See ./avatar-animation-and-audio.md and ./saraswati-stage-pipeline.md.

The asset baking pipeline#

A nightly job (tools/v3/fallback-bake/) consumes a CookedAssetManifest.json emitted from the UE5 staging cook and produces the Tier-2 variant plan. The flow and its parity cross-check:

Inputs, outputs, and budgets#

createV3FallbackBakePlan(manifest, options) (index.ts:154) maps each cooked asset to a fallback output under the prefix <tenant>/<version>/:

• Meshes → glTF 2.0 + Draco. Triangle count is reduced by the Nanite-aware factor 0.32 (Nanite meshes) or 0.58 (non-Nanite), clamped to [256, 80 000]. Byte budgets are derived (gltf = triangles × 28, draco = triangles × 6).
• Materials → simplified PBR KTX2, with virtualTextureRemoved / parallaxRemoved flags recording exactly which UE expressions were dropped; texture budget = sourceBytes × 0.42.
• Lightmaps → KTX2 at sourceBytes × 0.38 (the Lumen-replacement baked light).
• Audio → 96 kbps stereo Opus, budgeted at durationSeconds × 96 000 / 8.

Every output is wrapped in an artifact with a sha256- integrity tag, an s3://oshun-v3-fallback-assets/... URI, a content type, and a byte budget. The default renderer mode is per-tenant — saraswati-stage bakes for webgl2 (30 fps), everything else for webgpu (60 fps) — matching tenantScenes.ts.

The validation gate#

validateV3FallbackBakePlan (index.ts:295) fails the bake closed on any of: a missing artifact kind (all six of gltf/draco/ktx2-texture/ktx2-lightmap/ opus-audio/manifest must be present); a non-nightly schedule; total estimated venue load over the 30 MB budget; a failed render round-trip; any mesh over 80 K triangles; any non-simplified material; or audio that is not 96 kbps stereo. The render round-trip (createRenderRoundTrip, index.ts:398) estimates frame cost as 3.4 + totalTriangles/20 000 + venueLoadMb/6 and passes only when that is ≤ the mode's maxRenderCostMs — the same 16.7/33.4 ms budgets the runtime enforces, so a venue that would blow the frame budget is rejected at bake time rather than discovered on a user's device. The 30 MB venue budget is the same number the commons-venue parity profile carries (venueBudgetMb: 30).

What the bake does and does not do#

This is the most important honesty boundary in the pipeline. createV3FallbackBakeFiles (index.ts:338) emits the manifest as full JSON, the glTF entries as real glTF-2.0 documents (with KHR_draco_mesh_compression + KHR_texture_basisu in extensionsUsed and lightmap/texture paths in extras), but every other artifact — Draco buffers, KTX2 textures, KTX2 lightmaps, Opus audio — is written as a byte-budgeted text placeholder of the form "<kind>:<integrity>:<byteBudget>". The bake is a deterministic planning and manifest stage: it decides the LODs, the byte budgets, the S3 layout, the parity, and the render feasibility, and it proves all of that with a passing validator. It does not invoke a Draco encoder, a Basis/KTX2 transcoder, or an Opus encoder. Wiring those real transcoders behind the plan is the remaining work; the plan they would satisfy is fully specified and gated.

Realtime gateway negotiation#

gateway.ts selects a realtime transport per browser and network, mirroring how Tier-1 reaches the Lilith gateway. selectLilithGatewayTransportPlan (gateway.ts:83) prefers WebTransport over QUIC (/v3/realtime/webtransport) when the browser supports it and UDP is allowed; otherwise it falls back to a WebRTC data channel over TURN/TCP (/v3/realtime/webrtc, label oshun-v3-realtime) and records why it fell back (udp-blocked, or <browser>-webtransport-unavailable). The browser matrix is explicit: Chrome and Edge advertise WebTransport; Safari does not and always lands on the WebRTC data channel. If neither transport is available it throws.

The honest limit: connectLilithGatewayWithFallback (gateway.ts:119) resolves the plan, awaits a microtask, and returns { connected: true, echoPayload, handshakeRttMs } with the RTT hard-coded (24 ms for WebTransport, 42 ms for WebRTC). It is a transport-selection planner plus a simulated handshake, surfaced in the UI as a "linked" gateway badge — there is no live socket, no @oshun/multiplayer-protocol packet, and no presence/voice exchange in this tree. The real gateway connection belongs to ./world-server-and-gateway.md.

Offline and cold join: the service worker#

apps/v3/lilith-web-fallback/public/v3-tier2-sw.js is a real service worker with a stale-while-revalidate strategy: navigations are served from the oshun-v3-tier2-cache-v1 cache first and revalidated from the network; the install step pre-caches the three tenant routes plus the Commons reference image. A V3_CACHE_PROBE message handler answers over a MessageChannel with which required assets are cached, which is how the e2e service-worker-cache.spec.ts verifies offline readiness. serviceWorkerCache.ts records the cold/warm cold-join budgets (5000 ms cold, 3000 ms warm) and the observed figures (4615 / 2380 ms) — and like the first-load budget, those observed numbers are committed projections, not a measured device run.

Failure modes and edge cases#

• No WebGL2 at all → the renderer throws on construction rather than presenting a blank canvas; the shell can then route the user back toward Pixel Streaming or a native install (the monolith's stadium-tier upgrade nudge).
• WebGPU requested, init fails → silent drop to the WebGL2 bridge at the 30 fps budget; the canvas reports data-renderer-backend="webgl2-webgpu-fallback" so telemetry can see the degrade.
• Rapier WASM init failure → the pickup effect catches and sets pickupState = 'error' (FallbackThreeCanvas.tsx:215) instead of crashing the frame loop; rendering continues without prediction.
• Bake regression → any drift in triangle budget, venue size, audio codec, or render feasibility fails the validator closed, blocking promotion of the fallback variant.
• AudioContext suspended (autoplay policy) → the runtime awaits context.resume() and the start handler catches failures into a non-fatal error audio status.
• Unknown tenant route → [tenant]/page.tsx calls notFound(); only the three enumerated routes resolve.

How it connects to neighbouring systems#

Tier-2 is the terminal node of the routing ladder (./tier-routing-and-pixel-streaming.md), consuming the same staging cook that feeds Tier-1 (./tier1-ue5-client.md) through the bake. Its parity profiles are the web-side shadow of the tenant pipelines (./saraswati-stage-pipeline.md, ./tara-classes-aja-and-commons.md) and the broader authoring flow (./authoring-and-content-pipeline.md). Its accessibility and localization surfaces (reduced motion, color-vision palettes, photosensitive-safe, one-handed mobile, single-switch nav, 18-locale RTL) and its projected first-load/steady-FPS budgets are governed under ./observability-performance-security-and-launch.md, and its Commons tenancy bindings into V1 records connect through ./data-tenancy-and-residency.md and ./v1-integration-and-identity-bridge.md. For the system map and module inventory, start at ./product-promise-and-architecture.md and ./subsystem-glossary-and-layout.md.

Related#

• ../V3_ARCHITECTURE.md — the architecture hub
• ./tier-routing-and-pixel-streaming.md and ./tier1-ue5-client.md — the tiers above this one
• ./netcode-protocol-and-physics.md and ./world-server-and-gateway.md — the real wire protocol and gateway the fallback only simulates
• ./avatar-animation-and-audio.md — the VRM / MetaHuman dual-authoring path Tier-2 rasterizes (procedurally, today)
• ./observability-performance-security-and-launch.md — budgets, accessibility, and launch gates