# Avatars, Nameplates & Spatial Audio ```mermaid classDiagram class AvatarProfile { identityRef appearanceVersion rightsPolicy } class TierVariant { meshLOD animationLOD fallbackRepresentation } class NameplatePolicy { distanceBands privacyMode accessibilityMode } class SpatialAudioState { sourcePosition audiencePolicy captionRef } AvatarProfile --> TierVariant : realizes as AvatarProfile --> NameplatePolicy : disclosed by TierVariant --> SpatialAudioState : co-locates ``` Body, identity disclosure, and voice/audio presence compose without collapsing. Tier degradation can change geometry and animation, while privacy, block/mute, caption, nameplate, and spatial authority remain governed. V3 ("Lilith") is not a feed you scroll; it is a place you _inhabit_. A verified yoga instructor demonstrates an asana to a room of embodied students, a Saraswati artist performs to a stadium, and strangers free-roam a commons — and for any of that to feel like presence rather than a video call, three features have to land together. You need a **body** that is recognizably _yours_ and renders the same on a gaming PC and a phone browser; you need to know **who** the figures around you are without a wall of floating text drowning the room; and you need to **hear** those people and the music where they actually stand. This page covers the three features that deliver that: the **avatar pipeline and customization**, the **avatar nameplate LOD** that keeps a 4 096-seat concert legible, and the **spatial audio** layer that places every voice and music stream in 3D. It is the feature-side companion to the Metaverse Platform group; the architecture-side deep dive lives in [../architecture/avatar-animation-and-audio.md](../architecture/avatar-animation-and-audio.md), and the section hub is [../V3_features.md](../V3_features.md). ## What ships, honestly This layer sits unusually _high_ on the implemented-versus-spec spectrum for a metaverse client: the parts that are hard to get right — the import math, the LOD policy, the audio plans — are real code under test, and the parts that are "pending" are art assets, not logic. - **The nameplate LOD is fully real C++** with a tested density audit, not a design table. `FV3AvatarNameplateLodPolicy` (`V3/ue/Source/V3UI/Public/V3AvatarNameplateLod.h`) implements every rung of the ladder — distance bands, density cap, operator override, stadium crowd aggregation, the Tara badge and the locked AI-persona sponsor label — and `V3.UI.AvatarNameplateLod.DensityAudit` drives a 4 096-seat and a 64-seat audit through it. - **The avatar pipeline is a substantive TypeScript library** (~2 700 lines in `libs/v3/avatar-pipeline/src/index.ts`): a real VRM 1.0 importer with a round-trip identity proof, a 60-bone retarget table, a shared blendshape vocabulary, a validated 32-entry gallery, realm-scoped costume rules, and an Isis provenance bundle on every body. - **Spatial audio is real codec / HRTF / music-sync math** in `libs/v3/spatial-audio/src/index.ts`, plus a positional-voice gateway in `V3/ue/Source/V3Voice` with computed equal-power panning, and a Steam Audio VR profile in `V3/ue/Source/V3Audio`. - **The honest caveats.** The avatar _art_ — the `.vrm` gallery meshes, the MetaHuman masters, the authored proxies — is referenced content, not bytes in this repo. The Resonance / Steam Audio DSP runs inside engine plugins; this code owns the contract and fails loud when the plugin is absent. The voice latency probes are deterministic budget-model harnesses, not live captures. And the "6-channel directional layer" the monolith sketches for stadium music is design intent — the shipped renderers are stereo, HRTF, and first-order ambisonic. Each section below says where its claim is backed. ## The avatar pipeline: one body from two worlds V3 accepts avatars from two very different art sources and normalizes both to a single runtime identity: the **Oshun 60-bone skeleton** plus a shared blendshape vocabulary. That normalization is what lets the _same_ person walk from a high-fidelity UE5 client into a phone browser and stay recognizably themself. ### Sourcing: gallery, Ready-Player-Me, or generated A member gets a body one of three ways, each a real entry point in `avatar-pipeline`: - **The canonical Oshun gallery.** `createOshunCanonicalGallery()` builds exactly **32** base avatars, and `validateOshunCanonicalGallery()` refuses the catalog unless all 32 are `authored`, Isis-signed, and design-review-passed. The gallery is deliberately diverse along three axes the code enumerates: eight **body shapes** (`OSHUN_GALLERY_BODY_SHAPES`: `petite, athletic, curvy, broad, tall, compact, plus, androgynous`), four **age bands** (`young-adult, adult, middle-age, elder`), and eight **tradition presentations** (`tara-yoga-practice, tara-ritual, saraswati-classical, saraswati-folk, commons-contemporary, commons-devotional, diaspora-fusion, interfaith-neutral`) — so the starter set covers real body-shape, age, and cultural range rather than one default avatar. - **Ready-Player-Me import.** `ReadyPlayerMeAvatarPayload` / `validateReadyPlayerMePayload()` bridge an external RPM avatar into the pipeline, with a render-ready validation gate, so a member who already has a cross-app avatar can bring it. - **Generated avatars.** Text-to-avatar and generative-costume outputs are first-class but _gated_: the provenance bundle carries an `AvatarProvenanceSourceKind` of `generated-avatar` / `generated-costume` plus a `modelCardId` and `promptHash`, and generated costume requests pass an `isis-editorial:generative-costume:v1` review gate before they can be worn. Whatever the source, the body lands in one canonical format: **VRM 1.0 with glTF 2.0 extensions**, overlaid with Oshun-specific keys for tradition costuming, ritual implements, and concert props. ### The skeleton and the round-trip guarantee VRM is canonical for community avatars, so the importer has to be trustworthy. `importVrm1Document()` and `parseVrm1Json()` (`libs/v3/avatar-pipeline/src/index.ts:843`) parse the glTF + `VRMC_vrm` extension into an `ImportedVrm1Avatar`; `createOshun60RetargetTable()` (`:920`) binds it onto the 60-entry `OSHUN_60_BONE_NAMES` skeleton (`:163` — `root → pelvis → spine → arms / hands / finger chains / legs + IK targets`) and records any `missingRequiredBones`. The part that matters for trust is the **round-trip proof**: `roundTripImportedVrm1Avatar()` re-exports the avatar and `verifyVrm1IdentityRoundTrip()` checks that the bind survives serialization — identity is _verified_, not assumed, so an import that silently lost a bone is caught rather than shipped. The heavier retarget machinery (reference-animation generation and regression scoring against a canonical pose set) lives in `avatar-pipeline/src/retarget/vrm-to-oshun.ts`. The native side mirrors this: `BuildOshun60Table()` in `V3/ue/Source/V3Avatar/Private/V3MetaHumanRetarget.cpp` binds the same 60 Oshun bones to their MetaHuman counterparts, so the two renderers agree on the rig. ### Expression: one face vocabulary everywhere Customization is not only the body — it is the face. Both source types bind to a single blendshape vocabulary so an expression authored once plays everywhere: `OSHUN_VISEME_NAMES` (a 15-phoneme viseme set, `:290`), `OSHUN_EMOTION_BLENDSHAPES` (eight affects, `:308`), `OSHUN_GAZE_BLENDSHAPES` (three axes, `:319`), and `OSHUN_BROW_BLENDSHAPES` (`browUp, browDown`, `:325`). `createOshunBlendshapeMap()` (`:1089`) projects an imported avatar's expression bindings onto that vocabulary. Because the vocabulary is shared, a chat-tone-derived smile or a TTS-derived viseme means the same thing on a MetaHuman face and a lightweight VRM proxy face — the runtime that actually drives it at 60 Hz is `psyche-3d`, covered in the architecture companion. ### Costume: re-dress without re-loading, scoped by realm Wardrobe is the most-used customization surface, and V3 builds it on **material slots, not mesh swaps**, so a member re-costumes instantly without re-importing the avatar: `swapCostumeVariantsWithoutReload()` (`:1598`) hot-swaps wardrobe in place. Costuming is **realm-aware**: `AVATAR_COSTUME_SLOT_NAMES` (`:382`) and `REALM_COSTUME_SLOT_POLICIES` (`:400`) encode which slots each realm permits — `tara` (yoga / ritual attire), `saraswati` (stage outfits with stage-only effects), `commons` (everyday) — and `lilithSafetyCostumeRuleCheck()` enforces the per-realm safety rules, so a costume that is fine on a concert stage is not automatically allowed in a beginners' class. Because a premium persona's MetaHuman and its VRM proxy share the same material-slot identifiers, a per-realm wardrobe rule applies identically across both renderers. ### Provenance and the premium dual-authoring gate Every avatar carries an Isis provenance bundle — `AvatarProvenanceBundle` with a `sourceKind`, an `isis:ed25519:` signature, and (for generated content) a model card and prompt hash — so identity and origin travel _with_ the body and can be inspected later. The high-fidelity path adds a real release gate: verified personas ship **two parallel assets**, a MetaHuman master for Tier-1 and a hand-authored VRM proxy for the web, and `avatar-pipeline/src/premium/` owns that workflow — `dual-authoring.ts` binds the pair, `likeness-drift.ts` is the gate that an editorial owner uses to reject a proxy that drifts too far from the master, `saraswati-ga-personas.ts` enumerates the GA performer personas, and `tara-instructor-opt-in.ts` models the instructor opt-in queue. The honest line: the _pipeline_ is real and tested; the avatars it gates are content. One more identity rule lives outside this library — the **24-hour avatar-swap cooldown** (anti-harassment) is a `swapCooldownUntil` field on the `V3AvatarBinding` persistence model, bound to the V1 user id and a consent record; see [./identity-safety-provenance-foundations.md](./identity-safety-provenance-foundations.md). ## Avatar nameplate LOD: a legible room at any density Nameplates render above every visible avatar — display name, a badge (Tara / verified / AI persona / operator), a reputation band, an activity-state ring. In a 256-attendee class or a 4 096-seat concert, drawing every nameplate is both an unreadable wall of text and a draw-call disaster. V3's answer is a real LOD policy — and because it is **pure logic with no content dependency**, it is one of the most completely-shipped features on this page: `FV3AvatarNameplateLodPolicy::Evaluate()` (`V3/ue/Source/V3UI/Private/V3AvatarNameplateLod.cpp`). ### The ladder, as code `Evaluate()` takes a context (distance, how many nameplates are in view and this one's rank among them, whether the viewer is looking at and in eye-contact range, hover/focus, and a set of mode flags) and returns one of five render modes from `EV3AvatarNameplateLodMode`, each with a fixed, validated draw-call cost: | Situation | Render mode | Renders | Draw calls | | ------------------------------------------------- | ---------------- | ---------------------------------------------- | ---------- | | ≤ 5 m, looking-at, eye-contact class | `Full` | name + badge + reputation band + activity ring | 4 | | ≤ 15 m | `NameAndBadge` | display name + badge | 2 | | ≤ 40 m | `BadgeOnly` | badge icon only | 1 | | > 40 m, or rank > 32 nameplates in view, no hover | `Hidden` | nothing (restored on hover/focus) | 0 | | Stadium crowd band | `CrowdAggregate` | `"+1 200 in section"` aggregate count | 1 | | Solitary cell / reduced-cognitive-load (opt-out) | `Hidden` | nothing, with a recorded policy reason | 0 | The thresholds are named constants — `FullDistanceMeters = 5`, `NameAndBadgeDistanceMeters = 15`, `BadgeOnlyDistanceMeters = 40`, `MaxIndividualNameplatesInView = 32` — and the density cap is precise: a nameplate is hidden only when both the in-view count _and_ this nameplate's rank exceed 32 and it is not hovered, so the 32 nearest stay visible and a far one is restored the instant you hover or focus it. `FV3AvatarNameplateRenderModel::Validate()` makes those costs an invariant rather than a hope: a `Full` plate that does not draw exactly four elements, or a `Hidden` plate that draws anything, or one whose `ModeId` string disagrees with its mode, is rejected. ### Badges that cannot be stripped, and an operator override Two of the badge rules are safety features, not cosmetics, and the validator encodes both. A **Tara verified-instructor badge** can only occupy the `tara-verified-instructor` slot and only when the profile is actually verified. An **AI-persona sponsor label** is _locked_: `ApplyTaraAiPersonaSponsorLabel()` stamps the non-removable `"AI persona — sponsored by "` text, and `Validate()` rejects any attempt to render that persona's plate without the locked label — so an AI persona cannot quietly drop its disclosure. (The same lock is modeled on the policy side in `libs/v3/tara-studio`.) For moderation, operators get a **show-all override**: `bOperatorShowAllOverride` forces `Full` on every plate regardless of distance, density, solitary-cell, or reduced-cognitive-load state, and stamps `bOperatorOverrideApplied` so the override is auditable. ### The density audit is a test `ValidateLaunchDensityAudit()` is what turns "scales to a stadium" into a checked fact. It runs two synthetic crowds through `Evaluate()`. The **stadium** audit puts 4 096 attendees into 16 aggregate cells and asserts **zero** individual nameplates, ≤ 16 total draw calls, and an estimated cost ≤ 1.5 ms. The **class** audit walks 64 attendees by rank and asserts the readable distribution exactly — 4 `Full`, 12 `NameAndBadge`, 16 `BadgeOnly`, and 32 `Hidden`, i.e. never more than 32 visible plates. It also checks the accessibility behaviors: reduced- cognitive-load hides by default, solitary cells always hide, hover restores a hidden plate to `NameAndBadge`, and the operator override shows all. The automation test `V3.UI.AvatarNameplateLod.DensityAudit` (`V3/ue/Source/V3Tests/Private/V3AvatarNameplateLodTests.cpp`) drives the policy and asserts these outcomes, so a regression that made a concert un-legible or leaked individual plates into the crowd band would fail CI. The widget that consumes the model is `UV3AvatarNameplateWidget`. ## Spatial audio: hearing the room Audio is what turns a populated scene into a _room_. The design splits by client tier — Tier-1 UE5 uses MetaSounds with Resonance Audio (Steam Audio as a VR opt-in); Tier-2 web uses Web Audio with an ambisonic-to-stereo fallback — but both tiers share one set of mix-bus semantics, enforced in code: `MixBusKey` (`libs/v3/spatial-audio/src/index.ts:211`) is exactly `voice | music | effects | ambience | accessibility`, and `createDefaultMixBusRuntime()` builds those five buses for both paths, so the per-user HUD mix sliders mean the same thing everywhere. ### Voice: positioned, attenuated, and fail-loud Every avatar carries a voice source. Voice is **Opus 24 kbps mono in 20 ms frames** (`OPUS_24K_MONO_CODEC`, `:20`), and the spatialization is real math, not a stereo gimmick. In the UE gateway (`V3/ue/Source/V3Voice/Private/V3VoiceRealtimeGateway.cpp`), `BuildPositionalMetadata()` derives distance, azimuth (`atan2`) and elevation from the speaker and listener positions, and `ApplyResonanceAudioSpatialization()` computes an **equal-power pan** with distance attenuation (`atten = 1 / (1 + 0.15·distance)`, `LeftGain = atten·√(0.5·(1 − pan))`, `RightGain = atten·√(0.5·(1 + pan))`). The Resonance binding is an honest seam: availability is a literal `FModuleManager::ModuleExists("ResonanceAudio")` check, and the gateway **fails loud** — `"V3Voice must render positional voice with Resonance Audio"` — rather than pretending to spatialize when the plugin is absent. The latency tests (`V3.Voice.RealtimeGateway.SfuOpusSpatializationLatency`, `...RegionalLatencyP95`) are deterministic budget-model harnesses: they compose mouth-to-ear latency analytically against the 80 ms target and validate the _model_ and the panning math, which is the right scope for an automation test — not a live network measurement. ### Music sync: a stadium in lockstep A stadium concert only works if every attendee hears the music together. The sync is NTP-style: `createMusicStreamServerTimestamp()` stamps the stream, `estimateMusicSyncClock()` recovers the server-clock offset from a four-timestamp exchange with round-trip smoothing, and `mapMusicStreamTimestampToPlaybackTarget()` turns that into a client playout target. The guarantee is bounded, not best-effort: `simulateMusicSyncConcert()` (`:1290`) runs a **256-attendee, 60-minute** concert (`MUSIC_SYNC_DEFAULT_ATTENDEE_COUNT = 256`, 5 s sync interval, 120 ms playout buffer) and asserts a **P99 drift ≤ 25 ms** (`MUSIC_SYNC_P99_DRIFT_BUDGET_MS`). The richer "2-channel bed + 6-channel directional layer" the monolith describes for stadium music is design intent; the renderers that ship are stereo, HRTF convolution, and first-order ambisonic. ### Two renderers, and accessibility as a first-class mode The Tier-2 web renderer is real Web Audio: `createHrtfConvolutionPlan()` (`:672`) builds an `AudioWorkletNode → ConvolverNode → GainNode` chain, `decodeFirstOrderAmbisonicToStereo()` (`:729`) is the low-CPU fallback, and `selectSpatialRendererForDevice()` (`:747`) picks between them per device profile, sampling scene occlusion at 5 Hz. Accessibility is not bolted on: an audio-description bed (`createAudioDescriptionBedPlan()`, `:881`) rides the `accessibility` bus at a fixed audible gain, and a flat-stereo `createSpatialAudioOffModePlan()` (`:954`) bypasses ambisonics entirely for pixel-streaming, fallback, native-mobile and VR clients that need the intelligibility. Higher-fidelity VR occlusion is the `V3Audio` Steam Audio profile (`FV3SteamAudioVrRuntimeProfile::BuildDefaultVrProfile()`, `V3/ue/Source/V3Audio/Public/V3SteamAudioVr.h`): 48 kHz, 8 reflection bounces, a 30 Hz occlusion trace, HRTF + occlusion + reflections across the launch XR backends (Quest 3, Vision Pro, PSVR 2, Valve Index, Vive Focus 3), with a voice latency budget summing to 70 ms under the 80 ms ceiling. `Validate()` rejects a profile that drops a spatialization feature or blows the budget (`V3.Audio.SteamAudio.VRSpatialProfile`). As with voice, the DSP runs in the plugin; this code owns the contract and the budget. ## How it connects These three features are the embodiment substrate the rest of the metaverse reads. The avatar body, its blendshape face, and its nameplate are rendered into the rooms, presence digests and multiplayer sessions described in [./world-rooms-presence-multiplayer.md](./world-rooms-presence-multiplayer.md) — the nameplate LOD's "looking-at / eye-contact-class / stadium crowd band" flags come straight from that presence and room state, and the voice/music streams ride that session's wire protocol. Identity binding, the 24-hour swap cooldown, the Isis provenance bundle on every avatar, and the locked AI-persona sponsor label are governed by [./identity-safety-provenance-foundations.md](./identity-safety-provenance-foundations.md). The engine-level internals this page only summarizes — the hand-IK solver, the posture state machine, the Tara eyes-open consent policy, the `psyche-3d` runtime face, and the `aja-pose` webcam grader (`AJA_CANONICAL_30_ASANAS`, a ≥ 92 % per-asana accuracy bar) — are dissected in the architecture companion. The honest line throughout: the import math, the LOD policy, and the codec/spatialization plans are real and tested; the avatar art and the engine-plugin DSP they drive are the content and runtime they gate. The section hub is [../V3_features.md](../V3_features.md). ## Related - [World, Rooms, Presence & Multiplayer](./world-rooms-presence-multiplayer.md) — the room and presence state that feeds nameplate context and carries voice, expression, and music sync across the wire (forthcoming sibling page) - [Identity, Safety & Provenance Foundations](./identity-safety-provenance-foundations.md) — avatar↔account binding, the swap cooldown, Isis provenance, and the AI-persona disclosure lock (forthcoming sibling page) - [Avatar, Animation & Spatial Audio (architecture)](../architecture/avatar-animation-and-audio.md) — the C++/TS internals: hand IK, posture, the eyes-open policy, `psyche-3d`, `aja-pose`, and the voice/Steam-Audio runtimes - The section hub: [../V3_features.md](../V3_features.md)