Avatars, Nameplates & Spatial Audio

Body, identity disclosure, and voice/audio presence compose without collapsing. Tier degradation can change geometry and animation, while privacy, block/mute, caption, nameplate, and spatial authority remain governed.

V3 ("Lilith") is not a feed you scroll; it is a place you inhabit. A verified yoga instructor demonstrates an asana to a room of embodied students, a Saraswati artist performs to a stadium, and strangers free-roam a commons — and for any of that to feel like presence rather than a video call, three features have to land together. You need a body that is recognizably yours and renders the same on a gaming PC and a phone browser; you need to know who the figures around you are without a wall of floating text drowning the room; and you need to hear those people and the music where they actually stand. This page covers the three features that deliver that: the avatar pipeline and customization, the avatar nameplate LOD that keeps a 4 096-seat concert legible, and the spatial audio layer that places every voice and music stream in 3D. It is the feature-side companion to the Metaverse Platform group; the architecture-side deep dive lives in ../architecture/avatar-animation-and-audio.md, and the section hub is ../V3_features.md.

What ships, honestly#

This layer sits unusually high on the implemented-versus-spec spectrum for a metaverse client: the parts that are hard to get right — the import math, the LOD policy, the audio plans — are real code under test, and the parts that are "pending" are art assets, not logic.

• The nameplate LOD is fully real C++ with a tested density audit, not a design table. FV3AvatarNameplateLodPolicy (V3/ue/Source/V3UI/Public/V3AvatarNameplateLod.h) implements every rung of the ladder — distance bands, density cap, operator override, stadium crowd aggregation, the Tara badge and the locked AI-persona sponsor label — and V3.UI.AvatarNameplateLod.DensityAudit drives a 4 096-seat and a 64-seat audit through it.
• The avatar pipeline is a substantive TypeScript library (~2 700 lines in libs/v3/avatar-pipeline/src/index.ts): a real VRM 1.0 importer with a round-trip identity proof, a 60-bone retarget table, a shared blendshape vocabulary, a validated 32-entry gallery, realm-scoped costume rules, and an Isis provenance bundle on every body.
• Spatial audio is real codec / HRTF / music-sync math in libs/v3/spatial-audio/src/index.ts, plus a positional-voice gateway in V3/ue/Source/V3Voice with computed equal-power panning, and a Steam Audio VR profile in V3/ue/Source/V3Audio.
• The honest caveats. The avatar art — the .vrm gallery meshes, the MetaHuman masters, the authored proxies — is referenced content, not bytes in this repo. The Resonance / Steam Audio DSP runs inside engine plugins; this code owns the contract and fails loud when the plugin is absent. The voice latency probes are deterministic budget-model harnesses, not live captures. And the "6-channel directional layer" the monolith sketches for stadium music is design intent — the shipped renderers are stereo, HRTF, and first-order ambisonic. Each section below says where its claim is backed.

The avatar pipeline: one body from two worlds#

V3 accepts avatars from two very different art sources and normalizes both to a single runtime identity: the Oshun 60-bone skeleton plus a shared blendshape vocabulary. That normalization is what lets the same person walk from a high-fidelity UE5 client into a phone browser and stay recognizably themself.

Sourcing: gallery, Ready-Player-Me, or generated#

A member gets a body one of three ways, each a real entry point in avatar-pipeline:

• The canonical Oshun gallery. createOshunCanonicalGallery() builds exactly 32 base avatars, and validateOshunCanonicalGallery() refuses the catalog unless all 32 are authored, Isis-signed, and design-review-passed. The gallery is deliberately diverse along three axes the code enumerates: eight body shapes (OSHUN_GALLERY_BODY_SHAPES: petite, athletic, curvy, broad, tall, compact, plus, androgynous), four age bands (young-adult, adult, middle-age, elder), and eight tradition presentations (tara-yoga-practice, tara-ritual, saraswati-classical, saraswati-folk, commons-contemporary, commons-devotional, diaspora-fusion, interfaith-neutral) — so the starter set covers real body-shape, age, and cultural range rather than one default avatar.
• Ready-Player-Me import. ReadyPlayerMeAvatarPayload / validateReadyPlayerMePayload() bridge an external RPM avatar into the pipeline, with a render-ready validation gate, so a member who already has a cross-app avatar can bring it.
• Generated avatars. Text-to-avatar and generative-costume outputs are first-class but gated: the provenance bundle carries an AvatarProvenanceSourceKind of generated-avatar / generated-costume plus a modelCardId and promptHash, and generated costume requests pass an isis-editorial:generative-costume:v1 review gate before they can be worn.

Whatever the source, the body lands in one canonical format: VRM 1.0 with glTF 2.0 extensions, overlaid with Oshun-specific keys for tradition costuming, ritual implements, and concert props.

The skeleton and the round-trip guarantee#

VRM is canonical for community avatars, so the importer has to be trustworthy. importVrm1Document() and parseVrm1Json() (libs/v3/avatar-pipeline/src/index.ts:843) parse the glTF + VRMC_vrm extension into an ImportedVrm1Avatar; createOshun60RetargetTable() (:920) binds it onto the 60-entry OSHUN_60_BONE_NAMES skeleton (:163 — root → pelvis → spine → arms / hands / finger chains / legs + IK targets) and records any missingRequiredBones. The part that matters for trust is the round-trip proof: roundTripImportedVrm1Avatar() re-exports the avatar and verifyVrm1IdentityRoundTrip() checks that the bind survives serialization — identity is verified, not assumed, so an import that silently lost a bone is caught rather than shipped. The heavier retarget machinery (reference-animation generation and regression scoring against a canonical pose set) lives in avatar-pipeline/src/retarget/vrm-to-oshun.ts. The native side mirrors this: BuildOshun60Table() in V3/ue/Source/V3Avatar/Private/V3MetaHumanRetarget.cpp binds the same 60 Oshun bones to their MetaHuman counterparts, so the two renderers agree on the rig.

Expression: one face vocabulary everywhere#

Customization is not only the body — it is the face. Both source types bind to a single blendshape vocabulary so an expression authored once plays everywhere: OSHUN_VISEME_NAMES (a 15-phoneme viseme set, :290), OSHUN_EMOTION_BLENDSHAPES (eight affects, :308), OSHUN_GAZE_BLENDSHAPES (three axes, :319), and OSHUN_BROW_BLENDSHAPES (browUp, browDown, :325). createOshunBlendshapeMap() (:1089) projects an imported avatar's expression bindings onto that vocabulary. Because the vocabulary is shared, a chat-tone-derived smile or a TTS-derived viseme means the same thing on a MetaHuman face and a lightweight VRM proxy face — the runtime that actually drives it at 60 Hz is psyche-3d, covered in the architecture companion.

Costume: re-dress without re-loading, scoped by realm#

Wardrobe is the most-used customization surface, and V3 builds it on material slots, not mesh swaps, so a member re-costumes instantly without re-importing the avatar: swapCostumeVariantsWithoutReload() (:1598) hot-swaps wardrobe in place. Costuming is realm-aware: AVATAR_COSTUME_SLOT_NAMES (:382) and REALM_COSTUME_SLOT_POLICIES (:400) encode which slots each realm permits — tara (yoga / ritual attire), saraswati (stage outfits with stage-only effects), commons (everyday) — and lilithSafetyCostumeRuleCheck() enforces the per-realm safety rules, so a costume that is fine on a concert stage is not automatically allowed in a beginners' class. Because a premium persona's MetaHuman and its VRM proxy share the same material-slot identifiers, a per-realm wardrobe rule applies identically across both renderers.

Provenance and the premium dual-authoring gate#

Every avatar carries an Isis provenance bundle — AvatarProvenanceBundle with a sourceKind, an isis:ed25519: signature, and (for generated content) a model card and prompt hash — so identity and origin travel with the body and can be inspected later. The high-fidelity path adds a real release gate: verified personas ship two parallel assets, a MetaHuman master for Tier-1 and a hand-authored VRM proxy for the web, and avatar-pipeline/src/premium/ owns that workflow — dual-authoring.ts binds the pair, likeness-drift.ts is the gate that an editorial owner uses to reject a proxy that drifts too far from the master, saraswati-ga-personas.ts enumerates the GA performer personas, and tara-instructor-opt-in.ts models the instructor opt-in queue. The honest line: the pipeline is real and tested; the avatars it gates are content. One more identity rule lives outside this library — the 24-hour avatar-swap cooldown (anti-harassment) is a swapCooldownUntil field on the V3AvatarBinding persistence model, bound to the V1 user id and a consent record; see ./identity-safety-provenance-foundations.md.

Avatar nameplate LOD: a legible room at any density#

Nameplates render above every visible avatar — display name, a badge (Tara / verified / AI persona / operator), a reputation band, an activity-state ring. In a 256-attendee class or a 4 096-seat concert, drawing every nameplate is both an unreadable wall of text and a draw-call disaster. V3's answer is a real LOD policy — and because it is pure logic with no content dependency, it is one of the most completely-shipped features on this page: FV3AvatarNameplateLodPolicy::Evaluate() (V3/ue/Source/V3UI/Private/V3AvatarNameplateLod.cpp).

The ladder, as code#

Evaluate() takes a context (distance, how many nameplates are in view and this one's rank among them, whether the viewer is looking at and in eye-contact range, hover/focus, and a set of mode flags) and returns one of five render modes from EV3AvatarNameplateLodMode, each with a fixed, validated draw-call cost:

The thresholds are named constants — FullDistanceMeters = 5, NameAndBadgeDistanceMeters = 15, BadgeOnlyDistanceMeters = 40, MaxIndividualNameplatesInView = 32 — and the density cap is precise: a nameplate is hidden only when both the in-view count and this nameplate's rank exceed 32 and it is not hovered, so the 32 nearest stay visible and a far one is restored the instant you hover or focus it. FV3AvatarNameplateRenderModel::Validate() makes those costs an invariant rather than a hope: a Full plate that does not draw exactly four elements, or a Hidden plate that draws anything, or one whose ModeId string disagrees with its mode, is rejected.

Badges that cannot be stripped, and an operator override#

Two of the badge rules are safety features, not cosmetics, and the validator encodes both. A Tara verified-instructor badge can only occupy the tara-verified-instructor slot and only when the profile is actually verified. An AI-persona sponsor label is locked: ApplyTaraAiPersonaSponsorLabel() stamps the non-removable "AI persona — sponsored by <human>" text, and Validate() rejects any attempt to render that persona's plate without the locked label — so an AI persona cannot quietly drop its disclosure. (The same lock is modeled on the policy side in libs/v3/tara-studio.) For moderation, operators get a show-all override: bOperatorShowAllOverride forces Full on every plate regardless of distance, density, solitary-cell, or reduced-cognitive-load state, and stamps bOperatorOverrideApplied so the override is auditable.

The density audit is a test#

ValidateLaunchDensityAudit() is what turns "scales to a stadium" into a checked fact. It runs two synthetic crowds through Evaluate(). The stadium audit puts 4 096 attendees into 16 aggregate cells and asserts zero individual nameplates, ≤ 16 total draw calls, and an estimated cost ≤ 1.5 ms. The class audit walks 64 attendees by rank and asserts the readable distribution exactly — 4 Full, 12 NameAndBadge, 16 BadgeOnly, and 32 Hidden, i.e. never more than 32 visible plates. It also checks the accessibility behaviors: reduced- cognitive-load hides by default, solitary cells always hide, hover restores a hidden plate to NameAndBadge, and the operator override shows all. The automation test V3.UI.AvatarNameplateLod.DensityAudit (V3/ue/Source/V3Tests/Private/V3AvatarNameplateLodTests.cpp) drives the policy and asserts these outcomes, so a regression that made a concert un-legible or leaked individual plates into the crowd band would fail CI. The widget that consumes the model is UV3AvatarNameplateWidget.

Spatial audio: hearing the room#

Audio is what turns a populated scene into a room. The design splits by client tier — Tier-1 UE5 uses MetaSounds with Resonance Audio (Steam Audio as a VR opt-in); Tier-2 web uses Web Audio with an ambisonic-to-stereo fallback — but both tiers share one set of mix-bus semantics, enforced in code: MixBusKey (libs/v3/spatial-audio/src/index.ts:211) is exactly voice | music | effects | ambience | accessibility, and createDefaultMixBusRuntime() builds those five buses for both paths, so the per-user HUD mix sliders mean the same thing everywhere.

Voice: positioned, attenuated, and fail-loud#

Every avatar carries a voice source. Voice is Opus 24 kbps mono in 20 ms frames (OPUS_24K_MONO_CODEC, :20), and the spatialization is real math, not a stereo gimmick. In the UE gateway (V3/ue/Source/V3Voice/Private/V3VoiceRealtimeGateway.cpp), BuildPositionalMetadata() derives distance, azimuth (atan2) and elevation from the speaker and listener positions, and ApplyResonanceAudioSpatialization() computes an equal-power pan with distance attenuation (atten = 1 / (1 + 0.15·distance), LeftGain = atten·√(0.5·(1 − pan)), RightGain = atten·√(0.5·(1 + pan))). The Resonance binding is an honest seam: availability is a literal FModuleManager::ModuleExists("ResonanceAudio") check, and the gateway fails loud — "V3Voice must render positional voice with Resonance Audio" — rather than pretending to spatialize when the plugin is absent. The latency tests (V3.Voice.RealtimeGateway.SfuOpusSpatializationLatency, ...RegionalLatencyP95) are deterministic budget-model harnesses: they compose mouth-to-ear latency analytically against the 80 ms target and validate the model and the panning math, which is the right scope for an automation test — not a live network measurement.

Music sync: a stadium in lockstep#

A stadium concert only works if every attendee hears the music together. The sync is NTP-style: createMusicStreamServerTimestamp() stamps the stream, estimateMusicSyncClock() recovers the server-clock offset from a four-timestamp exchange with round-trip smoothing, and mapMusicStreamTimestampToPlaybackTarget() turns that into a client playout target. The guarantee is bounded, not best-effort: simulateMusicSyncConcert() (:1290) runs a 256-attendee, 60-minute concert (MUSIC_SYNC_DEFAULT_ATTENDEE_COUNT = 256, 5 s sync interval, 120 ms playout buffer) and asserts a P99 drift ≤ 25 ms (MUSIC_SYNC_P99_DRIFT_BUDGET_MS). The richer "2-channel bed + 6-channel directional layer" the monolith describes for stadium music is design intent; the renderers that ship are stereo, HRTF convolution, and first-order ambisonic.

Two renderers, and accessibility as a first-class mode#

The Tier-2 web renderer is real Web Audio: createHrtfConvolutionPlan() (:672) builds an AudioWorkletNode → ConvolverNode → GainNode chain, decodeFirstOrderAmbisonicToStereo() (:729) is the low-CPU fallback, and selectSpatialRendererForDevice() (:747) picks between them per device profile, sampling scene occlusion at 5 Hz. Accessibility is not bolted on: an audio-description bed (createAudioDescriptionBedPlan(), :881) rides the accessibility bus at a fixed audible gain, and a flat-stereo createSpatialAudioOffModePlan() (:954) bypasses ambisonics entirely for pixel-streaming, fallback, native-mobile and VR clients that need the intelligibility. Higher-fidelity VR occlusion is the V3Audio Steam Audio profile (FV3SteamAudioVrRuntimeProfile::BuildDefaultVrProfile(), V3/ue/Source/V3Audio/Public/V3SteamAudioVr.h): 48 kHz, 8 reflection bounces, a 30 Hz occlusion trace, HRTF + occlusion + reflections across the launch XR backends (Quest 3, Vision Pro, PSVR 2, Valve Index, Vive Focus 3), with a voice latency budget summing to 70 ms under the 80 ms ceiling. Validate() rejects a profile that drops a spatialization feature or blows the budget (V3.Audio.SteamAudio.VRSpatialProfile). As with voice, the DSP runs in the plugin; this code owns the contract and the budget.

How it connects#

These three features are the embodiment substrate the rest of the metaverse reads. The avatar body, its blendshape face, and its nameplate are rendered into the rooms, presence digests and multiplayer sessions described in ./world-rooms-presence-multiplayer.md — the nameplate LOD's "looking-at / eye-contact-class / stadium crowd band" flags come straight from that presence and room state, and the voice/music streams ride that session's wire protocol. Identity binding, the 24-hour swap cooldown, the Isis provenance bundle on every avatar, and the locked AI-persona sponsor label are governed by ./identity-safety-provenance-foundations.md. The engine-level internals this page only summarizes — the hand-IK solver, the posture state machine, the Tara eyes-open consent policy, the psyche-3d runtime face, and the aja-pose webcam grader (AJA_CANONICAL_30_ASANAS, a ≥ 92 % per-asana accuracy bar) — are dissected in the architecture companion. The honest line throughout: the import math, the LOD policy, and the codec/spatialization plans are real and tested; the avatar art and the engine-plugin DSP they drive are the content and runtime they gate. The section hub is ../V3_features.md.

Related#

• World, Rooms, Presence & Multiplayer — the room and presence state that feeds nameplate context and carries voice, expression, and music sync across the wire (forthcoming sibling page)
• Identity, Safety & Provenance Foundations — avatar↔account binding, the swap cooldown, Isis provenance, and the AI-persona disclosure lock (forthcoming sibling page)
• Avatar, Animation & Spatial Audio (architecture) — the C++/TS internals: hand IK, posture, the eyes-open policy, psyche-3d, aja-pose, and the voice/Steam-Audio runtimes
• The section hub: ../V3_features.md