The adaptive music system models Maya-style interactive scoring as data: states, substates, sections, layers, playlists, transitions, stingers, ducking rules, and middleware events. It is backend-agnostic, so the same authoring file can drive the internal audio runtime, Wwise, or FMOD.
Authoring Model#
- Use
MusicStatefor broad gameplay contexts: exploration, tension, combat, victory, defeat, and menu. - Use
MusicSubstatefor musical detail such as calm travel, stealth danger, light combat, heavy combat, boss, success, failure, main menu, or pause menu. - Define horizontal
MusicSectionentries for verse, chorus, bridge, low, medium, high, intro, ambient, and outro material. - Define vertical
MusicLayerDefinitionentries for stems such as ambience, harmony, melody, rhythm, percussion, bass, drone, and stings. - Attach
MusicIntensityCurvepoints to map a 0.0-1.0 gameplay intensity value to a preferred section kind, active layer IDs, and layer volume scale.
Synchronization#
Every transition and stinger can choose a SyncBoundary:
immediatestarts as soon as the command arrives.beatwaits for the next beat in the current clip analysis.barwaits for the next bar.phrasewaits for the next phrase usingbeatsPerBar * barsPerPhrase.section_endwaits for the current segment end.
Clips can provide their own BeatAnalysis, or the runtime can create one from
BPM, beats per bar, and bars per phrase. For build tools and editor workflows,
analyzeTempoFromOnsets derives BPM and beat positions from detected onset
times.
Runtime Flow#
- Load a
MusicAuthoringDefinition. - Call
start(state, substate)when gameplay enters the first context. - Call
updateIntensity(value)as combat pressure, stealth danger, speed, or narrative tension changes. - Call
setState(state, substate)to schedule a transition. - Call
update()each audio tick to apply pending transitions and ducking. - Call
triggerStinger(event)for one-shot musical phrases such as pickups, discoveries, achievements, victory, defeat, or danger spotted. - Call
triggerDucking(trigger)for dialogue, explosions, cinematics, and other priority sound events.
Memory Planning#
Each MusicClip and layer chooses a MemoryStrategy:
streamkeeps large loops off the heap and tracks expected stream chunks.preloadloads short or likely-upcoming material.pinkeeps essential menu, ambience, or transition material resident.
The engine builds a MusicMemoryPlan for the current or next state and evicts
large non-pinned preloads when the budget is exceeded. Use streamable combat
loops, preload likely state transitions, and pin only tiny shared stems.
Middleware#
The internal backend records the same high-level commands that native bindings can forward to Wwise or FMOD:
set_statefor the current music state.set_switchfor pending state changes and layer on/off switches.set_parameterfor continuous intensity values.post_eventfor transitions and stingers.load_bankandunload_bankfor future native bank integration.
createWwiseMusicBackend and createFmodMusicBackend provide test doubles with
the correct backend identity. Production adapters should implement
MusicPlaybackBackend and translate commands into the studio middleware API.
Debug Overlay#
getDebugOverlayState() returns the complete HUD/editor payload:
- current state and substate
- smoothed intensity
- tempo, beat, bar, and phrase counters
- current section and active layer IDs
- pending transition target and start time
- ducking volume
- memory budget, loaded clips, streaming clips, and evictions
- active backend kind
Keep the overlay compact in-game: show state/substate, section, layers, beat counter, next transition, and memory warnings first. Put detailed clip lists and middleware event logs in the editor inspector.