@aphrodite/cinematic-music-engine — Modular adaptive game music system with 18
specialised engines for scoring interactive experiences. All computations are
pure math with zero audio I/O — downstream systems consume mix states to drive
real audio playback.
Architecture#
┌─────────────────────────────────────────────────────────────────────┐
│ CORE ADAPTIVE LAYER │
│ ├── Adaptive Music Engine (state machine, stems, crossfades) │
│ ├── Vertical Orchestrator (layer activation, doubling, dynamics) │
│ └── Horizontal Sequencer (sections, phrases, stingers) │
├─────────────────────────────────────────────────────────────────────┤
│ PARAMETER SYSTEM │
│ └── RTPC Parameters (automation, triggers, blending) │
├─────────────────────────────────────────────────────────────────────┤
│ TIMBRAL SYSTEMS │
│ ├── Palette Engine (genre instruments, chords, scales) │
│ ├── Orchestra Engine (sample library, articulations) │
│ ├── Synths Engine (pads, basses, arps, glitch, sub-bass) │
│ ├── Choir Engine (vocal sections, phonemes, solos) │
│ ├── Percussion Engine (hits, patterns, builds, heartbeat) │
│ └── Ambient Engine (soundscapes, drones, pads, weather) │
├─────────────────────────────────────────────────────────────────────┤
│ SIGNAL & MAPPING │
│ ├── Biometric Engine (HR, HRV, breathing, GSR → music) │
│ ├── Biometric Mixing Engine (ducking, bass, reverb, stereo) │
│ └── Expression Mapping Engine (face, body → ornaments, keys) │
├─────────────────────────────────────────────────────────────────────┤
│ GAME CONTEXT │
│ ├── Game Scoring Engine (events → stingers, phrases, key shifts) │
│ ├── Cue System (event-triggered musical stingers) │
│ ├── Vocal Scoring Engine (pitch, intensity → accompaniment) │
│ ├── Emotional Arc Engine (narrative phases, leitmotifs, fatigue) │
│ ├── Boss Battle Engine (multi-phase, near-death, victory) │
│ └── Combo Engine (hit chains, tier progression, climax) │
├─────────────────────────────────────────────────────────────────────┤
│ ENVIRONMENTAL & MULTIPLAYER │
│ ├── Environment Engine (biomes, underwater, altitude, weather) │
│ └── Multiplayer Engine (team themes, tension, cooperative) │
└─────────────────────────────────────────────────────────────────────┘
Every engine follows a consistent pattern:
- Branded ID types for type safety
- Event constants enum
- Configuration interface with sensible defaults
- Mix state snapshot interface
- Statistics interface
- 4 factory functions:
create*Engine(),createCinematic*(),createAction*(),createAmbient*()
1. Adaptive Music Engine#
Core state machine for adaptive music.
Music States#
| State | Description |
|---|---|
Exploration |
Calm exploration music |
Tension |
Building tension |
Action |
Active gameplay music |
Climax |
Peak intensity |
Resolution |
Winding down |
Stem Types#
Strings, Brass, Woodwinds, Percussion, Choir, Synths, Bass,
Ambience
Transition Types#
| Type | Behaviour |
|---|---|
Immediate |
Hard cut |
Crossfade |
Smooth volume crossfade |
BeatSync |
Transition on next beat |
MeasureSync |
Transition on next measure |
Stinger |
Play stinger between states |
FadeOutFadeIn |
Fade out → silence → fade in |
Key Methods#
| Method | Description |
|---|---|
setState(state) |
Request state transition |
addStem(descriptor) |
Register a musical stem |
removeStem(id) |
Remove a stem |
setStemVolume(id, vol) |
Adjust stem volume |
muteStem(id) / unmuteStem(id) |
Mute/unmute |
queueTransition(transition) |
Queue a scheduled transition |
advanceTempo(deltaMs) |
Tick the tempo clock |
getSnapshot() |
Get current MixSnapshot |
Factories#
createAdaptiveMusicEngine(), createCinematicAdaptiveMusicEngine(),
createActionAdaptiveMusicEngine(), createAmbientAdaptiveMusicEngine()
2. Vertical Orchestrator#
Intensity-driven layer activation with doubling and dynamics.
Key Concepts#
- Layers activate/deactivate based on intensity thresholds (0–1)
- Doublings add octave/unison reinforcement at high intensity
- Automation drives crescendos and diminuendos over time
Layer Activation#
Each layer has a volumeCurve defining activation and full-volume thresholds:
interface LayerVolumeCurve {
activationThreshold: number; // intensity where layer fades in
fullVolumeThreshold: number; // intensity where layer is at max
maxVolume: number;
minVolume: number;
curve: CurveType; // Linear | Exponential | Logarithmic | SCurve
}
Doubling#
| Direction | Effect |
|---|---|
OctaveUp |
Double one octave higher |
OctaveDown |
Double one octave lower |
OctaveBoth |
Double both octaves |
Unison |
Double at same pitch |
Factories#
createVerticalOrchestrator(), createCinematicVerticalOrchestrator(),
createActionVerticalOrchestrator(), createAmbientVerticalOrchestrator()
3. Horizontal Sequencer#
Section-based music sequencing with phrase boundaries and stingers.
Section Tags#
Intro, Verse, Chorus, Bridge, Outro, Transition, Custom
Key Concepts#
- Sections have tempo, time signature, duration (measures), phrase boundaries
- Transition rules define which sections can flow to which, with costs
- Stingers are short musical phrases that play between sections
- Phrase boundaries mark natural transition points within sections
Factories#
createHorizontalSequencer(), createCinematicHorizontalSequencer(),
createActionHorizontalSequencer(), createAmbientHorizontalSequencer()
4. Parameter System (RTPC)#
FMOD/Wwise-inspired real-time parameter control.
Features#
- Parameters: Named values with ranges, smoothing, and interpolation
- Triggers: Fire events when parameters cross thresholds
- Automation: Keyframe curves that animate parameters over time
- Blending: Combine multiple parameter outputs
Trigger Modes#
| Mode | Fires when |
|---|---|
Rising |
Value crosses threshold upward |
Falling |
Value crosses threshold downward |
RisingOrFalling |
Either direction |
Equals |
Value matches threshold |
Range |
Value enters min–max range |
Automation Play Modes#
Once, Loop, PingPong
Blend Modes#
Average, Weighted, Max, Min, Multiply
Factories#
createParameterSystem(), createCinematicParameterSystem(),
createActionParameterSystem(), createAmbientParameterSystem()
5. Palette Engine#
Genre-specific instrumentation, chord progressions, and scales.
Genre Types (10)#
| Genre | Character |
|---|---|
Fantasy |
Orchestral, harp, choir, pentatonic |
SciFi |
Synths, pads, electronic, whole tone |
Horror |
Dissonant strings, phrygian, sparse |
Action |
Brass, percussion, driving rhythms |
Romance |
Strings, piano, warm, major/dorian |
Eastern |
Koto, shamisen, hirajoshi/in-sen/pelog |
Western |
Guitar, harmonica, mixolydian |
Steampunk |
Brass, mechanical, augmented |
Medieval |
Lute, recorder, dorian/aeolian |
Noir |
Jazz, sax, piano, chromatic |
Scale Types (20)#
Major, Minor, Dorian, Phrygian, Lydian, Mixolydian, Aeolian,
Locrian, HarmonicMinor, MelodicMinor, WholeTone, Chromatic,
Pentatonic, PentatonicMinor, Blues, HungarianMinor, Hirajoshi,
InSen, Pelog, Slendro
Mood Types (12)#
Triumphant, Mysterious, Melancholic, Tense, Serene, Heroic,
Ominous, Playful, Romantic, Aggressive, Ethereal, Solemn
Instrument Roles#
Lead, Harmony, Bass, Rhythm, Texture, Accent, Drone, Ornament
Texture Layers (6)#
Foundation, Core, Color, Sparkle, Atmosphere, Rhythmic — each
activates at a different intensity threshold.
Pre-Built Palettes#
FANTASY_PALETTE, SCIFI_PALETTE, HORROR_PALETTE, ACTION_PALETTE,
ROMANCE_PALETTE, EASTERN_PALETTE
Factories#
createPaletteEngine(), createCinematicPaletteEngine(),
createActionPaletteEngine(), createAmbientPaletteEngine()
6. Orchestra Engine#
Orchestral sample library with articulations, velocity layers, and round-robin.
Orchestra Sections (17)#
FirstViolins, SecondViolins, Violas, Cellos, Basses, Flutes, Oboes, Clarinets, Bassoons, FrenchHorns, Trumpets, Trombones, Tuba, Timpani, Harp, Piano, Celesta
Velocity Layers (7)#
pp (Pianissimo), p (Piano), mp (MezzoPiano), mf (MezzoForte), f
(Forte), ff (Fortissimo), fff (Fortississimo)
Articulation Types (14)#
Sustain, Staccato, Legato, Pizzicato, Tremolo, Trill, Spiccato,
ColLegno, Marcato, Harmonics, ConSordino, SulPonticello, SulTasto,
Glissando
Factories#
createOrchestraEngine(), createCinematicOrchestraEngine(),
createActionOrchestraEngine(), createAmbientOrchestraEngine()
7. Synths Engine#
Synthesiser system with pads, bass, arps, glitch effects, and sub-bass.
Oscillator Types#
Saw, Square, Sine, Triangle, Pulse, SuperSaw, WaveTable, Noise,
FM
Arpeggiator Patterns#
Up, Down, UpDown, DownUp, Random, Played, Converge, Diverge
Glitch Types (10)#
Stutter, BitCrush, GrainScatter, TapeStop, Glide, Reverse,
Retrigger, HalfSpeed, DoubleSpeed, VinylScratch
Sub-Bass Modes#
Sustained, Pulsed, Sidechained, RhythmLocked, HapticLinked
Presets#
Pads: WARM_ANALOG_PAD, DARK_PULSE_PAD, ETHEREAL_SHIMMER_PAD,
AGGRESSIVE_REESE_PAD
Arps: CYBERPUNK_ARP, AMBIENT_ARP, TRANCE_GATE_ARP, RETRO_SYNTH_ARP
Factories#
createSynthEngine(), createCinematicSynthEngine(),
createCyberpunkSynthEngine(), createAmbientSynthEngine()
8. Choir Engine#
Vocal section management with phonemes, crescendos, and solos.
Vocal Sections#
Soprano, Alto, Tenor, Bass
Phonemes#
Ah, Oh, Ee, Oo, Mm, Eh, Ih, Uh
Choir Sizes#
Solo, Chamber, Small, Medium, Large, Epic
Expression Types#
Sustained, Crescendo, Decrescendo, Sforzando, Swelling, Tremolo
Factories#
createChoirEngine(), createCinematicChoirEngine(),
createIntimateChoirEngine(), createEpicChoirEngine()
9. Percussion Engine#
Percussion hits, patterns, builds, and heartbeat synchronisation.
Instruments (22)#
Taiko, Timpani, BassDrum, Anvil, SnareDrum, TomHigh, TomMid, TomLow, TomFloor, Djembe, Dhol, Conga, Bongo, CrashCymbal, RideCymbal, HiHat, China, Gong, Shaker, Tambourine, Woodblock, TrianglePerc, Claves, Castanets
Hit Intensities#
Ghost, Soft, Medium, Hard, Accent, Flam, Rimshot
Pattern Styles#
March, WarDrums, Heartbeat, Tribal, Industrial, Ostinato, Shuffle,
Polyrhythm
Build Types#
CymbalRoll, SnareRoll, TimpaniRoll, TomEscalation, TaikoEscalation,
Crescendo
Factories#
createPercussionEngine(), createCinematicPercussionEngine(),
createActionPercussionEngine(), createAmbientPercussionEngine()
10. Ambient Engine#
Soundscape layers, drones, pads, and weather effects.
Soundscape Categories#
Nature: Forest, Ocean, Rain, Wind, Birds, Thunder, River, Cave, Desert, Swamp
Urban: City, Machinery, Crowds, Traffic, Industrial, Subway, Market, Construction
Ethereal: Space, Celestial, Mystical, Void, Crystal, Dreamscape, AncientRuins, DeepSea
Drone Waveforms#
Sine, Triangle, Sawtooth, Square, Noise, Pulse
Harmonic Movement#
Static, Drift, Pulse, Breathe, Evolve, Random
Weather States#
Clear, Cloudy, Rainy, Stormy, Foggy, Snowy, Windy
Factories#
createAmbientEngine(), createCinematicAmbientEngine(),
createActionAmbientEngine(), createMeditativeAmbientEngine()
11. Biometric Engine#
Maps biometric signals to musical parameters.
Biometric Signals#
| Signal | Description | Typical range |
|---|---|---|
HeartRate |
BPM | 40–200 |
HeartRateVariability |
ms between beats | 20–200 |
BreathingRate |
Breaths/min | 8–30 |
GalvanicSkinResponse |
Skin conductance | 0–1 |
SkinTemperature |
Temperature delta | -2–2 |
BloodOxygenation |
SpO2 | 90–100 |
Musical Responses#
Tempo, Intensity, Tension, PhraseLength, DynamicRange,
HarmonicComplexity, RhythmicDensity, OrchestratorWeight
Emotional Dimensions#
Valence, Arousal, Dominance, Tension, Engagement
Calibration#
The engine calibrates to each performer's baseline over a configurable window, then normalises readings against the calibrated range.
Factories#
createBiometricEngine(), createCinematicBiometricEngine(),
createActionBiometricEngine(), createMeditativeBiometricEngine()
12. Biometric Mixing Engine#
Audio mix adjustments driven by biometric data.
Mix Parameters#
| Parameter | Driven by | Effect |
|---|---|---|
| Ducking | Vocal activity | Attenuate music under vocals |
| Bass enhancement | Arousal + breathing | Boost sub/bass at high arousal |
| Reverb depth | Vulnerability (HRV) | Deepen reverb in vulnerable moments |
| Stereo width | Euphoria detection | Widen stereo at peak pleasure |
| Compression | Intensity | Tighten dynamics in intense moments |
Ducking Modes#
Off, Gentle, Standard, Aggressive
Mix Bands (7)#
Sub, Bass, LowMid, Mid, HighMid, Presence, Air
Emotional Intensity Levels#
Calm, Engaged, Elevated, Intense, Peak
Factories#
createBiometricMixEngine(), createCinematicBiometricMixEngine(),
createActionBiometricMixEngine(), createMeditativeBiometricMixEngine()
13. Expression Mapping Engine#
Facial expressions and body movement drive musical ornaments and key changes.
Facial Expressions (10)#
Smile, Frown, Surprise, ClosedEyes, RaisedBrows, Grimace,
Contempt, Neutral, Awe, Concentration
Ornament Types (10)#
GraceNote, Trill, Mordent, Turn, Glissando, Sforzando, Fermata,
Tremolo, Arpeggio, Portamento
Key Modifications#
MajorBrightening, MinorDarkening, ModalShift, ChromaticTension,
DiatonicRelease, EnharmonicReinterpretation
Texture Modes#
Sustained, Rhythmic, Minimalist, Dense, Ethereal, Aggressive,
Sparse
Movement Categories#
HeadTilt, HandGesture, BodySway, Lean, Jump, Stillness, Fidget,
Dance
Factories#
createExpressionMappingEngine(), createCinematicExpressionMappingEngine(),
createActionExpressionMappingEngine(),
createAmbientExpressionMappingEngine()
14. Game Scoring Engine#
Scores game events with musical stingers, phrases, and key/tempo shifts.
Game Event Categories (10)#
Impact, Achievement, Tension, Discovery, Failure, Ambient,
Transition, Social, Combat, Stealth
Musical Response Types (16)#
PercussionAccent, BrassStab, StringTremolo, Fanfare,
HarmonicResolution, Dissonance, RisingLine, DescendingPhrase,
MelodicPhrase, HarpGlissando, MinorKeyShift, MajorKeyShift,
OrchestralSwell, SuspensePad, ChoirAccent, ElectronicStinger
Scoring Priority#
Background, Low, Normal, High, Critical, Override
Factories#
createGameScoringEngine(), createCinematicGameScoringEngine(),
createActionGameScoringEngine(), createAmbientGameScoringEngine()
15. Cue System#
Event-triggered musical stingers with priority, context, and genre support.
Cue Categories (12)#
Victory, Defeat, Discovery, Combat, Stealth, Puzzle, Emotion,
Transition, Alert, Reward, Ambient, Climax
Game Contexts (8)#
Exploration, Combat, Stealth, Puzzle, Cinematic, Menu, Boss,
Peaceful
Stinger Library Genres (7)#
Orchestral, Electronic, Rock, Jazz, Ambient, World, Hybrid
Pre-built libraries: ORCHESTRAL_STINGER_LIBRARY, ELECTRONIC_STINGER_LIBRARY,
ROCK_STINGER_LIBRARY, JAZZ_STINGER_LIBRARY, AMBIENT_STINGER_LIBRARY,
WORLD_STINGER_LIBRARY, HYBRID_STINGER_LIBRARY
Factories#
createCueSystem(), createCinematicCueSystem(), createActionCueSystem(),
createAmbientCueSystem()
16. Vocal Scoring Engine#
Scores performer vocal reactions with harmonic accompaniment.
Vocal Categories#
Speech, Singing, Gasp, Moan, Laughter, Cry, Whisper, Shout,
Silence, Breathing
Harmonic Modes (10)#
Unison, ThirdAbove, ThirdBelow, FifthAbove, Octave, PowerChord,
MajorTriad, MinorTriad, OpenFifth, Cluster
Swell Shapes#
Linear, Exponential, Logarithmic, SCurve, Instant
Features#
- Pitch detection with note/cent tracking
- Intensity smoothing with peak/trend analysis
- Silence detection with configurable threshold
- Tension building from sustained silence
- Harmonic accompaniment generation
Factories#
createVocalScoringEngine(), createCinematicVocalScoring(),
createActionVocalScoring(), createAmbientVocalScoring()
17. Emotional Arc Engine#
Narrative phase tracking with leitmotifs, callbacks, and fatigue detection.
Narrative Phases (9)#
Prelude → Introduction → RisingAction → Development → Climax →
FallingAction → Denouement → Resolution → Coda
Leitmotif Categories#
Character, Emotion, Location, Event, Relationship, Theme
Fatigue Levels#
None, Mild, Moderate, High, Critical
Callback Triggers#
EmotionalPeak, NarrativeMirror, ThematicReturn, PerformerRecognition,
ClimaxBuild, ResolutionEcho
Features#
- Automatic narrative phase progression
- Leitmotif introduction and recall with strength tracking
- Listener fatigue detection from sustained high intensity
- Musical callbacks that reference earlier moments
Factories#
createArcEngine(), createCinematicArcEngine(), createActionArcEngine(),
createAmbientArcEngine()
18. Boss Battle Engine#
Multi-phase boss music with near-death intensity and victory fanfare.
Boss Phases#
Intro → Phase1 → Phase2 → Phase3 → Enrage / Stagger → Defeated →
Victory
Music Layers (8)#
BaseDrums, Strings, Brass, Choir, ElectronicPulse, Percussion,
Woodwinds, SoloInstrument
Bridge Types#
Crescendo, Breakdown, Silence, Buildup, DramaticPause
Near-Death Intensities#
Warning, Critical, LastStand
Features#
- HP-threshold-based phase transitions
- Layer activation/deactivation per phase
- Bridges between phases with configurable types
- Near-death state with heartbeat, tremolo, overlay
- Victory fanfare on boss defeat
Factories#
createBossBattleEngine(), createCinematicBossBattleEngine(),
createActionBossBattleEngine(), createAmbientBossBattleEngine()
19. Combo Engine#
Musical response to hit chains with tier progression and climax.
Combo Tiers (7)#
None → Bronze → Silver → Gold → Platinum → Diamond → Master
Combo Instrument Layers (8)#
BaseRhythm, Percussion, BassLine, Strings, Brass, Choir,
LeadMelody, FullOrchestra
Features#
- Count-based tier progression with configurable thresholds
- Per-tier layer activation, tempo multiplier, pitch raise
- Climax state at maximum combo
- Combo break severity levels:
Minor,Normal,Major,Total - Streak bonus events
Factories#
createComboEngine(), createCinematicComboEngine(),
createActionComboEngine(), createAmbientComboEngine()
20. Environment Engine#
Biome-themed music with underwater, altitude, and weather acoustics.
Biome Types (12)#
Forest, Desert, Tundra, Ocean, Mountain, Swamp, Volcanic,
Meadow, Cave, Urban, Ruins, Void
Acoustic Spaces (8)#
OpenAir, SmallRoom, LargeHall, Cave, Cathedral, Tunnel,
Underwater, Forest
Altitude Zones (6)#
SeaLevel, Lowland, Highland, Alpine, Peak, Stratospheric
Features#
- Per-biome theme definitions (scale, root, instruments, tempo, warmth)
- Acoustic profiles with reverb, reflections, frequency filtering
- Weather profiles affecting percussion, strings, dynamics
- Underwater effects (low-pass filter, attenuation, bubble layer)
- Altitude effects (orchestral thinning, wind, percussion dampening)
Factories#
createEnvironmentEngine(), createCinematicEnvironmentEngine(),
createActionEnvironmentEngine(), createAmbientEnvironmentEngine()
21. Multiplayer Engine#
Team-based competitive and cooperative music.
Competitive Modes#
FreeForAll, TeamVsTeam, Cooperative, BattleRoyale, Asymmetric
Team Roles#
Ally, Enemy, Neutral, Spectator
Momentum Directions#
Winning, Losing, Contested, Dominant, Desperate
Orchestra Scales#
Solo, Chamber, Ensemble, Orchestra, FullSymphony
Cooperative Harmony Types#
Unison, Thirds, Fifths, Octaves, FullChord, Counterpoint
Features#
- Per-team theme definitions (root, scale, instruments, aggression)
- Score-differential-driven tension system
- Momentum tracking with streak detection
- Crowd-size-to-orchestra-fullness scaling
- Cooperative harmony events when players collaborate
- Victory/defeat musical sequences
- Clutch moment detection (close score + low time)
Factories#
createMultiplayerEngine(), createCinematicMultiplayerEngine(),
createCompetitiveMultiplayerEngine(), createCooperativeMultiplayerEngine()