Eve library

Audio API

Guide

Revision 1

On this page

Generate music, synthesize voices, create sound effects, and integrate with DAWs.

Overview#

The Audio API provides comprehensive audio generation and processing capabilities including AI-powered music composition, voice synthesis, sound design, and integration with professional audio workstations.

Base Path: /v1/audio

Authentication: Bearer token required

Endpoints Summary#

Method Path Description
POST /music/generate Generate music from text prompt
POST /music/extend Extend existing music track
POST /music/remix Create remix/variation
GET /music/jobs/{jobId} Get music generation status
POST /voice/synthesize Text-to-speech synthesis
POST /voice/clone Clone voice from samples
POST /voice/batch Batch voice synthesis
GET /voice/list List available voices
POST /sfx/generate Generate sound effects
POST /foley/generate Generate foley sounds
POST /ambient/generate Generate ambient soundscapes
POST /process/normalize Normalize audio levels
POST /process/denoise Remove noise from audio
POST /process/stems Separate audio into stems
POST /process/spatial Apply spatial audio processing
POST /process/convert Convert audio format
POST /analyze Analyze audio properties
POST /daw/connect Connect to DAW
GET /daw/status Get DAW connection status
POST /daw/transport Control DAW transport

Music Generation#

Generate Music#

Create music from a text description using AI.

http
POST /v1/audio/music/generate
Authorization: Bearer <token>
Content-Type: application/json

{
  "prompt": "Epic orchestral battle theme with brass fanfares and thundering percussion",
  "genre": "orchestral",
  "mood": "epic",
  "tempo": 140,
  "duration": 60,
  "key": "D minor",
  "instruments": ["strings", "brass", "percussion", "choir"],
  "provider": "suno",
  "format": "wav",
  "sampleRate": 48000,
  "projectId": "proj_abc123"
}

Request Parameters:

Field Type Required Description
prompt string Yes Text description of desired music
genre string No Music genre (see genre list)
mood string No Emotional mood
tempo number No Beats per minute (40-200)
duration number No Length in seconds (5-300)
key string No Musical key (e.g., "C major", "D minor")
instruments string[] No Preferred instruments
provider string No AI provider: suno, udio, stable_audio
format string No Output format: wav, mp3, flac, ogg
sampleRate number No Sample rate: 44100, 48000, 96000
stereo boolean No Stereo output (default: true)
lyrics string No Lyrics for vocal tracks
reference string No Reference track URL for style matching
seed number No Random seed for reproducibility

Genres:

Genre Sub-genres
orchestral epic, romantic, dramatic, ambient
electronic edm, synthwave, ambient, techno, house
rock classic, metal, indie, alternative
jazz smooth, bebop, fusion, swing
hip_hop trap, boom_bap, lo_fi
pop synth_pop, indie_pop, dance_pop
ambient dark, light, nature, space
cinematic action, horror, romance, adventure
folk acoustic, celtic, world
game 8bit, chiptune, fantasy, sci_fi

Response (202 Accepted):

json
{
  "success": true,
  "data": {
    "jobId": "job_music_abc123",
    "status": "pending",
    "estimatedTime": 45,
    "message": "Music generation queued"
  }
}

Extend Music#

Extend an existing music track.

http
POST /v1/audio/music/extend
Authorization: Bearer <token>
Content-Type: application/json

{
  "audioUrl": "https://example.com/track.wav",
  "direction": "after",
  "duration": 30,
  "prompt": "Build to a climactic finish",
  "seamless": true
}

Parameters:

Field Type Description
audioUrl string Source audio URL
direction string before, after, or both
duration number Extension length in seconds
prompt string Guidance for extension
seamless boolean Ensure seamless transition

Remix Track#

Create a remix or variation of existing music.

http
POST /v1/audio/music/remix
Authorization: Bearer <token>
Content-Type: application/json

{
  "audioUrl": "https://example.com/original.wav",
  "style": "lo_fi",
  "tempo": 0.8,
  "preserveMelody": true,
  "addElements": ["vinyl_crackle", "rain"],
  "removeElements": ["drums"]
}

Voice Synthesis#

Text-to-Speech#

Convert text to natural-sounding speech.

http
POST /v1/audio/voice/synthesize
Authorization: Bearer <token>
Content-Type: application/json

{
  "text": "Welcome to the kingdom of Eldoria, brave adventurer.",
  "voice": "narrator_male_deep",
  "language": "en-US",
  "speed": 1.0,
  "pitch": 0,
  "emotion": "warm",
  "format": "wav",
  "sampleRate": 48000
}

Request Parameters:

Field Type Required Description
text string Yes Text to synthesize (max 5000 chars)
voice string Yes Voice ID or name
language string No Language code (default: en-US)
speed number No Speed multiplier (0.5-2.0)
pitch number No Pitch adjustment (-12 to +12 semitones)
emotion string No Emotional tone
emphasis object[] No Word-level emphasis markers
ssml boolean No Parse text as SSML
format string No Output format
sampleRate number No Sample rate

Emotions:

neutral, happy, sad, angry, fearful, surprised, disgusted, warm, cold, serious, playful, mysterious, authoritative

Response:

json
{
  "success": true,
  "data": {
    "audioUrl": "https://cdn.yemaya.io/audio/tts_abc123.wav",
    "duration": 4.5,
    "wordTimings": [
      { "word": "Welcome", "start": 0.0, "end": 0.45 },
      { "word": "to", "start": 0.45, "end": 0.55 },
      { "word": "the", "start": 0.55, "end": 0.65 }
    ],
    "format": "wav",
    "sampleRate": 48000
  }
}

Clone Voice#

Create a custom voice from audio samples.

http
POST /v1/audio/voice/clone
Authorization: Bearer <token>
Content-Type: application/json

{
  "name": "Character_Hero",
  "samples": [
    { "url": "https://example.com/sample1.wav", "transcript": "Sample text one" },
    { "url": "https://example.com/sample2.wav", "transcript": "Sample text two" },
    { "url": "https://example.com/sample3.wav", "transcript": "Sample text three" }
  ],
  "description": "Male hero character, mid-30s, confident tone",
  "language": "en-US",
  "consent": true
}

Requirements:

  • Minimum 3 samples, recommended 10+
  • Each sample 10-30 seconds
  • Clear audio, minimal background noise
  • Varied speech patterns and emotions

Response:

json
{
  "success": true,
  "data": {
    "voiceId": "voice_custom_abc123",
    "name": "Character_Hero",
    "status": "training",
    "estimatedTime": 300,
    "message": "Voice clone training started"
  }
}

Batch Synthesis#

Synthesize multiple text segments efficiently.

http
POST /v1/audio/voice/batch
Authorization: Bearer <token>
Content-Type: application/json

{
  "segments": [
    { "id": "line_001", "text": "Hello traveler.", "voice": "npc_merchant" },
    { "id": "line_002", "text": "What brings you here?", "voice": "npc_merchant" },
    { "id": "line_003", "text": "I have rare goods.", "voice": "npc_merchant" }
  ],
  "format": "wav",
  "sampleRate": 48000,
  "callbackUrl": "https://your-server.com/callback"
}

List Available Voices#

Get all available voices.

http
GET /v1/audio/voice/list?language=en&category=character
Authorization: Bearer <token>

Response:

json
{
  "success": true,
  "data": [
    {
      "id": "narrator_male_deep",
      "name": "Deep Male Narrator",
      "category": "narrator",
      "language": "en-US",
      "gender": "male",
      "age": "adult",
      "accent": "american",
      "style": "narrative",
      "preview": "https://cdn.yemaya.io/voices/preview/narrator_male_deep.mp3"
    }
  ]
}

Sound Effects#

Generate SFX#

Create sound effects from descriptions.

http
POST /v1/audio/sfx/generate
Authorization: Bearer <token>
Content-Type: application/json

{
  "prompt": "Heavy sword clash with metallic ring and sparks",
  "category": "combat",
  "duration": 2.0,
  "variations": 3,
  "format": "wav"
}

Categories:

Category Examples
combat sword clashes, impacts, whooshes
magic spells, portals, enchantments
nature wind, rain, thunder, animals
mechanical engines, gears, machines
ui clicks, notifications, transitions
footsteps various surfaces and speeds
explosions various sizes and types
vehicles cars, ships, aircraft
creatures monster sounds, growls, roars
ambient room tones, city sounds

Generate Foley#

Create realistic foley sounds for video/animation sync.

http
POST /v1/audio/foley/generate
Authorization: Bearer <token>
Content-Type: application/json

{
  "videoUrl": "https://example.com/scene.mp4",
  "categories": ["footsteps", "cloth", "props"],
  "autoSync": true,
  "surfaceType": "stone",
  "intensity": "normal"
}

Generate Ambient#

Create ambient soundscapes.

http
POST /v1/audio/ambient/generate
Authorization: Bearer <token>
Content-Type: application/json

{
  "prompt": "Medieval tavern with fireplace, murmuring crowd, occasional laughter",
  "duration": 300,
  "loopable": true,
  "layers": [
    { "type": "base", "element": "room_tone" },
    { "type": "constant", "element": "fireplace_crackle" },
    { "type": "random", "element": "crowd_murmur", "density": 0.7 },
    { "type": "random", "element": "laughter", "density": 0.1 }
  ]
}

Audio Processing#

Normalize Audio#

Normalize audio levels.

http
POST /v1/audio/process/normalize
Authorization: Bearer <token>
Content-Type: application/json

{
  "audioUrl": "https://example.com/audio.wav",
  "targetLufs": -14,
  "truePeak": -1.0,
  "method": "loudness"
}

Remove Noise#

Apply noise reduction.

http
POST /v1/audio/process/denoise
Authorization: Bearer <token>
Content-Type: application/json

{
  "audioUrl": "https://example.com/noisy_audio.wav",
  "noiseReduction": 0.7,
  "preserveVoice": true,
  "method": "ai"
}

Separate Stems#

Separate audio into individual stems.

http
POST /v1/audio/process/stems
Authorization: Bearer <token>
Content-Type: application/json

{
  "audioUrl": "https://example.com/song.wav",
  "stems": ["vocals", "drums", "bass", "other"],
  "format": "wav"
}

Response:

json
{
  "success": true,
  "data": {
    "stems": {
      "vocals": "https://cdn.yemaya.io/stems/abc123_vocals.wav",
      "drums": "https://cdn.yemaya.io/stems/abc123_drums.wav",
      "bass": "https://cdn.yemaya.io/stems/abc123_bass.wav",
      "other": "https://cdn.yemaya.io/stems/abc123_other.wav"
    }
  }
}

Spatial Audio Processing#

Apply spatial/3D audio processing.

http
POST /v1/audio/process/spatial
Authorization: Bearer <token>
Content-Type: application/json

{
  "audioUrl": "https://example.com/mono.wav",
  "format": "ambisonics",
  "order": 1,
  "position": { "azimuth": 45, "elevation": 0, "distance": 2.0 },
  "roomSize": "medium",
  "reverb": 0.3
}

Analyze Audio#

Get detailed audio analysis.

http
POST /v1/audio/analyze
Authorization: Bearer <token>
Content-Type: application/json

{
  "audioUrl": "https://example.com/audio.wav",
  "analysis": ["loudness", "spectrum", "tempo", "key", "beats", "segments"]
}

Response:

json
{
  "success": true,
  "data": {
    "duration": 180.5,
    "sampleRate": 48000,
    "channels": 2,
    "bitDepth": 24,
    "loudness": {
      "integrated": -14.2,
      "truePeak": -0.8,
      "range": 8.5
    },
    "tempo": {
      "bpm": 128,
      "confidence": 0.95
    },
    "key": {
      "key": "A",
      "mode": "minor",
      "confidence": 0.87
    },
    "beats": [
      { "time": 0.0, "confidence": 0.9 },
      { "time": 0.469, "confidence": 0.95 }
    ],
    "segments": [
      { "start": 0, "end": 32, "label": "intro" },
      { "start": 32, "end": 96, "label": "verse" }
    ]
  }
}

DAW Integration#

Connect to DAW#

Establish connection to a digital audio workstation.

http
POST /v1/audio/daw/connect
Authorization: Bearer <token>
Content-Type: application/json

{
  "daw": "ableton",
  "host": "localhost",
  "port": 9000,
  "protocol": "osc"
}

Supported DAWs:

DAW Protocol
ableton OSC, Max for Live
logic OSC
protools EUCON
reaper OSC, ReaScript
cubase OSC
flstudio MIDI

Get DAW Status#

Check DAW connection status.

http
GET /v1/audio/daw/status
Authorization: Bearer <token>

Control Transport#

Control DAW playback.

http
POST /v1/audio/daw/transport
Authorization: Bearer <token>
Content-Type: application/json

{
  "action": "play",
  "position": "1.1.1",
  "loop": { "start": "1.1.1", "end": "5.1.1" }
}

Actions: play, stop, pause, record, seek, loop, rewind, forward


WebSocket Events#

Music Generation Progress#

json
{
  "type": "generation:progress",
  "jobId": "job_music_abc123",
  "jobType": "music_generation",
  "progress": 65,
  "stage": "rendering",
  "message": "Rendering final audio...",
  "estimatedSecondsRemaining": 15
}

Error Codes#

Code Description
INVALID_PROMPT Music prompt is invalid
VOICE_NOT_FOUND Voice ID does not exist
AUDIO_TOO_LONG Audio exceeds duration limit
UNSUPPORTED_FORMAT Audio format not supported
CLONE_FAILED Voice cloning failed
DAW_DISCONNECTED DAW connection lost
SYNTHESIS_FAILED Speech synthesis failed

Rate Limits#

Endpoint Limit
Music generation 5/minute
Voice synthesis 30/minute
SFX generation 20/minute
Processing 50/minute
DAW commands 100/minute