Generate music, synthesize voices, create sound effects, and integrate with DAWs.
Overview#
The Audio API provides comprehensive audio generation and processing capabilities including AI-powered music composition, voice synthesis, sound design, and integration with professional audio workstations.
Base Path: /v1/audio
Authentication: Bearer token required
Endpoints Summary#
| Method | Path | Description |
|---|---|---|
| POST | /music/generate |
Generate music from text prompt |
| POST | /music/extend |
Extend existing music track |
| POST | /music/remix |
Create remix/variation |
| GET | /music/jobs/{jobId} |
Get music generation status |
| POST | /voice/synthesize |
Text-to-speech synthesis |
| POST | /voice/clone |
Clone voice from samples |
| POST | /voice/batch |
Batch voice synthesis |
| GET | /voice/list |
List available voices |
| POST | /sfx/generate |
Generate sound effects |
| POST | /foley/generate |
Generate foley sounds |
| POST | /ambient/generate |
Generate ambient soundscapes |
| POST | /process/normalize |
Normalize audio levels |
| POST | /process/denoise |
Remove noise from audio |
| POST | /process/stems |
Separate audio into stems |
| POST | /process/spatial |
Apply spatial audio processing |
| POST | /process/convert |
Convert audio format |
| POST | /analyze |
Analyze audio properties |
| POST | /daw/connect |
Connect to DAW |
| GET | /daw/status |
Get DAW connection status |
| POST | /daw/transport |
Control DAW transport |
Music Generation#
Generate Music#
Create music from a text description using AI.
POST /v1/audio/music/generate
Authorization: Bearer <token>
Content-Type: application/json
{
"prompt": "Epic orchestral battle theme with brass fanfares and thundering percussion",
"genre": "orchestral",
"mood": "epic",
"tempo": 140,
"duration": 60,
"key": "D minor",
"instruments": ["strings", "brass", "percussion", "choir"],
"provider": "suno",
"format": "wav",
"sampleRate": 48000,
"projectId": "proj_abc123"
}
Request Parameters:
| Field | Type | Required | Description |
|---|---|---|---|
prompt |
string | Yes | Text description of desired music |
genre |
string | No | Music genre (see genre list) |
mood |
string | No | Emotional mood |
tempo |
number | No | Beats per minute (40-200) |
duration |
number | No | Length in seconds (5-300) |
key |
string | No | Musical key (e.g., "C major", "D minor") |
instruments |
string[] | No | Preferred instruments |
provider |
string | No | AI provider: suno, udio, stable_audio |
format |
string | No | Output format: wav, mp3, flac, ogg |
sampleRate |
number | No | Sample rate: 44100, 48000, 96000 |
stereo |
boolean | No | Stereo output (default: true) |
lyrics |
string | No | Lyrics for vocal tracks |
reference |
string | No | Reference track URL for style matching |
seed |
number | No | Random seed for reproducibility |
Genres:
| Genre | Sub-genres |
|---|---|
orchestral |
epic, romantic, dramatic, ambient |
electronic |
edm, synthwave, ambient, techno, house |
rock |
classic, metal, indie, alternative |
jazz |
smooth, bebop, fusion, swing |
hip_hop |
trap, boom_bap, lo_fi |
pop |
synth_pop, indie_pop, dance_pop |
ambient |
dark, light, nature, space |
cinematic |
action, horror, romance, adventure |
folk |
acoustic, celtic, world |
game |
8bit, chiptune, fantasy, sci_fi |
Response (202 Accepted):
{
"success": true,
"data": {
"jobId": "job_music_abc123",
"status": "pending",
"estimatedTime": 45,
"message": "Music generation queued"
}
}
Extend Music#
Extend an existing music track.
POST /v1/audio/music/extend
Authorization: Bearer <token>
Content-Type: application/json
{
"audioUrl": "https://example.com/track.wav",
"direction": "after",
"duration": 30,
"prompt": "Build to a climactic finish",
"seamless": true
}
Parameters:
| Field | Type | Description |
|---|---|---|
audioUrl |
string | Source audio URL |
direction |
string | before, after, or both |
duration |
number | Extension length in seconds |
prompt |
string | Guidance for extension |
seamless |
boolean | Ensure seamless transition |
Remix Track#
Create a remix or variation of existing music.
POST /v1/audio/music/remix
Authorization: Bearer <token>
Content-Type: application/json
{
"audioUrl": "https://example.com/original.wav",
"style": "lo_fi",
"tempo": 0.8,
"preserveMelody": true,
"addElements": ["vinyl_crackle", "rain"],
"removeElements": ["drums"]
}
Voice Synthesis#
Text-to-Speech#
Convert text to natural-sounding speech.
POST /v1/audio/voice/synthesize
Authorization: Bearer <token>
Content-Type: application/json
{
"text": "Welcome to the kingdom of Eldoria, brave adventurer.",
"voice": "narrator_male_deep",
"language": "en-US",
"speed": 1.0,
"pitch": 0,
"emotion": "warm",
"format": "wav",
"sampleRate": 48000
}
Request Parameters:
| Field | Type | Required | Description |
|---|---|---|---|
text |
string | Yes | Text to synthesize (max 5000 chars) |
voice |
string | Yes | Voice ID or name |
language |
string | No | Language code (default: en-US) |
speed |
number | No | Speed multiplier (0.5-2.0) |
pitch |
number | No | Pitch adjustment (-12 to +12 semitones) |
emotion |
string | No | Emotional tone |
emphasis |
object[] | No | Word-level emphasis markers |
ssml |
boolean | No | Parse text as SSML |
format |
string | No | Output format |
sampleRate |
number | No | Sample rate |
Emotions:
neutral, happy, sad, angry, fearful, surprised, disgusted, warm,
cold, serious, playful, mysterious, authoritative
Response:
{
"success": true,
"data": {
"audioUrl": "https://cdn.yemaya.io/audio/tts_abc123.wav",
"duration": 4.5,
"wordTimings": [
{ "word": "Welcome", "start": 0.0, "end": 0.45 },
{ "word": "to", "start": 0.45, "end": 0.55 },
{ "word": "the", "start": 0.55, "end": 0.65 }
],
"format": "wav",
"sampleRate": 48000
}
}
Clone Voice#
Create a custom voice from audio samples.
POST /v1/audio/voice/clone
Authorization: Bearer <token>
Content-Type: application/json
{
"name": "Character_Hero",
"samples": [
{ "url": "https://example.com/sample1.wav", "transcript": "Sample text one" },
{ "url": "https://example.com/sample2.wav", "transcript": "Sample text two" },
{ "url": "https://example.com/sample3.wav", "transcript": "Sample text three" }
],
"description": "Male hero character, mid-30s, confident tone",
"language": "en-US",
"consent": true
}
Requirements:
- Minimum 3 samples, recommended 10+
- Each sample 10-30 seconds
- Clear audio, minimal background noise
- Varied speech patterns and emotions
Response:
{
"success": true,
"data": {
"voiceId": "voice_custom_abc123",
"name": "Character_Hero",
"status": "training",
"estimatedTime": 300,
"message": "Voice clone training started"
}
}
Batch Synthesis#
Synthesize multiple text segments efficiently.
POST /v1/audio/voice/batch
Authorization: Bearer <token>
Content-Type: application/json
{
"segments": [
{ "id": "line_001", "text": "Hello traveler.", "voice": "npc_merchant" },
{ "id": "line_002", "text": "What brings you here?", "voice": "npc_merchant" },
{ "id": "line_003", "text": "I have rare goods.", "voice": "npc_merchant" }
],
"format": "wav",
"sampleRate": 48000,
"callbackUrl": "https://your-server.com/callback"
}
List Available Voices#
Get all available voices.
GET /v1/audio/voice/list?language=en&category=character
Authorization: Bearer <token>
Response:
{
"success": true,
"data": [
{
"id": "narrator_male_deep",
"name": "Deep Male Narrator",
"category": "narrator",
"language": "en-US",
"gender": "male",
"age": "adult",
"accent": "american",
"style": "narrative",
"preview": "https://cdn.yemaya.io/voices/preview/narrator_male_deep.mp3"
}
]
}
Sound Effects#
Generate SFX#
Create sound effects from descriptions.
POST /v1/audio/sfx/generate
Authorization: Bearer <token>
Content-Type: application/json
{
"prompt": "Heavy sword clash with metallic ring and sparks",
"category": "combat",
"duration": 2.0,
"variations": 3,
"format": "wav"
}
Categories:
| Category | Examples |
|---|---|
combat |
sword clashes, impacts, whooshes |
magic |
spells, portals, enchantments |
nature |
wind, rain, thunder, animals |
mechanical |
engines, gears, machines |
ui |
clicks, notifications, transitions |
footsteps |
various surfaces and speeds |
explosions |
various sizes and types |
vehicles |
cars, ships, aircraft |
creatures |
monster sounds, growls, roars |
ambient |
room tones, city sounds |
Generate Foley#
Create realistic foley sounds for video/animation sync.
POST /v1/audio/foley/generate
Authorization: Bearer <token>
Content-Type: application/json
{
"videoUrl": "https://example.com/scene.mp4",
"categories": ["footsteps", "cloth", "props"],
"autoSync": true,
"surfaceType": "stone",
"intensity": "normal"
}
Generate Ambient#
Create ambient soundscapes.
POST /v1/audio/ambient/generate
Authorization: Bearer <token>
Content-Type: application/json
{
"prompt": "Medieval tavern with fireplace, murmuring crowd, occasional laughter",
"duration": 300,
"loopable": true,
"layers": [
{ "type": "base", "element": "room_tone" },
{ "type": "constant", "element": "fireplace_crackle" },
{ "type": "random", "element": "crowd_murmur", "density": 0.7 },
{ "type": "random", "element": "laughter", "density": 0.1 }
]
}
Audio Processing#
Normalize Audio#
Normalize audio levels.
POST /v1/audio/process/normalize
Authorization: Bearer <token>
Content-Type: application/json
{
"audioUrl": "https://example.com/audio.wav",
"targetLufs": -14,
"truePeak": -1.0,
"method": "loudness"
}
Remove Noise#
Apply noise reduction.
POST /v1/audio/process/denoise
Authorization: Bearer <token>
Content-Type: application/json
{
"audioUrl": "https://example.com/noisy_audio.wav",
"noiseReduction": 0.7,
"preserveVoice": true,
"method": "ai"
}
Separate Stems#
Separate audio into individual stems.
POST /v1/audio/process/stems
Authorization: Bearer <token>
Content-Type: application/json
{
"audioUrl": "https://example.com/song.wav",
"stems": ["vocals", "drums", "bass", "other"],
"format": "wav"
}
Response:
{
"success": true,
"data": {
"stems": {
"vocals": "https://cdn.yemaya.io/stems/abc123_vocals.wav",
"drums": "https://cdn.yemaya.io/stems/abc123_drums.wav",
"bass": "https://cdn.yemaya.io/stems/abc123_bass.wav",
"other": "https://cdn.yemaya.io/stems/abc123_other.wav"
}
}
}
Spatial Audio Processing#
Apply spatial/3D audio processing.
POST /v1/audio/process/spatial
Authorization: Bearer <token>
Content-Type: application/json
{
"audioUrl": "https://example.com/mono.wav",
"format": "ambisonics",
"order": 1,
"position": { "azimuth": 45, "elevation": 0, "distance": 2.0 },
"roomSize": "medium",
"reverb": 0.3
}
Analyze Audio#
Get detailed audio analysis.
POST /v1/audio/analyze
Authorization: Bearer <token>
Content-Type: application/json
{
"audioUrl": "https://example.com/audio.wav",
"analysis": ["loudness", "spectrum", "tempo", "key", "beats", "segments"]
}
Response:
{
"success": true,
"data": {
"duration": 180.5,
"sampleRate": 48000,
"channels": 2,
"bitDepth": 24,
"loudness": {
"integrated": -14.2,
"truePeak": -0.8,
"range": 8.5
},
"tempo": {
"bpm": 128,
"confidence": 0.95
},
"key": {
"key": "A",
"mode": "minor",
"confidence": 0.87
},
"beats": [
{ "time": 0.0, "confidence": 0.9 },
{ "time": 0.469, "confidence": 0.95 }
],
"segments": [
{ "start": 0, "end": 32, "label": "intro" },
{ "start": 32, "end": 96, "label": "verse" }
]
}
}
DAW Integration#
Connect to DAW#
Establish connection to a digital audio workstation.
POST /v1/audio/daw/connect
Authorization: Bearer <token>
Content-Type: application/json
{
"daw": "ableton",
"host": "localhost",
"port": 9000,
"protocol": "osc"
}
Supported DAWs:
| DAW | Protocol |
|---|---|
ableton |
OSC, Max for Live |
logic |
OSC |
protools |
EUCON |
reaper |
OSC, ReaScript |
cubase |
OSC |
flstudio |
MIDI |
Get DAW Status#
Check DAW connection status.
GET /v1/audio/daw/status
Authorization: Bearer <token>
Control Transport#
Control DAW playback.
POST /v1/audio/daw/transport
Authorization: Bearer <token>
Content-Type: application/json
{
"action": "play",
"position": "1.1.1",
"loop": { "start": "1.1.1", "end": "5.1.1" }
}
Actions: play, stop, pause, record, seek, loop, rewind,
forward
WebSocket Events#
Music Generation Progress#
{
"type": "generation:progress",
"jobId": "job_music_abc123",
"jobType": "music_generation",
"progress": 65,
"stage": "rendering",
"message": "Rendering final audio...",
"estimatedSecondsRemaining": 15
}
Error Codes#
| Code | Description |
|---|---|
INVALID_PROMPT |
Music prompt is invalid |
VOICE_NOT_FOUND |
Voice ID does not exist |
AUDIO_TOO_LONG |
Audio exceeds duration limit |
UNSUPPORTED_FORMAT |
Audio format not supported |
CLONE_FAILED |
Voice cloning failed |
DAW_DISCONNECTED |
DAW connection lost |
SYNTHESIS_FAILED |
Speech synthesis failed |
Rate Limits#
| Endpoint | Limit |
|---|---|
| Music generation | 5/minute |
| Voice synthesis | 30/minute |
| SFX generation | 20/minute |
| Processing | 50/minute |
| DAW commands | 100/minute |