# Audio API

Generate music, synthesize voices, create sound effects, and integrate with
DAWs.

## Overview

The Audio API provides comprehensive audio generation and processing
capabilities including AI-powered music composition, voice synthesis, sound
design, and integration with professional audio workstations.

**Base Path:** `/v1/audio`

**Authentication:** Bearer token required

## Endpoints Summary

| Method | Path                  | Description                     |
| ------ | --------------------- | ------------------------------- |
| POST   | `/music/generate`     | Generate music from text prompt |
| POST   | `/music/extend`       | Extend existing music track     |
| POST   | `/music/remix`        | Create remix/variation          |
| GET    | `/music/jobs/{jobId}` | Get music generation status     |
| POST   | `/voice/synthesize`   | Text-to-speech synthesis        |
| POST   | `/voice/clone`        | Clone voice from samples        |
| POST   | `/voice/batch`        | Batch voice synthesis           |
| GET    | `/voice/list`         | List available voices           |
| POST   | `/sfx/generate`       | Generate sound effects          |
| POST   | `/foley/generate`     | Generate foley sounds           |
| POST   | `/ambient/generate`   | Generate ambient soundscapes    |
| POST   | `/process/normalize`  | Normalize audio levels          |
| POST   | `/process/denoise`    | Remove noise from audio         |
| POST   | `/process/stems`      | Separate audio into stems       |
| POST   | `/process/spatial`    | Apply spatial audio processing  |
| POST   | `/process/convert`    | Convert audio format            |
| POST   | `/analyze`            | Analyze audio properties        |
| POST   | `/daw/connect`        | Connect to DAW                  |
| GET    | `/daw/status`         | Get DAW connection status       |
| POST   | `/daw/transport`      | Control DAW transport           |

---

## Music Generation

### Generate Music

Create music from a text description using AI.

```http
POST /v1/audio/music/generate
Authorization: Bearer <token>
Content-Type: application/json

{
  "prompt": "Epic orchestral battle theme with brass fanfares and thundering percussion",
  "genre": "orchestral",
  "mood": "epic",
  "tempo": 140,
  "duration": 60,
  "key": "D minor",
  "instruments": ["strings", "brass", "percussion", "choir"],
  "provider": "suno",
  "format": "wav",
  "sampleRate": 48000,
  "projectId": "proj_abc123"
}
```

**Request Parameters:**

| Field         | Type     | Required | Description                                 |
| ------------- | -------- | -------- | ------------------------------------------- |
| `prompt`      | string   | Yes      | Text description of desired music           |
| `genre`       | string   | No       | Music genre (see genre list)                |
| `mood`        | string   | No       | Emotional mood                              |
| `tempo`       | number   | No       | Beats per minute (40-200)                   |
| `duration`    | number   | No       | Length in seconds (5-300)                   |
| `key`         | string   | No       | Musical key (e.g., "C major", "D minor")    |
| `instruments` | string[] | No       | Preferred instruments                       |
| `provider`    | string   | No       | AI provider: `suno`, `udio`, `stable_audio` |
| `format`      | string   | No       | Output format: `wav`, `mp3`, `flac`, `ogg`  |
| `sampleRate`  | number   | No       | Sample rate: 44100, 48000, 96000            |
| `stereo`      | boolean  | No       | Stereo output (default: true)               |
| `lyrics`      | string   | No       | Lyrics for vocal tracks                     |
| `reference`   | string   | No       | Reference track URL for style matching      |
| `seed`        | number   | No       | Random seed for reproducibility             |

**Genres:**

| Genre        | Sub-genres                             |
| ------------ | -------------------------------------- |
| `orchestral` | epic, romantic, dramatic, ambient      |
| `electronic` | edm, synthwave, ambient, techno, house |
| `rock`       | classic, metal, indie, alternative     |
| `jazz`       | smooth, bebop, fusion, swing           |
| `hip_hop`    | trap, boom_bap, lo_fi                  |
| `pop`        | synth_pop, indie_pop, dance_pop        |
| `ambient`    | dark, light, nature, space             |
| `cinematic`  | action, horror, romance, adventure     |
| `folk`       | acoustic, celtic, world                |
| `game`       | 8bit, chiptune, fantasy, sci_fi        |

**Response (202 Accepted):**

```json
{
  "success": true,
  "data": {
    "jobId": "job_music_abc123",
    "status": "pending",
    "estimatedTime": 45,
    "message": "Music generation queued"
  }
}
```

### Extend Music

Extend an existing music track.

```http
POST /v1/audio/music/extend
Authorization: Bearer <token>
Content-Type: application/json

{
  "audioUrl": "https://example.com/track.wav",
  "direction": "after",
  "duration": 30,
  "prompt": "Build to a climactic finish",
  "seamless": true
}
```

**Parameters:**

| Field       | Type    | Description                  |
| ----------- | ------- | ---------------------------- |
| `audioUrl`  | string  | Source audio URL             |
| `direction` | string  | `before`, `after`, or `both` |
| `duration`  | number  | Extension length in seconds  |
| `prompt`    | string  | Guidance for extension       |
| `seamless`  | boolean | Ensure seamless transition   |

### Remix Track

Create a remix or variation of existing music.

```http
POST /v1/audio/music/remix
Authorization: Bearer <token>
Content-Type: application/json

{
  "audioUrl": "https://example.com/original.wav",
  "style": "lo_fi",
  "tempo": 0.8,
  "preserveMelody": true,
  "addElements": ["vinyl_crackle", "rain"],
  "removeElements": ["drums"]
}
```

---

## Voice Synthesis

### Text-to-Speech

Convert text to natural-sounding speech.

```http
POST /v1/audio/voice/synthesize
Authorization: Bearer <token>
Content-Type: application/json

{
  "text": "Welcome to the kingdom of Eldoria, brave adventurer.",
  "voice": "narrator_male_deep",
  "language": "en-US",
  "speed": 1.0,
  "pitch": 0,
  "emotion": "warm",
  "format": "wav",
  "sampleRate": 48000
}
```

**Request Parameters:**

| Field        | Type     | Required | Description                             |
| ------------ | -------- | -------- | --------------------------------------- |
| `text`       | string   | Yes      | Text to synthesize (max 5000 chars)     |
| `voice`      | string   | Yes      | Voice ID or name                        |
| `language`   | string   | No       | Language code (default: en-US)          |
| `speed`      | number   | No       | Speed multiplier (0.5-2.0)              |
| `pitch`      | number   | No       | Pitch adjustment (-12 to +12 semitones) |
| `emotion`    | string   | No       | Emotional tone                          |
| `emphasis`   | object[] | No       | Word-level emphasis markers             |
| `ssml`       | boolean  | No       | Parse text as SSML                      |
| `format`     | string   | No       | Output format                           |
| `sampleRate` | number   | No       | Sample rate                             |

**Emotions:**

`neutral`, `happy`, `sad`, `angry`, `fearful`, `surprised`, `disgusted`, `warm`,
`cold`, `serious`, `playful`, `mysterious`, `authoritative`

**Response:**

```json
{
  "success": true,
  "data": {
    "audioUrl": "https://cdn.yemaya.io/audio/tts_abc123.wav",
    "duration": 4.5,
    "wordTimings": [
      { "word": "Welcome", "start": 0.0, "end": 0.45 },
      { "word": "to", "start": 0.45, "end": 0.55 },
      { "word": "the", "start": 0.55, "end": 0.65 }
    ],
    "format": "wav",
    "sampleRate": 48000
  }
}
```

### Clone Voice

Create a custom voice from audio samples.

```http
POST /v1/audio/voice/clone
Authorization: Bearer <token>
Content-Type: application/json

{
  "name": "Character_Hero",
  "samples": [
    { "url": "https://example.com/sample1.wav", "transcript": "Sample text one" },
    { "url": "https://example.com/sample2.wav", "transcript": "Sample text two" },
    { "url": "https://example.com/sample3.wav", "transcript": "Sample text three" }
  ],
  "description": "Male hero character, mid-30s, confident tone",
  "language": "en-US",
  "consent": true
}
```

**Requirements:**

- Minimum 3 samples, recommended 10+
- Each sample 10-30 seconds
- Clear audio, minimal background noise
- Varied speech patterns and emotions

**Response:**

```json
{
  "success": true,
  "data": {
    "voiceId": "voice_custom_abc123",
    "name": "Character_Hero",
    "status": "training",
    "estimatedTime": 300,
    "message": "Voice clone training started"
  }
}
```

### Batch Synthesis

Synthesize multiple text segments efficiently.

```http
POST /v1/audio/voice/batch
Authorization: Bearer <token>
Content-Type: application/json

{
  "segments": [
    { "id": "line_001", "text": "Hello traveler.", "voice": "npc_merchant" },
    { "id": "line_002", "text": "What brings you here?", "voice": "npc_merchant" },
    { "id": "line_003", "text": "I have rare goods.", "voice": "npc_merchant" }
  ],
  "format": "wav",
  "sampleRate": 48000,
  "callbackUrl": "https://your-server.com/callback"
}
```

### List Available Voices

Get all available voices.

```http
GET /v1/audio/voice/list?language=en&category=character
Authorization: Bearer <token>
```

**Response:**

```json
{
  "success": true,
  "data": [
    {
      "id": "narrator_male_deep",
      "name": "Deep Male Narrator",
      "category": "narrator",
      "language": "en-US",
      "gender": "male",
      "age": "adult",
      "accent": "american",
      "style": "narrative",
      "preview": "https://cdn.yemaya.io/voices/preview/narrator_male_deep.mp3"
    }
  ]
}
```

---

## Sound Effects

### Generate SFX

Create sound effects from descriptions.

```http
POST /v1/audio/sfx/generate
Authorization: Bearer <token>
Content-Type: application/json

{
  "prompt": "Heavy sword clash with metallic ring and sparks",
  "category": "combat",
  "duration": 2.0,
  "variations": 3,
  "format": "wav"
}
```

**Categories:**

| Category     | Examples                           |
| ------------ | ---------------------------------- |
| `combat`     | sword clashes, impacts, whooshes   |
| `magic`      | spells, portals, enchantments      |
| `nature`     | wind, rain, thunder, animals       |
| `mechanical` | engines, gears, machines           |
| `ui`         | clicks, notifications, transitions |
| `footsteps`  | various surfaces and speeds        |
| `explosions` | various sizes and types            |
| `vehicles`   | cars, ships, aircraft              |
| `creatures`  | monster sounds, growls, roars      |
| `ambient`    | room tones, city sounds            |

### Generate Foley

Create realistic foley sounds for video/animation sync.

```http
POST /v1/audio/foley/generate
Authorization: Bearer <token>
Content-Type: application/json

{
  "videoUrl": "https://example.com/scene.mp4",
  "categories": ["footsteps", "cloth", "props"],
  "autoSync": true,
  "surfaceType": "stone",
  "intensity": "normal"
}
```

### Generate Ambient

Create ambient soundscapes.

```http
POST /v1/audio/ambient/generate
Authorization: Bearer <token>
Content-Type: application/json

{
  "prompt": "Medieval tavern with fireplace, murmuring crowd, occasional laughter",
  "duration": 300,
  "loopable": true,
  "layers": [
    { "type": "base", "element": "room_tone" },
    { "type": "constant", "element": "fireplace_crackle" },
    { "type": "random", "element": "crowd_murmur", "density": 0.7 },
    { "type": "random", "element": "laughter", "density": 0.1 }
  ]
}
```

---

## Audio Processing

### Normalize Audio

Normalize audio levels.

```http
POST /v1/audio/process/normalize
Authorization: Bearer <token>
Content-Type: application/json

{
  "audioUrl": "https://example.com/audio.wav",
  "targetLufs": -14,
  "truePeak": -1.0,
  "method": "loudness"
}
```

### Remove Noise

Apply noise reduction.

```http
POST /v1/audio/process/denoise
Authorization: Bearer <token>
Content-Type: application/json

{
  "audioUrl": "https://example.com/noisy_audio.wav",
  "noiseReduction": 0.7,
  "preserveVoice": true,
  "method": "ai"
}
```

### Separate Stems

Separate audio into individual stems.

```http
POST /v1/audio/process/stems
Authorization: Bearer <token>
Content-Type: application/json

{
  "audioUrl": "https://example.com/song.wav",
  "stems": ["vocals", "drums", "bass", "other"],
  "format": "wav"
}
```

**Response:**

```json
{
  "success": true,
  "data": {
    "stems": {
      "vocals": "https://cdn.yemaya.io/stems/abc123_vocals.wav",
      "drums": "https://cdn.yemaya.io/stems/abc123_drums.wav",
      "bass": "https://cdn.yemaya.io/stems/abc123_bass.wav",
      "other": "https://cdn.yemaya.io/stems/abc123_other.wav"
    }
  }
}
```

### Spatial Audio Processing

Apply spatial/3D audio processing.

```http
POST /v1/audio/process/spatial
Authorization: Bearer <token>
Content-Type: application/json

{
  "audioUrl": "https://example.com/mono.wav",
  "format": "ambisonics",
  "order": 1,
  "position": { "azimuth": 45, "elevation": 0, "distance": 2.0 },
  "roomSize": "medium",
  "reverb": 0.3
}
```

### Analyze Audio

Get detailed audio analysis.

```http
POST /v1/audio/analyze
Authorization: Bearer <token>
Content-Type: application/json

{
  "audioUrl": "https://example.com/audio.wav",
  "analysis": ["loudness", "spectrum", "tempo", "key", "beats", "segments"]
}
```

**Response:**

```json
{
  "success": true,
  "data": {
    "duration": 180.5,
    "sampleRate": 48000,
    "channels": 2,
    "bitDepth": 24,
    "loudness": {
      "integrated": -14.2,
      "truePeak": -0.8,
      "range": 8.5
    },
    "tempo": {
      "bpm": 128,
      "confidence": 0.95
    },
    "key": {
      "key": "A",
      "mode": "minor",
      "confidence": 0.87
    },
    "beats": [
      { "time": 0.0, "confidence": 0.9 },
      { "time": 0.469, "confidence": 0.95 }
    ],
    "segments": [
      { "start": 0, "end": 32, "label": "intro" },
      { "start": 32, "end": 96, "label": "verse" }
    ]
  }
}
```

---

## DAW Integration

### Connect to DAW

Establish connection to a digital audio workstation.

```http
POST /v1/audio/daw/connect
Authorization: Bearer <token>
Content-Type: application/json

{
  "daw": "ableton",
  "host": "localhost",
  "port": 9000,
  "protocol": "osc"
}
```

**Supported DAWs:**

| DAW        | Protocol          |
| ---------- | ----------------- |
| `ableton`  | OSC, Max for Live |
| `logic`    | OSC               |
| `protools` | EUCON             |
| `reaper`   | OSC, ReaScript    |
| `cubase`   | OSC               |
| `flstudio` | MIDI              |

### Get DAW Status

Check DAW connection status.

```http
GET /v1/audio/daw/status
Authorization: Bearer <token>
```

### Control Transport

Control DAW playback.

```http
POST /v1/audio/daw/transport
Authorization: Bearer <token>
Content-Type: application/json

{
  "action": "play",
  "position": "1.1.1",
  "loop": { "start": "1.1.1", "end": "5.1.1" }
}
```

**Actions:** `play`, `stop`, `pause`, `record`, `seek`, `loop`, `rewind`,
`forward`

---

## WebSocket Events

### Music Generation Progress

```json
{
  "type": "generation:progress",
  "jobId": "job_music_abc123",
  "jobType": "music_generation",
  "progress": 65,
  "stage": "rendering",
  "message": "Rendering final audio...",
  "estimatedSecondsRemaining": 15
}
```

---

## Error Codes

| Code                 | Description                  |
| -------------------- | ---------------------------- |
| `INVALID_PROMPT`     | Music prompt is invalid      |
| `VOICE_NOT_FOUND`    | Voice ID does not exist      |
| `AUDIO_TOO_LONG`     | Audio exceeds duration limit |
| `UNSUPPORTED_FORMAT` | Audio format not supported   |
| `CLONE_FAILED`       | Voice cloning failed         |
| `DAW_DISCONNECTED`   | DAW connection lost          |
| `SYNTHESIS_FAILED`   | Speech synthesis failed      |

---

## Rate Limits

| Endpoint         | Limit      |
| ---------------- | ---------- |
| Music generation | 5/minute   |
| Voice synthesis  | 30/minute  |
| SFX generation   | 20/minute  |
| Processing       | 50/minute  |
| DAW commands     | 100/minute |
