Video Tutorial 2: Streaming and Real-Time AI

Duration: 12-15 minutes Format: Screen recording with voiceover Target Audience: Developers building real-time AI applications

Video Script#

Opening (0:00 - 0:30)#

[Screen: Title card]

Welcome back to the Oshun AI Integrations series. In this video, we'll explore streaming - a technique that dramatically improves user experience by delivering AI outputs in real-time instead of waiting for complete results.

[Screen: Before/After comparison]

Here's the difference: without streaming, users wait several seconds staring at a loading spinner. With streaming, they start receiving output immediately.

Part 1: Why Streaming? (0:30 - 2:00)#

[Screen: Diagram showing traditional vs streaming]

Traditional:
[Request] ──────────────── [Wait 5s] ────────────────▶ [Complete Response]

Streaming:
[Request] ─▶ [Chunk] ─▶ [Chunk] ─▶ [Chunk] ─▶ [Chunk] ─▶ [Complete]
            50ms      100ms      150ms      200ms       ...

In traditional synthesis, the entire audio is generated on the server before any data is sent. With streaming, audio chunks are sent as soon as they're ready.

This matters especially for:

• Voice assistants that need to feel responsive
• Long-form content where users shouldn't wait
• Interactive applications like chatbots

Part 2: Basic Streaming Setup (2:00 - 5:00)#

[Screen: Code editor]

Let's implement streaming audio synthesis. First, we need to start a streaming session.

const session = await elevenlabs.startStreamingSession({
  voiceId: 'EXAVITQu4vr4xnSDxMaL',
  modelId: 'eleven_flash_v2_5', // Optimized for streaming
  optimizeStreamingLatency: 4,   // Maximum optimization
});

console.log(`Session started: ${session.sessionId}`);

Notice we're using eleven_flash_v2_5 - this model is specifically optimized for low-latency streaming.

[Screen: Add event handlers]

Now we set up event handlers to receive audio chunks.

const chunks: Buffer[] = [];

session.on('audio', (chunk: Buffer) => {
  chunks.push(chunk);
  console.log(`Received chunk: ${chunk.length} bytes`);
});

session.on('end', () => {
  const totalAudio = Buffer.concat(chunks);
  console.log(`Total: ${totalAudio.length} bytes`);
});

session.on('error', (error: Error) => {
  console.error('Stream error:', error.message);
});

[Screen: Send text]

Finally, we send our text and wait for the stream to complete.

await session.sendText('Hello! This is streaming audio synthesis.');
await session.flush();
await session.close();

[Screen: Run and show output]

npx ts-node streaming-demo.ts
# Session started: sess_abc123
# Received chunk: 4096 bytes
# Received chunk: 4096 bytes
# Received chunk: 4096 bytes
# ...
# Total: 45632 bytes

Notice how chunks arrive continuously instead of all at once!

Part 3: Real-Time Playback (5:00 - 8:00)#

[Screen: Audio buffer concept]

In a real application, you'd play audio chunks as they arrive. Let's build a simple audio buffer to manage this.

class AudioBuffer {
  private chunks: Buffer[] = [];
  private totalBytes = 0;

  addChunk(chunk: Buffer): void {
    this.chunks.push(chunk);
    this.totalBytes += chunk.length;
  }

  // Estimate duration (44100Hz, 16-bit, mono)
  getEstimatedDuration(): number {
    return this.totalBytes / (44100 * 2);
  }

  getBuffer(): Buffer {
    return Buffer.concat(this.chunks);
  }
}

[Screen: Use with streaming]

const buffer = new AudioBuffer();

session.on('audio', (chunk) => {
  buffer.addChunk(chunk);
  console.log(`Duration so far: ${buffer.getEstimatedDuration().toFixed(2)}s`);

  // In a real app, you'd send this chunk to an audio player
  playAudioChunk(chunk);
});

[Screen: Timeline visualization]

Here's what happens over time:

Time:  0ms   100ms  200ms  300ms  400ms  500ms ...
       │      │      │      │      │      │
Audio: ■──────■──────■──────■──────■──────■──────
       │      └──────┴──────┴──────┴──────┘
       │              Playing as received
       └── First chunk starts playing almost immediately

Part 4: Interactive Conversations (8:00 - 11:00)#

[Screen: Conversation concept]

Streaming really shines in interactive applications. Let's build a simple voice assistant that responds to multiple messages.

async function interactiveAssistant(responses: string[]): Promise<void> {
  const session = await elevenlabs.startStreamingSession({
    voiceId: 'EXAVITQu4vr4xnSDxMaL',
    modelId: 'eleven_flash_v2_5',
    optimizeStreamingLatency: 4,
  });

  for (const response of responses) {
    console.log(`Speaking: "${response}"`);

    await session.sendText(response);
    await session.flush(); // Wait for this response to complete

    // Simulate thinking time between responses
    await new Promise(r => setTimeout(r, 500));
  }

  await session.close();
}

[Screen: Demo the assistant]

await interactiveAssistant([
  'Hello! Welcome to our AI assistant.',
  'How can I help you today?',
  'I can answer questions about many topics.',
]);

[Screen: Run and show]

Watch how each response starts playing almost immediately, creating a natural conversational flow.

[Screen: Integration with LLM]

In practice, you'd combine this with a language model. As the LLM generates tokens, you stream them to the TTS:

// Pseudo-code for LLM integration
llm.on('token', (token) => {
  textBuffer += token;

  // Send complete sentences to TTS
  if (token.match(/[.!?]/)) {
    session.sendText(textBuffer);
    textBuffer = '';
  }
});

Part 5: Best Practices (11:00 - 13:00)#

[Screen: Tips slide]

Here are some best practices for streaming:

1. Use the right model eleven_flash_v2_5 is optimized for streaming. Use it unless you need features from other models.

2. Set optimization level The optimizeStreamingLatency option ranges from 0 to 4. Higher values reduce latency but may affect quality slightly.

3. Handle errors gracefully Streams can disconnect. Always have error handlers:

session.on('error', (error) => {
  console.error('Stream error:', error);
  // Attempt reconnection or fallback
});

4. Clean up resources Always close sessions when done:

try {
  // Use the session...
} finally {
  await session.close();
}

5. Consider buffering Buffer a few chunks before playing to handle network jitter:

const MIN_BUFFER = 3; // chunks
let playbackStarted = false;

session.on('audio', (chunk) => {
  buffer.addChunk(chunk);

  if (!playbackStarted && buffer.chunkCount >= MIN_BUFFER) {
    startPlayback();
    playbackStarted = true;
  }
});

Closing (13:00 - 14:00)#

[Screen: Summary]

To recap, streaming enables:

• Near-instant response times
• Better user experience
• More efficient memory usage
• Natural conversation flows

The key points are:

• Use streaming-optimized models
• Handle chunks as they arrive
• Implement proper error handling
• Clean up resources

[Screen: Next steps]

In the next video, we'll cover batch processing and how to efficiently handle multiple AI requests. See you there!

B-Roll Suggestions#

• Side-by-side comparison of loading vs streaming
• Timeline animation showing chunks arriving
• Real-time console output as chunks arrive
• Audio waveform building up progressively
• Chat interface with instant responses

Notes for Recording#

1. Emphasize the responsiveness difference
2. Show real latency numbers when possible
3. Demonstrate actual audio playing
4. Keep explanations practical and actionable