# Streaming API

This document provides comprehensive documentation for the Lilith Conversation
Service Streaming API, including SSE (Server-Sent Events) and WebSocket
streaming for real-time chat responses.

## Overview

The Streaming API enables real-time chat interactions with:

- **SSE Streaming**: Token-by-token HTTP streaming with backpressure handling
- **WebSocket Streaming**: Full-duplex communication with heartbeat
- **Adaptive Throttling**: Automatic flow control based on client capacity
- **Nagle Optimization**: Efficient network utilization with batched writes

---

## Table of Contents

1. [SSE Streaming](#sse-streaming)
2. [WebSocket Streaming](#websocket-streaming)
3. [Event Types](#event-types)
4. [Backpressure Handling](#backpressure-handling)
5. [Error Handling](#error-handling)

---

## SSE Streaming

### Overview

Server-Sent Events (SSE) provides unidirectional streaming from server to client
over HTTP. Enable streaming by setting `options.stream: true` in chat requests.

### Endpoint

**Endpoint:** `POST /v1/chat`

**Request Body:**

| Field                  | Type    | Required | Description                  |
| ---------------------- | ------- | -------- | ---------------------------- |
| `persona`              | string  | No       | AI persona to use            |
| `input.text`           | string  | Yes      | User message text            |
| `options.stream`       | boolean | Yes      | Must be `true` for streaming |
| `options.message_type` | string  | No       | Message type (default: user) |
| `conversation_id`      | string  | No       | Existing conversation ID     |

**Request:**

```bash
curl -X POST "https://api.lilith.app/v1/chat" \
  -H "Authorization: Bearer <access_token>" \
  -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" \
  -d '{
    "persona": "meditation_guide",
    "input": { "text": "Guide me through a breathing exercise" },
    "options": { "stream": true },
    "conversation_id": "conv_123"
  }'
```

### Response Headers

```
HTTP/1.1 200 OK
Content-Type: text/event-stream
Cache-Control: no-cache, no-transform
Connection: keep-alive
x-correlation-id: corr-abc123
```

### Event Stream Format

```
event: token
data: {"text": "Let's "}

event: token
data: {"text": "begin "}

event: token
data: {"text": "with "}

event: token
data: {"text": "deep "}

event: token
data: {"text": "breathing."}

event: citations
data: [{"source_uri": "https://example.org/wisdom", "span": {"start": 0, "end": 36}, "license": "PD", "confidence": 0.85}]

event: complete
data: {"conversation_id": "conv_123", "metrics": {"tokens_in": 8, "tokens_out": 5, "latency_ms": 250, "citations_count": 1}}

event: done
data: {"message_id": "msg-xyz789", "conversation_id": "conv_123", "metrics": {"tokens_in": 8, "tokens_out": 5, "latency_ms": 250}, "retry_info": {"attempts": 1, "total_duration_ms": 260, "exhausted": false}}
```

### JavaScript Client Example

```javascript
const response = await fetch('https://api.lilith.app/v1/chat', {
  method: 'POST',
  headers: {
    Authorization: 'Bearer <access_token>',
    'Content-Type': 'application/json',
    Accept: 'text/event-stream',
  },
  body: JSON.stringify({
    persona: 'meditation_guide',
    input: { text: 'Guide me through breathing' },
    options: { stream: true },
  }),
});

const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = '';

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  buffer += decoder.decode(value, { stream: true });
  const lines = buffer.split('\n');
  buffer = lines.pop() || '';

  for (const line of lines) {
    if (line.startsWith('event: ')) {
      const eventType = line.slice(7);
      // Handle event type
    } else if (line.startsWith('data: ')) {
      const data = JSON.parse(line.slice(6));
      if (data.text) {
        // Append token to output
        process.stdout.write(data.text);
      }
    }
  }
}
```

---

## WebSocket Streaming

### Overview

WebSocket provides full-duplex communication for real-time chat with
bidirectional messaging, heartbeat, and conversation history retrieval.

### Connection

**Endpoint:** `wss://api.lilith.app/v1/ws/chat`

```javascript
const ws = new WebSocket('wss://api.lilith.app/v1/ws/chat');

ws.onopen = () => {
  console.log('Connected');
};

ws.onmessage = (event) => {
  const message = JSON.parse(event.data);
  handleMessage(message);
};

ws.onerror = (error) => {
  console.error('WebSocket error:', error);
};

ws.onclose = () => {
  console.log('Disconnected');
};
```

### Connection Response

Upon connection, the server sends a connection confirmation:

```json
{
  "type": "connection",
  "data": {
    "connection_id": "ws-abc123",
    "status": "connected",
    "server_info": {
      "service": "conversation",
      "version": "1.0"
    }
  }
}
```

### Client-to-Server Messages

#### Chat Message

Send a chat message for processing:

```json
{
  "type": "chat",
  "persona": "meditation_guide",
  "input": { "text": "How do I meditate?" },
  "conversation_id": "conv_123",
  "options": {
    "stream": true,
    "message_type": "user"
  }
}
```

#### Ping (Heartbeat)

Send a ping to keep the connection alive:

```json
{
  "type": "ping"
}
```

#### History Request

Request conversation history:

```json
{
  "type": "history",
  "conversation_id": "conv_123"
}
```

### Server-to-Client Messages

#### Message Received

Acknowledgment that a message was received:

```json
{
  "type": "message_received",
  "data": {
    "conversation_id": "conv_123",
    "message_id": "msg-xyz789",
    "persona": "meditation_guide",
    "options": { "message_type": "user" },
    "received_at": "2024-01-01T00:00:00.000Z"
  }
}
```

#### Start Event

Indicates response generation has started:

```json
{
  "type": "start",
  "data": {
    "conversation_id": "conv_123",
    "message_id": "msg-xyz789",
    "persona": "meditation_guide",
    "options": { "message_type": "user" }
  }
}
```

#### Token Event

Individual token/word in the response:

```json
{
  "type": "token",
  "data": {
    "text": "meditation ",
    "sequence": 5,
    "message_id": "msg-xyz789"
  }
}
```

#### Response Complete

Full response completed:

```json
{
  "type": "response_complete",
  "data": {
    "conversation_id": "conv_123",
    "message_id": "msg-xyz789",
    "persona": "meditation_guide",
    "full_text": "(WebSocket) reply to: How do I meditate?",
    "citations": [],
    "options": { "message_type": "user" },
    "metrics": {
      "tokens_in": 5,
      "tokens_out": 8,
      "latency_ms": 150
    }
  }
}
```

#### Pong (Heartbeat Response)

Response to ping:

```json
{
  "type": "pong",
  "data": {
    "timestamp": "2024-01-01T00:00:00.000Z"
  }
}
```

#### History Response

Conversation history:

```json
{
  "type": "history",
  "data": {
    "conversation_id": "conv_123",
    "messages": [
      {
        "id": "msg-001",
        "role": "user",
        "content": "Hello",
        "created_at": "2024-01-01T00:00:00.000Z"
      }
    ]
  }
}
```

#### Error Event

Error notification:

```json
{
  "type": "error",
  "data": {
    "code": "empty_message",
    "message": "Empty message text is not allowed"
  }
}
```

### WebSocket Status Endpoint

Check active WebSocket connections:

**Endpoint:** `GET /v1/ws/status`

**Response:**

```json
{
  "connected_clients": 5,
  "connections": [
    {
      "connection_id": "ws-abc123",
      "metadata": {
        "connected_at": "2024-01-01T00:00:00.000Z",
        "client_ip": "192.168.1.1",
        "user_agent": "Mozilla/5.0..."
      }
    }
  ]
}
```

---

## Event Types

### SSE Event Types

| Event       | Description                               |
| ----------- | ----------------------------------------- |
| `token`     | Individual token/word in response         |
| `citations` | Citation references for the response      |
| `complete`  | Response generation complete with metrics |
| `done`      | Final event with full metadata            |
| `error`     | Error during streaming                    |

### WebSocket Message Types

| Type                | Direction | Description                  |
| ------------------- | --------- | ---------------------------- |
| `connection`        | S→C       | Connection established       |
| `chat`              | C→S       | Send chat message            |
| `message_received`  | S→C       | Message acknowledged         |
| `start`             | S→C       | Response generation starting |
| `token`             | S→C       | Individual token             |
| `response_complete` | S→C       | Full response ready          |
| `complete`          | S→C       | Final completion event       |
| `ping`              | C→S       | Heartbeat request            |
| `pong`              | S→C       | Heartbeat response           |
| `history`           | Both      | Request/receive history      |
| `error`             | S→C       | Error notification           |

---

## Backpressure Handling

### SSE Backpressure

The server implements adaptive backpressure handling:

1. **Buffer Monitoring**: Tracks buffer size (max 64KB)
2. **Adaptive Delay**: Adjusts throttle based on write success
3. **Drain Events**: Pauses on buffer pressure, resumes on drain
4. **Nagle Batching**: Combines small writes for efficiency

**Configuration:**

```
maxBufferSize: 64KB
baseThrottleDelay: 50ms
adaptiveMultiplier: 1.0 - 5.0x
nagleMaxChunkSize: 8KB
nagleFlushInterval: 100ms
```

### WebSocket Backpressure

WebSocket implements similar adaptive handling:

```
maxBufferSize: 32KB
baseThrottleDelay: 100ms
adaptiveMultiplier: 1.0 - 4.0x
```

When `bufferedAmount` exceeds threshold, the server:

1. Increases delay multiplier (up to 4x)
2. Waits for buffer to drain below 50%
3. Gradually reduces delay as pressure eases

---

## Error Handling

### SSE Errors

Errors are sent as events before stream ends:

```
event: error
data: {"code": "chat.streaming_error", "message": "Error during streaming", "details": "Connection timeout"}
```

### WebSocket Errors

Errors are sent as messages:

```json
{
  "type": "error",
  "data": {
    "code": "invalid_message",
    "message": "Failed to parse message payload",
    "details": "Unexpected token in JSON"
  }
}
```

### Error Codes

| Code                   | Description                     |
| ---------------------- | ------------------------------- |
| `empty_message`        | Message text is empty           |
| `invalid_message`      | Invalid message format          |
| `unknown_message_type` | Unrecognized message type       |
| `chat.streaming_error` | Error during stream processing  |
| `rate_limit_exceeded`  | Too many requests               |
| `circuit_breaker_open` | Service temporarily unavailable |

---

## SDK Integration

### TypeScript/JavaScript

```typescript
import { LilithClient } from '@lilith/sdk';

const client = new LilithClient({
  apiKey: process.env.LILITH_API_KEY,
});

// SSE Streaming
const stream = await client.conversation.chatStream({
  persona: 'meditation_guide',
  text: 'Guide me through breathing',
  conversationId: 'conv_123',
});

for await (const event of stream) {
  if (event.type === 'token') {
    process.stdout.write(event.data.text);
  } else if (event.type === 'complete') {
    console.log('\n\nMetrics:', event.data.metrics);
  }
}

// WebSocket
const ws = client.conversation.connectWebSocket();

ws.on('connected', (data) => {
  console.log('Connected:', data.connection_id);
});

ws.on('token', (data) => {
  process.stdout.write(data.text);
});

ws.on('response_complete', (data) => {
  console.log('\n\nComplete:', data.full_text);
});

// Send message
ws.send({
  type: 'chat',
  persona: 'meditation_guide',
  input: { text: 'How do I meditate?' },
  conversation_id: 'conv_123',
});

// Heartbeat
setInterval(() => {
  ws.send({ type: 'ping' });
}, 30000);
```

---

## Related Documentation

- [Thread API](./thread-api.md) - Thread and message management
- [Chat API](./chat-api.md) - Non-streaming chat
- [Context API](./context-api.md) - Context window management
- [Authentication](./authentication.md) - API authentication

---

## Questions?

- **Technical Support**: api-support@lilith.app
- **Documentation**: [GitHub Issues](https://github.com/lilith/platform/issues)
