# OpenRouter Integration

Reference documentation for the OpenRouter AI provider supporting chat
completions, embeddings, image generation, model routing, and cost management.

## Overview

The OpenRouter provider offers:

- **Chat Completions**: Text generation with streaming and function calling
- **Embeddings**: Text embeddings for semantic search
- **Image Generation**: Image creation via modalities parameter
- **Model Capabilities**: Dynamic model discovery and filtering
- **Cost Tracking**: Token counting and budget management
- **Circuit Breaker**: Automatic failure recovery
- **Fallback Chains**: Automatic model failover

## Configuration

### Environment Variables

| Variable             | Description        | Required |
| -------------------- | ------------------ | -------- |
| `OPENROUTER_API_KEY` | OpenRouter API key | Yes      |

### Provider Initialization

```typescript
import { OpenRouterProvider } from './providers/openrouter-provider';

const provider = new OpenRouterProvider({
  apiKey: process.env.OPENROUTER_API_KEY,
  baseURL: 'https://openrouter.ai/api/v1',
  defaultModel: 'anthropic/claude-sonnet-4.5',
  defaultEmbeddingModel: 'openai/text-embedding-3-small',
  defaultImageModel: 'google/gemini-2.5-flash-image-preview',
  maxRetries: 3,
  timeout: 30000,
  maxFailures: 5, // Circuit breaker threshold
  resetTimeout: 60000, // Circuit breaker reset time
  appName: 'Lilith',
  appUrl: 'https://lilith.ai',
  modelCacheTTL: 3600000, // 1 hour
});
```

### Configuration Options

| Option                  | Type   | Default                                 | Description                   |
| ----------------------- | ------ | --------------------------------------- | ----------------------------- |
| `apiKey`                | string | `OPENROUTER_API_KEY`                    | API key                       |
| `baseURL`               | string | `https://openrouter.ai/api/v1`          | API base URL                  |
| `defaultModel`          | string | `anthropic/claude-sonnet-4.5`           | Default chat model            |
| `defaultEmbeddingModel` | string | `openai/text-embedding-3-small`         | Default embedding model       |
| `defaultImageModel`     | string | `google/gemini-2.5-flash-image-preview` | Default image model           |
| `maxRetries`            | number | 3                                       | Max retry attempts            |
| `timeout`               | number | 30000                                   | Request timeout (ms)          |
| `maxFailures`           | number | 5                                       | Failures before circuit opens |
| `resetTimeout`          | number | 60000                                   | Circuit reset time (ms)       |
| `appName`               | string | `Lilith`                                | App name header               |
| `appUrl`                | string | `https://lilith.ai`                     | App URL header                |
| `modelCacheTTL`         | number | 3600000                                 | Model cache TTL (ms)          |

## Chat Completions

### Basic Completion

```typescript
const response = await provider.complete({
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'What is meditation?' },
  ],
  model: 'anthropic/claude-sonnet-4.5',
  max_tokens: 1000,
  temperature: 0.7,
});

console.log(response.content);
// { content, role, tool_calls, finish_reason, usage, model, id, provider, latency }
```

### Alternative Prompt Format

```typescript
const response = await provider.complete({
  system_prompt: 'You are a Zen master.',
  user_prompt: 'What is the sound of one hand clapping?',
  temperature: 0.8,
});
```

### Completion Parameters

| Parameter           | Type      | Description                              |
| ------------------- | --------- | ---------------------------------------- |
| `messages`          | Message[] | Array of chat messages                   |
| `system_prompt`     | string    | System message (alternative to messages) |
| `user_prompt`       | string    | User message (alternative to messages)   |
| `model`             | string    | Model ID (optional)                      |
| `max_tokens`        | number    | Max response tokens                      |
| `temperature`       | number    | Sampling temperature (0-2)               |
| `top_p`             | number    | Nucleus sampling                         |
| `frequency_penalty` | number    | Repetition penalty                       |
| `presence_penalty`  | number    | Topic diversity                          |
| `tools`             | Tool[]    | Function definitions                     |
| `tool_choice`       | string    | Tool selection mode                      |
| `response_format`   | object    | Response format (e.g., JSON mode)        |
| `stream`            | boolean   | Enable streaming                         |

### Streaming Completion

```typescript
const stream = await provider.completeStream({
  messages: [{ role: 'user', content: 'Tell me a story.' }],
  max_tokens: 500,
});

for await (const chunk of stream) {
  const content = chunk.choices?.[0]?.delta?.content;
  if (content) {
    process.stdout.write(content);
  }
}
```

### Function Calling

```typescript
const response = await provider.complete({
  messages: [{ role: 'user', content: 'What is the weather in Tokyo?' }],
  tools: [
    {
      type: 'function',
      function: {
        name: 'get_weather',
        description: 'Get weather for a location',
        parameters: {
          type: 'object',
          properties: {
            location: { type: 'string', description: 'City name' },
          },
          required: ['location'],
        },
      },
    },
  ],
  tool_choice: 'auto',
});

if (response.tool_calls) {
  for (const call of response.tool_calls) {
    console.log(`Function: ${call.function.name}`);
    console.log(`Arguments: ${call.function.arguments}`);
  }
}
```

### JSON Mode

```typescript
const response = await provider.complete({
  messages: [{ role: 'user', content: 'List 3 meditation types as JSON' }],
  response_format: { type: 'json_object' },
});

const data = JSON.parse(response.content);
```

## Completion Response

```typescript
interface CompletionResponse {
  content: string;
  role: string;
  tool_calls?: any[];
  finish_reason: string;
  usage: {
    prompt_tokens: number;
    completion_tokens: number;
    total_tokens: number;
  };
  model: string;
  id: string;
  provider: string;
  latency?: number;
}
```

## Embeddings

### Create Embedding

```typescript
const response = await provider.createEmbedding({
  input: 'Mindfulness is the practice of present moment awareness.',
  model: 'openai/text-embedding-3-small',
  dimensions: 1536,
});

console.log(response.data[0].embedding);
// [0.012, -0.034, 0.056, ...]
```

### Batch Embeddings

```typescript
const response = await provider.createEmbedding({
  input: ['First text to embed', 'Second text to embed', 'Third text to embed'],
});

for (const item of response.data) {
  console.log(`Index ${item.index}: ${item.embedding.length} dimensions`);
}
```

### Embedding Response

```typescript
interface OpenRouterEmbeddingResponse {
  object: string;
  model: string;
  data: Array<{
    object: string;
    embedding: number[];
    index: number;
  }>;
  usage: {
    prompt_tokens: number;
    total_tokens: number;
  };
}
```

## Image Generation

### Generate Image

```typescript
const response = await provider.generateImage({
  prompt: 'A serene zen garden with a stone path',
  model: 'google/gemini-2.5-flash-image-preview',
  imageConfig: {
    aspectRatio: '16:9',
    imageSize: '2K',
  },
});

for (const imageUrl of response.images) {
  console.log('Generated image:', imageUrl);
}
```

### Image Configuration

| Option        | Values                                      | Description        |
| ------------- | ------------------------------------------- | ------------------ |
| `aspectRatio` | `1:1`, `4:3`, `3:4`, `16:9`, `9:16`, `21:9` | Image aspect ratio |
| `imageSize`   | `1K`, `2K`, `4K`                            | Image resolution   |

### Image Response

```typescript
interface OpenRouterImageResponse {
  id: string;
  model: string;
  images: string[];
  usage: {
    prompt_tokens: number;
    completion_tokens: number;
    total_tokens: number;
  };
}
```

## Model Capabilities

### Fetch Available Models

```typescript
const models = await provider.fetchModelCapabilities();

for (const model of models) {
  console.log(`${model.name} (${model.id})`);
  console.log(`  Context: ${model.contextLength} tokens`);
  console.log(`  Input: ${model.inputModalities.join(', ')}`);
  console.log(`  Output: ${model.outputModalities.join(', ')}`);
  console.log(
    `  Pricing: $${model.pricing.prompt}/1K prompt, $${model.pricing.completion}/1K completion`
  );
}
```

### Model Capabilities

```typescript
interface OpenRouterModelCapabilities {
  id: string;
  name: string;
  contextLength: number;
  pricing: {
    prompt: string; // Price per 1K tokens
    completion: string;
  };
  inputModalities: string[]; // ['text'], ['text', 'image'], etc.
  outputModalities: string[]; // ['text'], ['embedding'], ['image'], etc.
  architecture?: {
    modality: string;
  };
}
```

### Filter by Capability

```typescript
// Get embedding models
const embeddingModels = provider.getEmbeddingModels();

// Get image generation models
const imageModels = provider.getImageGenerationModels();

// Custom filtering
const visionModels = provider.getModelsByCapability({
  inputModalities: ['text', 'image'],
  outputModalities: ['text'],
});
```

## Model Routing

### Fallback Chains

Configure automatic model failover:

```typescript
provider.configureFallbackChain({
  chat: [
    'anthropic/claude-sonnet-4.5',
    'openai/gpt-4o',
    'google/gemini-1.5-pro',
  ],
  embeddings: [
    'openai/text-embedding-3-small',
    'openai/text-embedding-3-large',
  ],
  images: ['google/gemini-2.5-flash-image-preview', 'openai/dall-e-3'],
});
```

### Use with Fallback

```typescript
// Automatically tries fallback models on failure
const response = await provider.completeWithFallback({
  messages: [{ role: 'user', content: 'Hello!' }],
});

const embedding = await provider.createEmbeddingWithFallback({
  input: 'Text to embed',
});

const image = await provider.generateImageWithFallback({
  prompt: 'A beautiful sunset',
});
```

### Fallback Events

```typescript
provider.on('fallback-triggered', ({ failedModel, fallbackModel, error }) => {
  console.log(`Fallback: ${failedModel} -> ${fallbackModel}`);
  console.log(`Reason: ${error}`);
});
```

### Failed Model Tracking

```typescript
// Get list of failed models
const failed = provider.getFailedModels();

// Clear failed models (allow retry)
provider.clearFailedModels();
```

## Cost Management

### Token Tracking

Usage is automatically tracked for all requests:

```typescript
const stats = provider.getUsageStats();
// {
//   totalPromptTokens: 15000,
//   totalCompletionTokens: 8000,
//   totalTokens: 23000,
//   requestCount: 150,
//   estimatedCost: 0.45
// }
```

### Per-Model Usage

```typescript
const byModel = provider.getUsageByModel();
// {
//   'anthropic/claude-sonnet-4.5': { totalTokens: 15000, estimatedCost: 0.30 },
//   'openai/gpt-4o': { totalTokens: 8000, estimatedCost: 0.15 }
// }
```

### Budget Limits

```typescript
// Set budget limit (in dollars)
provider.setBudgetLimit(10.00);

// Set warning threshold (80% by default)
provider.setBudgetWarningThreshold(0.8);

// Listen for warnings
provider.on('budget-warning', ({ currentCost, budgetLimit, percentUsed }) => {
  console.log(`Warning: ${percentUsed.toFixed(1)}% of budget used`);
  console.log(`Current: $${currentCost.toFixed(2)} / $${budgetLimit}`);
});

// Requests throw error when budget exceeded
try {
  await provider.complete({ ... });
} catch (error) {
  if (error.code === 402) {
    console.log('Budget limit exceeded');
  }
}
```

## Circuit Breaker

### Availability Check

```typescript
if (provider.isAvailable()) {
  const response = await provider.complete({ ... });
} else {
  console.log('Provider unavailable, circuit is open');
}
```

### Circuit Breaker States

| State       | Description                       |
| ----------- | --------------------------------- |
| `closed`    | Normal operation                  |
| `open`      | Rejecting requests after failures |
| `half-open` | Testing with single request       |

### Circuit Breaker Events

```typescript
provider.on('circuit-open', ({ provider, error }) => {
  console.log(`Circuit opened for ${provider}: ${error}`);
  // Alert monitoring system
});
```

### Configuration

```typescript
// 5 failures opens circuit
maxFailures: 5;

// 60 second reset timeout
resetTimeout: 60000;
```

### State Transitions

```
CLOSED → 5 failures → OPEN
    ↓
OPEN (60 second wait)
    ↓
HALF-OPEN (test request)
    ↓
    ├─ Success → CLOSED
    │
    └─ Failure → OPEN
```

## Retry Logic

### Exponential Backoff

Requests are automatically retried with exponential backoff:

```typescript
// Retry delays: 1s, 2s, 4s, 8s... (max 10s)
const backoffMs = Math.min(1000 * Math.pow(2, attempt - 1), 10000);
```

### Retryable Errors

| Status | Description   | Retried |
| ------ | ------------- | ------- |
| 429    | Rate limited  | Yes     |
| 500+   | Server errors | Yes     |
| 4xx    | Client errors | No      |

## Health Check

```typescript
const healthy = await provider.healthCheck();
// true if API responds within 5 seconds
```

## Events

The provider emits events for monitoring:

| Event                | Data                                        | Description              |
| -------------------- | ------------------------------------------- | ------------------------ |
| `request-complete`   | `{ latency, model, tokenUsage }`            | Request completed        |
| `circuit-open`       | `{ provider, error }`                       | Circuit breaker opened   |
| `budget-warning`     | `{ currentCost, budgetLimit, percentUsed }` | Budget threshold reached |
| `fallback-triggered` | `{ failedModel, fallbackModel, error }`     | Model failover occurred  |

### Event Listening

```typescript
provider.on('request-complete', ({ latency, model, tokenUsage }) => {
  console.log(`${model}: ${tokenUsage.total_tokens} tokens, ${latency}ms`);
});

provider.on('circuit-open', ({ provider, error }) => {
  alerting.send(`OpenRouter circuit open: ${error}`);
});
```

## Error Handling

### OpenRouterError

```typescript
class OpenRouterError extends Error {
  code: number; // HTTP status code
  providerName?: string; // Upstream provider name
}
```

### Error Codes

| Code | Description                      |
| ---- | -------------------------------- |
| 400  | Bad request (invalid parameters) |
| 401  | Invalid API key                  |
| 402  | Budget limit exceeded            |
| 429  | Rate limited                     |
| 500  | Internal server error            |
| 503  | Circuit breaker open             |

### Error Handling Example

```typescript
try {
  const response = await provider.complete({ ... });
} catch (error) {
  if (error instanceof OpenRouterError) {
    switch (error.code) {
      case 402:
        console.log('Budget exceeded');
        break;
      case 429:
        console.log('Rate limited, retry after cooldown');
        break;
      case 503:
        console.log('Service unavailable');
        break;
      default:
        console.log(`Error ${error.code}: ${error.message}`);
    }
  }
}
```

## Cleanup

```typescript
// Shutdown provider and release resources
provider.shutdown();
```

## Related Documentation

- [Prompt Management](./prompt-management.md) - Template system
- [AI Service](./ai-service.md) - Main AI orchestration
- [BFF Patterns](./bff-patterns.md) - API gateway patterns
