Reference documentation for the OpenRouter AI provider supporting chat completions, embeddings, image generation, model routing, and cost management.
Overview#
The OpenRouter provider offers:
- Chat Completions: Text generation with streaming and function calling
- Embeddings: Text embeddings for semantic search
- Image Generation: Image creation via modalities parameter
- Model Capabilities: Dynamic model discovery and filtering
- Cost Tracking: Token counting and budget management
- Circuit Breaker: Automatic failure recovery
- Fallback Chains: Automatic model failover
Configuration#
Environment Variables#
| Variable | Description | Required |
|---|---|---|
OPENROUTER_API_KEY |
OpenRouter API key | Yes |
Provider Initialization#
typescript
import { OpenRouterProvider } from './providers/openrouter-provider';
const provider = new OpenRouterProvider({
apiKey: process.env.OPENROUTER_API_KEY,
baseURL: 'https://openrouter.ai/api/v1',
defaultModel: 'anthropic/claude-sonnet-4.5',
defaultEmbeddingModel: 'openai/text-embedding-3-small',
defaultImageModel: 'google/gemini-2.5-flash-image-preview',
maxRetries: 3,
timeout: 30000,
maxFailures: 5, // Circuit breaker threshold
resetTimeout: 60000, // Circuit breaker reset time
appName: 'Lilith',
appUrl: 'https://lilith.ai',
modelCacheTTL: 3600000, // 1 hour
});
Configuration Options#
| Option | Type | Default | Description |
|---|---|---|---|
apiKey |
string | OPENROUTER_API_KEY |
API key |
baseURL |
string | https://openrouter.ai/api/v1 |
API base URL |
defaultModel |
string | anthropic/claude-sonnet-4.5 |
Default chat model |
defaultEmbeddingModel |
string | openai/text-embedding-3-small |
Default embedding model |
defaultImageModel |
string | google/gemini-2.5-flash-image-preview |
Default image model |
maxRetries |
number | 3 | Max retry attempts |
timeout |
number | 30000 | Request timeout (ms) |
maxFailures |
number | 5 | Failures before circuit opens |
resetTimeout |
number | 60000 | Circuit reset time (ms) |
appName |
string | Lilith |
App name header |
appUrl |
string | https://lilith.ai |
App URL header |
modelCacheTTL |
number | 3600000 | Model cache TTL (ms) |
Chat Completions#
Basic Completion#
typescript
const response = await provider.complete({
messages: [
{ role: 'system', content: 'You are a helpful assistant.' },
{ role: 'user', content: 'What is meditation?' },
],
model: 'anthropic/claude-sonnet-4.5',
max_tokens: 1000,
temperature: 0.7,
});
console.log(response.content);
// { content, role, tool_calls, finish_reason, usage, model, id, provider, latency }
Alternative Prompt Format#
typescript
const response = await provider.complete({
system_prompt: 'You are a Zen master.',
user_prompt: 'What is the sound of one hand clapping?',
temperature: 0.8,
});
Completion Parameters#
| Parameter | Type | Description |
|---|---|---|
messages |
Message[] | Array of chat messages |
system_prompt |
string | System message (alternative to messages) |
user_prompt |
string | User message (alternative to messages) |
model |
string | Model ID (optional) |
max_tokens |
number | Max response tokens |
temperature |
number | Sampling temperature (0-2) |
top_p |
number | Nucleus sampling |
frequency_penalty |
number | Repetition penalty |
presence_penalty |
number | Topic diversity |
tools |
Tool[] | Function definitions |
tool_choice |
string | Tool selection mode |
response_format |
object | Response format (e.g., JSON mode) |
stream |
boolean | Enable streaming |
Streaming Completion#
typescript
const stream = await provider.completeStream({
messages: [{ role: 'user', content: 'Tell me a story.' }],
max_tokens: 500,
});
for await (const chunk of stream) {
const content = chunk.choices?.[0]?.delta?.content;
if (content) {
process.stdout.write(content);
}
}
Function Calling#
typescript
const response = await provider.complete({
messages: [{ role: 'user', content: 'What is the weather in Tokyo?' }],
tools: [
{
type: 'function',
function: {
name: 'get_weather',
description: 'Get weather for a location',
parameters: {
type: 'object',
properties: {
location: { type: 'string', description: 'City name' },
},
required: ['location'],
},
},
},
],
tool_choice: 'auto',
});
if (response.tool_calls) {
for (const call of response.tool_calls) {
console.log(`Function: ${call.function.name}`);
console.log(`Arguments: ${call.function.arguments}`);
}
}
JSON Mode#
typescript
const response = await provider.complete({
messages: [{ role: 'user', content: 'List 3 meditation types as JSON' }],
response_format: { type: 'json_object' },
});
const data = JSON.parse(response.content);
Completion Response#
typescript
interface CompletionResponse {
content: string;
role: string;
tool_calls?: any[];
finish_reason: string;
usage: {
prompt_tokens: number;
completion_tokens: number;
total_tokens: number;
};
model: string;
id: string;
provider: string;
latency?: number;
}
Embeddings#
Create Embedding#
typescript
const response = await provider.createEmbedding({
input: 'Mindfulness is the practice of present moment awareness.',
model: 'openai/text-embedding-3-small',
dimensions: 1536,
});
console.log(response.data[0].embedding);
// [0.012, -0.034, 0.056, ...]
Batch Embeddings#
typescript
const response = await provider.createEmbedding({
input: ['First text to embed', 'Second text to embed', 'Third text to embed'],
});
for (const item of response.data) {
console.log(`Index ${item.index}: ${item.embedding.length} dimensions`);
}
Embedding Response#
typescript
interface OpenRouterEmbeddingResponse {
object: string;
model: string;
data: Array<{
object: string;
embedding: number[];
index: number;
}>;
usage: {
prompt_tokens: number;
total_tokens: number;
};
}
Image Generation#
Generate Image#
typescript
const response = await provider.generateImage({
prompt: 'A serene zen garden with a stone path',
model: 'google/gemini-2.5-flash-image-preview',
imageConfig: {
aspectRatio: '16:9',
imageSize: '2K',
},
});
for (const imageUrl of response.images) {
console.log('Generated image:', imageUrl);
}
Image Configuration#
| Option | Values | Description |
|---|---|---|
aspectRatio |
1:1, 4:3, 3:4, 16:9, 9:16, 21:9 |
Image aspect ratio |
imageSize |
1K, 2K, 4K |
Image resolution |
Image Response#
typescript
interface OpenRouterImageResponse {
id: string;
model: string;
images: string[];
usage: {
prompt_tokens: number;
completion_tokens: number;
total_tokens: number;
};
}
Model Capabilities#
Fetch Available Models#
typescript
const models = await provider.fetchModelCapabilities();
for (const model of models) {
console.log(`${model.name} (${model.id})`);
console.log(` Context: ${model.contextLength} tokens`);
console.log(` Input: ${model.inputModalities.join(', ')}`);
console.log(` Output: ${model.outputModalities.join(', ')}`);
console.log(
` Pricing: $${model.pricing.prompt}/1K prompt, $${model.pricing.completion}/1K completion`
);
}
Model Capabilities#
typescript
interface OpenRouterModelCapabilities {
id: string;
name: string;
contextLength: number;
pricing: {
prompt: string; // Price per 1K tokens
completion: string;
};
inputModalities: string[]; // ['text'], ['text', 'image'], etc.
outputModalities: string[]; // ['text'], ['embedding'], ['image'], etc.
architecture?: {
modality: string;
};
}
Filter by Capability#
typescript
// Get embedding models
const embeddingModels = provider.getEmbeddingModels();
// Get image generation models
const imageModels = provider.getImageGenerationModels();
// Custom filtering
const visionModels = provider.getModelsByCapability({
inputModalities: ['text', 'image'],
outputModalities: ['text'],
});
Model Routing#
Fallback Chains#
Configure automatic model failover:
typescript
provider.configureFallbackChain({
chat: [
'anthropic/claude-sonnet-4.5',
'openai/gpt-4o',
'google/gemini-1.5-pro',
],
embeddings: [
'openai/text-embedding-3-small',
'openai/text-embedding-3-large',
],
images: ['google/gemini-2.5-flash-image-preview', 'openai/dall-e-3'],
});
Use with Fallback#
typescript
// Automatically tries fallback models on failure
const response = await provider.completeWithFallback({
messages: [{ role: 'user', content: 'Hello!' }],
});
const embedding = await provider.createEmbeddingWithFallback({
input: 'Text to embed',
});
const image = await provider.generateImageWithFallback({
prompt: 'A beautiful sunset',
});
Fallback Events#
typescript
provider.on('fallback-triggered', ({ failedModel, fallbackModel, error }) => {
console.log(`Fallback: ${failedModel} -> ${fallbackModel}`);
console.log(`Reason: ${error}`);
});
Failed Model Tracking#
typescript
// Get list of failed models
const failed = provider.getFailedModels();
// Clear failed models (allow retry)
provider.clearFailedModels();
Cost Management#
Token Tracking#
Usage is automatically tracked for all requests:
typescript
const stats = provider.getUsageStats();
// {
// totalPromptTokens: 15000,
// totalCompletionTokens: 8000,
// totalTokens: 23000,
// requestCount: 150,
// estimatedCost: 0.45
// }
Per-Model Usage#
typescript
const byModel = provider.getUsageByModel();
// {
// 'anthropic/claude-sonnet-4.5': { totalTokens: 15000, estimatedCost: 0.30 },
// 'openai/gpt-4o': { totalTokens: 8000, estimatedCost: 0.15 }
// }
Budget Limits#
typescript
// Set budget limit (in dollars)
provider.setBudgetLimit(10.00);
// Set warning threshold (80% by default)
provider.setBudgetWarningThreshold(0.8);
// Listen for warnings
provider.on('budget-warning', ({ currentCost, budgetLimit, percentUsed }) => {
console.log(`Warning: ${percentUsed.toFixed(1)}% of budget used`);
console.log(`Current: $${currentCost.toFixed(2)} / $${budgetLimit}`);
});
// Requests throw error when budget exceeded
try {
await provider.complete({ ... });
} catch (error) {
if (error.code === 402) {
console.log('Budget limit exceeded');
}
}
Circuit Breaker#
Availability Check#
typescript
if (provider.isAvailable()) {
const response = await provider.complete({ ... });
} else {
console.log('Provider unavailable, circuit is open');
}
Circuit Breaker States#
| State | Description |
|---|---|
closed |
Normal operation |
open |
Rejecting requests after failures |
half-open |
Testing with single request |
Circuit Breaker Events#
typescript
provider.on('circuit-open', ({ provider, error }) => {
console.log(`Circuit opened for ${provider}: ${error}`);
// Alert monitoring system
});
Configuration#
typescript
// 5 failures opens circuit
maxFailures: 5;
// 60 second reset timeout
resetTimeout: 60000;
State Transitions#
text
CLOSED → 5 failures → OPEN
↓
OPEN (60 second wait)
↓
HALF-OPEN (test request)
↓
├─ Success → CLOSED
│
└─ Failure → OPEN
Retry Logic#
Exponential Backoff#
Requests are automatically retried with exponential backoff:
typescript
// Retry delays: 1s, 2s, 4s, 8s... (max 10s)
const backoffMs = Math.min(1000 * Math.pow(2, attempt - 1), 10000);
Retryable Errors#
| Status | Description | Retried |
|---|---|---|
| 429 | Rate limited | Yes |
| 500+ | Server errors | Yes |
| 4xx | Client errors | No |
Health Check#
typescript
const healthy = await provider.healthCheck();
// true if API responds within 5 seconds
Events#
The provider emits events for monitoring:
| Event | Data | Description |
|---|---|---|
request-complete |
{ latency, model, tokenUsage } |
Request completed |
circuit-open |
{ provider, error } |
Circuit breaker opened |
budget-warning |
{ currentCost, budgetLimit, percentUsed } |
Budget threshold reached |
fallback-triggered |
{ failedModel, fallbackModel, error } |
Model failover occurred |
Event Listening#
typescript
provider.on('request-complete', ({ latency, model, tokenUsage }) => {
console.log(`${model}: ${tokenUsage.total_tokens} tokens, ${latency}ms`);
});
provider.on('circuit-open', ({ provider, error }) => {
alerting.send(`OpenRouter circuit open: ${error}`);
});
Error Handling#
OpenRouterError#
typescript
class OpenRouterError extends Error {
code: number; // HTTP status code
providerName?: string; // Upstream provider name
}
Error Codes#
| Code | Description |
|---|---|
| 400 | Bad request (invalid parameters) |
| 401 | Invalid API key |
| 402 | Budget limit exceeded |
| 429 | Rate limited |
| 500 | Internal server error |
| 503 | Circuit breaker open |
Error Handling Example#
typescript
try {
const response = await provider.complete({ ... });
} catch (error) {
if (error instanceof OpenRouterError) {
switch (error.code) {
case 402:
console.log('Budget exceeded');
break;
case 429:
console.log('Rate limited, retry after cooldown');
break;
case 503:
console.log('Service unavailable');
break;
default:
console.log(`Error ${error.code}: ${error.message}`);
}
}
}
Cleanup#
typescript
// Shutdown provider and release resources
provider.shutdown();
Related Documentation#
- Prompt Management - Template system
- AI Service - Main AI orchestration
- BFF Patterns - API gateway patterns