This guide covers integration with the OpenRouter API for text generation, embeddings, and image generation.
Overview#
The OpenRouterProvider provides a unified interface to 200+ AI models through the OpenRouter API, including:
- Chat Completions: Claude, GPT-4, Llama, Gemini, and more
- Embeddings: OpenAI text-embedding-3, voyage-ai, and others
- Image Generation: Gemini 2.5 Flash, DALL-E, Stable Diffusion via modality routing
Installation & Setup#
Environment Variables#
bash
export OPENROUTER_API_KEY="your-api-key"
Basic Initialization#
typescript
import { OpenRouterProvider } from '../providers/openrouter-provider.js';
const provider = new OpenRouterProvider({
apiKey: process.env.OPENROUTER_API_KEY,
defaultModel: 'anthropic/claude-sonnet-4.5',
defaultEmbeddingModel: 'openai/text-embedding-3-small',
defaultImageModel: 'google/gemini-2.5-flash-image-preview',
maxRetries: 3,
timeout: 30000
});
Configuration Options#
| Option | Type | Default | Description |
|---|---|---|---|
apiKey |
string | OPENROUTER_API_KEY env |
Your OpenRouter API key |
baseURL |
string | https://openrouter.ai/api/v1 |
API base URL |
defaultModel |
string | anthropic/claude-sonnet-4.5 |
Default chat model |
defaultEmbeddingModel |
string | openai/text-embedding-3-small |
Default embedding model |
defaultImageModel |
string | google/gemini-2.5-flash-image-preview |
Default image model |
maxRetries |
number | 3 |
Max retry attempts |
timeout |
number | 30000 |
Request timeout (ms) |
maxFailures |
number | 5 |
Circuit breaker threshold |
resetTimeout |
number | 60000 |
Circuit breaker reset (ms) |
appName |
string | Lilith |
App name for tracking |
appUrl |
string | https://lilith.ai |
App URL for tracking |
modelCacheTTL |
number | 3600000 |
Model cache TTL (ms) |
Chat Completions#
Basic Text Generation#
typescript
const response = await provider.complete({
user_prompt: 'Write a haiku about meditation',
max_tokens: 100,
temperature: 0.7
});
console.log(response.content);
// Output: "Still water reflects / Thoughts drift like autumn leaves fall / Peace in emptiness"
With System Prompt#
typescript
const response = await provider.complete({
system_prompt: 'You are a meditation teacher with 20 years of experience.',
user_prompt: 'Explain the benefits of mindfulness.',
model: 'anthropic/claude-sonnet-4.5',
max_tokens: 500
});
Using Messages Array#
typescript
const response = await provider.complete({
messages: [
{ role: 'system', content: 'You are a helpful assistant.' },
{ role: 'user', content: 'What is mindfulness?' },
{ role: 'assistant', content: 'Mindfulness is the practice of...' },
{ role: 'user', content: 'How do I get started?' }
],
max_tokens: 300
});
Streaming Responses#
typescript
const stream = await provider.completeStream({
messages: [{ role: 'user', content: 'Tell me a story about zen.' }],
max_tokens: 1000
});
for await (const chunk of stream) {
if (chunk.choices?.[0]?.delta?.content) {
process.stdout.write(chunk.choices[0].delta.content);
}
}
Tool Use (Function Calling)#
typescript
const response = await provider.complete({
messages: [{ role: 'user', content: 'What is the weather in Tokyo?' }],
tools: [
{
type: 'function',
function: {
name: 'get_weather',
description: 'Get current weather for a location',
parameters: {
type: 'object',
properties: {
location: { type: 'string', description: 'City name' }
},
required: ['location']
}
}
}
],
tool_choice: 'auto'
});
if (response.tool_calls) {
// Handle tool calls
console.log(response.tool_calls);
}
JSON Mode#
typescript
const response = await provider.complete({
user_prompt: 'List 3 meditation techniques as JSON',
response_format: { type: 'json_object' },
max_tokens: 500
});
const data = JSON.parse(response.content);
Embeddings#
Single Text Embedding#
typescript
const response = await provider.createEmbedding({
input: 'Mindfulness is the practice of being present.',
model: 'openai/text-embedding-3-small'
});
const embedding = response.data[0].embedding;
// Returns: number[] with 1536 dimensions
Batch Embeddings#
typescript
const response = await provider.createEmbedding({
input: [
'First meditation technique',
'Second meditation technique',
'Third meditation technique'
]
});
const embeddings = response.data.map(d => d.embedding);
With Custom Dimensions#
typescript
const response = await provider.createEmbedding({
input: 'Text to embed',
model: 'openai/text-embedding-3-large',
dimensions: 512 // Reduce dimensions for faster search
});
Image Generation#
Basic Image Generation#
typescript
const response = await provider.generateImage({
prompt: 'A serene zen garden at sunset, watercolor style',
model: 'google/gemini-2.5-flash-image-preview'
});
const imageBase64 = response.images[0];
With Image Configuration#
typescript
const response = await provider.generateImage({
prompt: 'A lotus flower floating on still water',
imageConfig: {
aspectRatio: '16:9',
imageSize: '2K'
},
numImages: 2
});
Aspect Ratio Options#
1:1- Square (default)4:3- Landscape traditional3:4- Portrait traditional16:9- Widescreen landscape9:16- Widescreen portrait21:9- Ultra-wide
Image Size Options#
1K- Lower resolution, faster2K- Standard resolution4K- High resolution, slower
Model Discovery#
Fetch Available Models#
typescript
const models = await provider.fetchModelCapabilities();
console.log(`Found ${models.length} models`);
models.forEach(m => {
console.log(`${m.id}: ${m.name} (context: ${m.contextLength})`);
});
Filter by Capability#
typescript
// Get all embedding models
const embeddingModels = provider.getEmbeddingModels();
// Get all image generation models
const imageModels = provider.getImageGenerationModels();
// Custom filter
const multimodalModels = provider.getModelsByCapability({
inputModalities: ['text', 'image'],
outputModalities: ['text']
});
Model Capabilities Response#
typescript
interface OpenRouterModelCapabilities {
id: string; // e.g., 'anthropic/claude-sonnet-4.5'
name: string; // e.g., 'Claude Sonnet 4.5'
contextLength: number; // e.g., 200000
pricing: {
prompt: string; // Cost per 1K tokens (e.g., '0.003')
completion: string; // Cost per 1K tokens (e.g., '0.015')
};
inputModalities: string[]; // e.g., ['text', 'image']
outputModalities: string[]; // e.g., ['text']
}
Budget & Cost Tracking#
Set Budget Limit#
typescript
// Set a $10 budget limit
provider.setBudgetLimit(10.0);
// Set warning threshold at 80%
provider.setBudgetWarningThreshold(0.8);
// Listen for budget warnings
provider.on('budget-warning', (data) => {
console.log(`Budget warning: ${data.percentUsed}% used`);
console.log(`Current cost: $${data.currentCost.toFixed(4)}`);
});
Track Usage#
typescript
// Get total usage stats
const stats = provider.getUsageStats();
console.log(`Total tokens: ${stats.totalTokens}`);
console.log(`Estimated cost: $${stats.estimatedCost.toFixed(4)}`);
// Get per-model breakdown
const byModel = provider.getUsageByModel();
for (const [model, usage] of Object.entries(byModel)) {
console.log(`${model}: ${usage.totalTokens} tokens, $${usage.estimatedCost.toFixed(4)}`);
}
Fallback Chains#
Configure Model Fallbacks#
typescript
provider.configureFallbackChain({
chat: [
'anthropic/claude-sonnet-4.5',
'openai/gpt-4o',
'google/gemini-2.5-pro-preview-05-06'
],
embeddings: [
'openai/text-embedding-3-small',
'voyage-ai/voyage-3'
],
images: [
'google/gemini-2.5-flash-image-preview',
'openai/dall-e-3'
]
});
Handle Model Failures#
typescript
try {
await provider.complete({ user_prompt: 'Hello' });
} catch (error) {
// Mark model as failed for fallback
provider.markModelFailed('anthropic/claude-sonnet-4.5');
// Get next available model from chain
const chain = provider.getFallbackChain();
const failedModels = provider.getFailedModels();
const nextModel = chain.chat.find(m => !failedModels.includes(m));
// Retry with fallback
await provider.complete({
user_prompt: 'Hello',
model: nextModel
});
}
Circuit Breaker#
The provider includes an automatic circuit breaker for resilience:
typescript
// Check if provider is available
if (provider.isAvailable()) {
await provider.complete({ user_prompt: 'Hello' });
}
// Listen for circuit breaker events
provider.on('circuit-open', (data) => {
console.log(`Circuit breaker opened: ${data.error}`);
});
// Get current state
const config = provider.getConfig();
console.log(`Circuit state: ${config.circuitBreaker.state}`);
console.log(`Failures: ${config.circuitBreaker.failures}`);
Circuit Breaker States#
- closed: Normal operation, requests allowed
- open: Too many failures, requests blocked
- half-open: Testing if service recovered
Health Checks#
typescript
const healthy = await provider.healthCheck();
if (!healthy) {
console.log('OpenRouter API is unreachable');
}
Event Handling#
typescript
// Request completion events
provider.on('request-complete', (data) => {
console.log(`Latency: ${data.latency}ms`);
console.log(`Model: ${data.model}`);
console.log(`Tokens: ${data.tokenUsage.total_tokens}`);
});
// Budget events
provider.on('budget-warning', (data) => {
console.log(`Budget ${data.percentUsed}% used`);
});
// Circuit breaker events
provider.on('circuit-open', (data) => {
console.log(`Provider unavailable: ${data.error}`);
});
Error Handling#
typescript
import { OpenRouterError } from '../providers/openrouter-provider.js';
try {
await provider.complete({ user_prompt: 'Hello' });
} catch (error) {
if (error instanceof OpenRouterError) {
switch (error.code) {
case 401:
console.log('Invalid API key');
break;
case 402:
console.log('Budget limit exceeded');
break;
case 429:
console.log('Rate limited, retry later');
break;
case 503:
console.log('Circuit breaker open');
break;
default:
console.log(`API error ${error.code}: ${error.message}`);
}
}
}
Best Practices#
1. Use Appropriate Models#
typescript
// Fast, cheap for simple tasks
const simple = await provider.complete({
model: 'openai/gpt-4o-mini',
user_prompt: 'Summarize this text'
});
// Powerful for complex reasoning
const complex = await provider.complete({
model: 'anthropic/claude-sonnet-4.5',
user_prompt: 'Analyze this philosophical argument'
});
2. Optimize Token Usage#
typescript
// Limit output for cost control
const response = await provider.complete({
user_prompt: 'Brief summary',
max_tokens: 100 // Cap output tokens
});
// Use smaller embedding dimensions
const embeddings = await provider.createEmbedding({
input: 'Text',
dimensions: 512 // Instead of 1536
});
3. Handle Rate Limits#
typescript
// Built-in retry with exponential backoff
const provider = new OpenRouterProvider({
maxRetries: 3,
timeout: 30000
});
4. Monitor Costs#
typescript
// Always set budget limits in production
provider.setBudgetLimit(100);
provider.setBudgetWarningThreshold(0.7); // Warn at 70%
// Periodically check usage
setInterval(() => {
const stats = provider.getUsageStats();
if (stats.estimatedCost > 50) {
console.warn('High usage detected');
}
}, 60000);
Recommended Models#
Chat/Text Generation#
| Use Case | Model | Context | Notes |
|---|---|---|---|
| General | anthropic/claude-sonnet-4.5 |
200K | Best balance |
| Fast/Cheap | openai/gpt-4o-mini |
128K | Quick responses |
| Long Context | google/gemini-2.5-pro-preview-05-06 |
2M | Massive context |
| Coding | anthropic/claude-sonnet-4.5 |
200K | Code generation |
Embeddings#
| Use Case | Model | Dimensions | Notes |
|---|---|---|---|
| General | openai/text-embedding-3-small |
1536 | Good default |
| High Quality | openai/text-embedding-3-large |
3072 | Better accuracy |
| Fast Search | Any with dimensions: 512 |
512 | Faster similarity |
Image Generation#
| Use Case | Model | Notes |
|---|---|---|
| Fast | google/gemini-2.5-flash-image-preview |
Quick generation |
| Quality | openai/dall-e-3 |
Higher quality |