Eve library

OpenRouter Integration Guide

Guide

Revision 1

On this page

This guide covers integration with the OpenRouter API for text generation, embeddings, and image generation.

Overview#

The OpenRouterProvider provides a unified interface to 200+ AI models through the OpenRouter API, including:

  • Chat Completions: Claude, GPT-4, Llama, Gemini, and more
  • Embeddings: OpenAI text-embedding-3, voyage-ai, and others
  • Image Generation: Gemini 2.5 Flash, DALL-E, Stable Diffusion via modality routing

Installation & Setup#

Environment Variables#

bash
export OPENROUTER_API_KEY="your-api-key"

Basic Initialization#

typescript
import { OpenRouterProvider } from '../providers/openrouter-provider.js';

const provider = new OpenRouterProvider({
  apiKey: process.env.OPENROUTER_API_KEY,
  defaultModel: 'anthropic/claude-sonnet-4.5',
  defaultEmbeddingModel: 'openai/text-embedding-3-small',
  defaultImageModel: 'google/gemini-2.5-flash-image-preview',
  maxRetries: 3,
  timeout: 30000
});

Configuration Options#

Option Type Default Description
apiKey string OPENROUTER_API_KEY env Your OpenRouter API key
baseURL string https://openrouter.ai/api/v1 API base URL
defaultModel string anthropic/claude-sonnet-4.5 Default chat model
defaultEmbeddingModel string openai/text-embedding-3-small Default embedding model
defaultImageModel string google/gemini-2.5-flash-image-preview Default image model
maxRetries number 3 Max retry attempts
timeout number 30000 Request timeout (ms)
maxFailures number 5 Circuit breaker threshold
resetTimeout number 60000 Circuit breaker reset (ms)
appName string Lilith App name for tracking
appUrl string https://lilith.ai App URL for tracking
modelCacheTTL number 3600000 Model cache TTL (ms)

Chat Completions#

Basic Text Generation#

typescript
const response = await provider.complete({
  user_prompt: 'Write a haiku about meditation',
  max_tokens: 100,
  temperature: 0.7
});

console.log(response.content);
// Output: "Still water reflects / Thoughts drift like autumn leaves fall / Peace in emptiness"

With System Prompt#

typescript
const response = await provider.complete({
  system_prompt: 'You are a meditation teacher with 20 years of experience.',
  user_prompt: 'Explain the benefits of mindfulness.',
  model: 'anthropic/claude-sonnet-4.5',
  max_tokens: 500
});

Using Messages Array#

typescript
const response = await provider.complete({
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'What is mindfulness?' },
    { role: 'assistant', content: 'Mindfulness is the practice of...' },
    { role: 'user', content: 'How do I get started?' }
  ],
  max_tokens: 300
});

Streaming Responses#

typescript
const stream = await provider.completeStream({
  messages: [{ role: 'user', content: 'Tell me a story about zen.' }],
  max_tokens: 1000
});

for await (const chunk of stream) {
  if (chunk.choices?.[0]?.delta?.content) {
    process.stdout.write(chunk.choices[0].delta.content);
  }
}

Tool Use (Function Calling)#

typescript
const response = await provider.complete({
  messages: [{ role: 'user', content: 'What is the weather in Tokyo?' }],
  tools: [
    {
      type: 'function',
      function: {
        name: 'get_weather',
        description: 'Get current weather for a location',
        parameters: {
          type: 'object',
          properties: {
            location: { type: 'string', description: 'City name' }
          },
          required: ['location']
        }
      }
    }
  ],
  tool_choice: 'auto'
});

if (response.tool_calls) {
  // Handle tool calls
  console.log(response.tool_calls);
}

JSON Mode#

typescript
const response = await provider.complete({
  user_prompt: 'List 3 meditation techniques as JSON',
  response_format: { type: 'json_object' },
  max_tokens: 500
});

const data = JSON.parse(response.content);

Embeddings#

Single Text Embedding#

typescript
const response = await provider.createEmbedding({
  input: 'Mindfulness is the practice of being present.',
  model: 'openai/text-embedding-3-small'
});

const embedding = response.data[0].embedding;
// Returns: number[] with 1536 dimensions

Batch Embeddings#

typescript
const response = await provider.createEmbedding({
  input: [
    'First meditation technique',
    'Second meditation technique',
    'Third meditation technique'
  ]
});

const embeddings = response.data.map(d => d.embedding);

With Custom Dimensions#

typescript
const response = await provider.createEmbedding({
  input: 'Text to embed',
  model: 'openai/text-embedding-3-large',
  dimensions: 512  // Reduce dimensions for faster search
});

Image Generation#

Basic Image Generation#

typescript
const response = await provider.generateImage({
  prompt: 'A serene zen garden at sunset, watercolor style',
  model: 'google/gemini-2.5-flash-image-preview'
});

const imageBase64 = response.images[0];

With Image Configuration#

typescript
const response = await provider.generateImage({
  prompt: 'A lotus flower floating on still water',
  imageConfig: {
    aspectRatio: '16:9',
    imageSize: '2K'
  },
  numImages: 2
});

Aspect Ratio Options#

  • 1:1 - Square (default)
  • 4:3 - Landscape traditional
  • 3:4 - Portrait traditional
  • 16:9 - Widescreen landscape
  • 9:16 - Widescreen portrait
  • 21:9 - Ultra-wide

Image Size Options#

  • 1K - Lower resolution, faster
  • 2K - Standard resolution
  • 4K - High resolution, slower

Model Discovery#

Fetch Available Models#

typescript
const models = await provider.fetchModelCapabilities();

console.log(`Found ${models.length} models`);
models.forEach(m => {
  console.log(`${m.id}: ${m.name} (context: ${m.contextLength})`);
});

Filter by Capability#

typescript
// Get all embedding models
const embeddingModels = provider.getEmbeddingModels();

// Get all image generation models
const imageModels = provider.getImageGenerationModels();

// Custom filter
const multimodalModels = provider.getModelsByCapability({
  inputModalities: ['text', 'image'],
  outputModalities: ['text']
});

Model Capabilities Response#

typescript
interface OpenRouterModelCapabilities {
  id: string;                    // e.g., 'anthropic/claude-sonnet-4.5'
  name: string;                  // e.g., 'Claude Sonnet 4.5'
  contextLength: number;         // e.g., 200000
  pricing: {
    prompt: string;              // Cost per 1K tokens (e.g., '0.003')
    completion: string;          // Cost per 1K tokens (e.g., '0.015')
  };
  inputModalities: string[];     // e.g., ['text', 'image']
  outputModalities: string[];    // e.g., ['text']
}

Budget & Cost Tracking#

Set Budget Limit#

typescript
// Set a $10 budget limit
provider.setBudgetLimit(10.0);

// Set warning threshold at 80%
provider.setBudgetWarningThreshold(0.8);

// Listen for budget warnings
provider.on('budget-warning', (data) => {
  console.log(`Budget warning: ${data.percentUsed}% used`);
  console.log(`Current cost: $${data.currentCost.toFixed(4)}`);
});

Track Usage#

typescript
// Get total usage stats
const stats = provider.getUsageStats();
console.log(`Total tokens: ${stats.totalTokens}`);
console.log(`Estimated cost: $${stats.estimatedCost.toFixed(4)}`);

// Get per-model breakdown
const byModel = provider.getUsageByModel();
for (const [model, usage] of Object.entries(byModel)) {
  console.log(`${model}: ${usage.totalTokens} tokens, $${usage.estimatedCost.toFixed(4)}`);
}

Fallback Chains#

Configure Model Fallbacks#

typescript
provider.configureFallbackChain({
  chat: [
    'anthropic/claude-sonnet-4.5',
    'openai/gpt-4o',
    'google/gemini-2.5-pro-preview-05-06'
  ],
  embeddings: [
    'openai/text-embedding-3-small',
    'voyage-ai/voyage-3'
  ],
  images: [
    'google/gemini-2.5-flash-image-preview',
    'openai/dall-e-3'
  ]
});

Handle Model Failures#

typescript
try {
  await provider.complete({ user_prompt: 'Hello' });
} catch (error) {
  // Mark model as failed for fallback
  provider.markModelFailed('anthropic/claude-sonnet-4.5');

  // Get next available model from chain
  const chain = provider.getFallbackChain();
  const failedModels = provider.getFailedModels();
  const nextModel = chain.chat.find(m => !failedModels.includes(m));

  // Retry with fallback
  await provider.complete({
    user_prompt: 'Hello',
    model: nextModel
  });
}

Circuit Breaker#

The provider includes an automatic circuit breaker for resilience:

typescript
// Check if provider is available
if (provider.isAvailable()) {
  await provider.complete({ user_prompt: 'Hello' });
}

// Listen for circuit breaker events
provider.on('circuit-open', (data) => {
  console.log(`Circuit breaker opened: ${data.error}`);
});

// Get current state
const config = provider.getConfig();
console.log(`Circuit state: ${config.circuitBreaker.state}`);
console.log(`Failures: ${config.circuitBreaker.failures}`);

Circuit Breaker States#

  • closed: Normal operation, requests allowed
  • open: Too many failures, requests blocked
  • half-open: Testing if service recovered

Health Checks#

typescript
const healthy = await provider.healthCheck();
if (!healthy) {
  console.log('OpenRouter API is unreachable');
}

Event Handling#

typescript
// Request completion events
provider.on('request-complete', (data) => {
  console.log(`Latency: ${data.latency}ms`);
  console.log(`Model: ${data.model}`);
  console.log(`Tokens: ${data.tokenUsage.total_tokens}`);
});

// Budget events
provider.on('budget-warning', (data) => {
  console.log(`Budget ${data.percentUsed}% used`);
});

// Circuit breaker events
provider.on('circuit-open', (data) => {
  console.log(`Provider unavailable: ${data.error}`);
});

Error Handling#

typescript
import { OpenRouterError } from '../providers/openrouter-provider.js';

try {
  await provider.complete({ user_prompt: 'Hello' });
} catch (error) {
  if (error instanceof OpenRouterError) {
    switch (error.code) {
      case 401:
        console.log('Invalid API key');
        break;
      case 402:
        console.log('Budget limit exceeded');
        break;
      case 429:
        console.log('Rate limited, retry later');
        break;
      case 503:
        console.log('Circuit breaker open');
        break;
      default:
        console.log(`API error ${error.code}: ${error.message}`);
    }
  }
}

Best Practices#

1. Use Appropriate Models#

typescript
// Fast, cheap for simple tasks
const simple = await provider.complete({
  model: 'openai/gpt-4o-mini',
  user_prompt: 'Summarize this text'
});

// Powerful for complex reasoning
const complex = await provider.complete({
  model: 'anthropic/claude-sonnet-4.5',
  user_prompt: 'Analyze this philosophical argument'
});

2. Optimize Token Usage#

typescript
// Limit output for cost control
const response = await provider.complete({
  user_prompt: 'Brief summary',
  max_tokens: 100  // Cap output tokens
});

// Use smaller embedding dimensions
const embeddings = await provider.createEmbedding({
  input: 'Text',
  dimensions: 512  // Instead of 1536
});

3. Handle Rate Limits#

typescript
// Built-in retry with exponential backoff
const provider = new OpenRouterProvider({
  maxRetries: 3,
  timeout: 30000
});

4. Monitor Costs#

typescript
// Always set budget limits in production
provider.setBudgetLimit(100);
provider.setBudgetWarningThreshold(0.7);  // Warn at 70%

// Periodically check usage
setInterval(() => {
  const stats = provider.getUsageStats();
  if (stats.estimatedCost > 50) {
    console.warn('High usage detected');
  }
}, 60000);

Chat/Text Generation#

Use Case Model Context Notes
General anthropic/claude-sonnet-4.5 200K Best balance
Fast/Cheap openai/gpt-4o-mini 128K Quick responses
Long Context google/gemini-2.5-pro-preview-05-06 2M Massive context
Coding anthropic/claude-sonnet-4.5 200K Code generation

Embeddings#

Use Case Model Dimensions Notes
General openai/text-embedding-3-small 1536 Good default
High Quality openai/text-embedding-3-large 3072 Better accuracy
Fast Search Any with dimensions: 512 512 Faster similarity

Image Generation#

Use Case Model Notes
Fast google/gemini-2.5-flash-image-preview Quick generation
Quality openai/dall-e-3 Higher quality