Eve library

OpenRouter Integration

Guide

Revision 1

On this page

Reference documentation for the OpenRouter AI provider supporting chat completions, embeddings, image generation, model routing, and cost management.

Overview#

The OpenRouter provider offers:

  • Chat Completions: Text generation with streaming and function calling
  • Embeddings: Text embeddings for semantic search
  • Image Generation: Image creation via modalities parameter
  • Model Capabilities: Dynamic model discovery and filtering
  • Cost Tracking: Token counting and budget management
  • Circuit Breaker: Automatic failure recovery
  • Fallback Chains: Automatic model failover

Configuration#

Environment Variables#

Variable Description Required
OPENROUTER_API_KEY OpenRouter API key Yes

Provider Initialization#

typescript
import { OpenRouterProvider } from './providers/openrouter-provider';

const provider = new OpenRouterProvider({
  apiKey: process.env.OPENROUTER_API_KEY,
  baseURL: 'https://openrouter.ai/api/v1',
  defaultModel: 'anthropic/claude-sonnet-4.5',
  defaultEmbeddingModel: 'openai/text-embedding-3-small',
  defaultImageModel: 'google/gemini-2.5-flash-image-preview',
  maxRetries: 3,
  timeout: 30000,
  maxFailures: 5, // Circuit breaker threshold
  resetTimeout: 60000, // Circuit breaker reset time
  appName: 'Lilith',
  appUrl: 'https://lilith.ai',
  modelCacheTTL: 3600000, // 1 hour
});

Configuration Options#

Option Type Default Description
apiKey string OPENROUTER_API_KEY API key
baseURL string https://openrouter.ai/api/v1 API base URL
defaultModel string anthropic/claude-sonnet-4.5 Default chat model
defaultEmbeddingModel string openai/text-embedding-3-small Default embedding model
defaultImageModel string google/gemini-2.5-flash-image-preview Default image model
maxRetries number 3 Max retry attempts
timeout number 30000 Request timeout (ms)
maxFailures number 5 Failures before circuit opens
resetTimeout number 60000 Circuit reset time (ms)
appName string Lilith App name header
appUrl string https://lilith.ai App URL header
modelCacheTTL number 3600000 Model cache TTL (ms)

Chat Completions#

Basic Completion#

typescript
const response = await provider.complete({
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'What is meditation?' },
  ],
  model: 'anthropic/claude-sonnet-4.5',
  max_tokens: 1000,
  temperature: 0.7,
});

console.log(response.content);
// { content, role, tool_calls, finish_reason, usage, model, id, provider, latency }

Alternative Prompt Format#

typescript
const response = await provider.complete({
  system_prompt: 'You are a Zen master.',
  user_prompt: 'What is the sound of one hand clapping?',
  temperature: 0.8,
});

Completion Parameters#

Parameter Type Description
messages Message[] Array of chat messages
system_prompt string System message (alternative to messages)
user_prompt string User message (alternative to messages)
model string Model ID (optional)
max_tokens number Max response tokens
temperature number Sampling temperature (0-2)
top_p number Nucleus sampling
frequency_penalty number Repetition penalty
presence_penalty number Topic diversity
tools Tool[] Function definitions
tool_choice string Tool selection mode
response_format object Response format (e.g., JSON mode)
stream boolean Enable streaming

Streaming Completion#

typescript
const stream = await provider.completeStream({
  messages: [{ role: 'user', content: 'Tell me a story.' }],
  max_tokens: 500,
});

for await (const chunk of stream) {
  const content = chunk.choices?.[0]?.delta?.content;
  if (content) {
    process.stdout.write(content);
  }
}

Function Calling#

typescript
const response = await provider.complete({
  messages: [{ role: 'user', content: 'What is the weather in Tokyo?' }],
  tools: [
    {
      type: 'function',
      function: {
        name: 'get_weather',
        description: 'Get weather for a location',
        parameters: {
          type: 'object',
          properties: {
            location: { type: 'string', description: 'City name' },
          },
          required: ['location'],
        },
      },
    },
  ],
  tool_choice: 'auto',
});

if (response.tool_calls) {
  for (const call of response.tool_calls) {
    console.log(`Function: ${call.function.name}`);
    console.log(`Arguments: ${call.function.arguments}`);
  }
}

JSON Mode#

typescript
const response = await provider.complete({
  messages: [{ role: 'user', content: 'List 3 meditation types as JSON' }],
  response_format: { type: 'json_object' },
});

const data = JSON.parse(response.content);

Completion Response#

typescript
interface CompletionResponse {
  content: string;
  role: string;
  tool_calls?: any[];
  finish_reason: string;
  usage: {
    prompt_tokens: number;
    completion_tokens: number;
    total_tokens: number;
  };
  model: string;
  id: string;
  provider: string;
  latency?: number;
}

Embeddings#

Create Embedding#

typescript
const response = await provider.createEmbedding({
  input: 'Mindfulness is the practice of present moment awareness.',
  model: 'openai/text-embedding-3-small',
  dimensions: 1536,
});

console.log(response.data[0].embedding);
// [0.012, -0.034, 0.056, ...]

Batch Embeddings#

typescript
const response = await provider.createEmbedding({
  input: ['First text to embed', 'Second text to embed', 'Third text to embed'],
});

for (const item of response.data) {
  console.log(`Index ${item.index}: ${item.embedding.length} dimensions`);
}

Embedding Response#

typescript
interface OpenRouterEmbeddingResponse {
  object: string;
  model: string;
  data: Array<{
    object: string;
    embedding: number[];
    index: number;
  }>;
  usage: {
    prompt_tokens: number;
    total_tokens: number;
  };
}

Image Generation#

Generate Image#

typescript
const response = await provider.generateImage({
  prompt: 'A serene zen garden with a stone path',
  model: 'google/gemini-2.5-flash-image-preview',
  imageConfig: {
    aspectRatio: '16:9',
    imageSize: '2K',
  },
});

for (const imageUrl of response.images) {
  console.log('Generated image:', imageUrl);
}

Image Configuration#

Option Values Description
aspectRatio 1:1, 4:3, 3:4, 16:9, 9:16, 21:9 Image aspect ratio
imageSize 1K, 2K, 4K Image resolution

Image Response#

typescript
interface OpenRouterImageResponse {
  id: string;
  model: string;
  images: string[];
  usage: {
    prompt_tokens: number;
    completion_tokens: number;
    total_tokens: number;
  };
}

Model Capabilities#

Fetch Available Models#

typescript
const models = await provider.fetchModelCapabilities();

for (const model of models) {
  console.log(`${model.name} (${model.id})`);
  console.log(`  Context: ${model.contextLength} tokens`);
  console.log(`  Input: ${model.inputModalities.join(', ')}`);
  console.log(`  Output: ${model.outputModalities.join(', ')}`);
  console.log(
    `  Pricing: $${model.pricing.prompt}/1K prompt, $${model.pricing.completion}/1K completion`
  );
}

Model Capabilities#

typescript
interface OpenRouterModelCapabilities {
  id: string;
  name: string;
  contextLength: number;
  pricing: {
    prompt: string; // Price per 1K tokens
    completion: string;
  };
  inputModalities: string[]; // ['text'], ['text', 'image'], etc.
  outputModalities: string[]; // ['text'], ['embedding'], ['image'], etc.
  architecture?: {
    modality: string;
  };
}

Filter by Capability#

typescript
// Get embedding models
const embeddingModels = provider.getEmbeddingModels();

// Get image generation models
const imageModels = provider.getImageGenerationModels();

// Custom filtering
const visionModels = provider.getModelsByCapability({
  inputModalities: ['text', 'image'],
  outputModalities: ['text'],
});

Model Routing#

Fallback Chains#

Configure automatic model failover:

typescript
provider.configureFallbackChain({
  chat: [
    'anthropic/claude-sonnet-4.5',
    'openai/gpt-4o',
    'google/gemini-1.5-pro',
  ],
  embeddings: [
    'openai/text-embedding-3-small',
    'openai/text-embedding-3-large',
  ],
  images: ['google/gemini-2.5-flash-image-preview', 'openai/dall-e-3'],
});

Use with Fallback#

typescript
// Automatically tries fallback models on failure
const response = await provider.completeWithFallback({
  messages: [{ role: 'user', content: 'Hello!' }],
});

const embedding = await provider.createEmbeddingWithFallback({
  input: 'Text to embed',
});

const image = await provider.generateImageWithFallback({
  prompt: 'A beautiful sunset',
});

Fallback Events#

typescript
provider.on('fallback-triggered', ({ failedModel, fallbackModel, error }) => {
  console.log(`Fallback: ${failedModel} -> ${fallbackModel}`);
  console.log(`Reason: ${error}`);
});

Failed Model Tracking#

typescript
// Get list of failed models
const failed = provider.getFailedModels();

// Clear failed models (allow retry)
provider.clearFailedModels();

Cost Management#

Token Tracking#

Usage is automatically tracked for all requests:

typescript
const stats = provider.getUsageStats();
// {
//   totalPromptTokens: 15000,
//   totalCompletionTokens: 8000,
//   totalTokens: 23000,
//   requestCount: 150,
//   estimatedCost: 0.45
// }

Per-Model Usage#

typescript
const byModel = provider.getUsageByModel();
// {
//   'anthropic/claude-sonnet-4.5': { totalTokens: 15000, estimatedCost: 0.30 },
//   'openai/gpt-4o': { totalTokens: 8000, estimatedCost: 0.15 }
// }

Budget Limits#

typescript
// Set budget limit (in dollars)
provider.setBudgetLimit(10.00);

// Set warning threshold (80% by default)
provider.setBudgetWarningThreshold(0.8);

// Listen for warnings
provider.on('budget-warning', ({ currentCost, budgetLimit, percentUsed }) => {
  console.log(`Warning: ${percentUsed.toFixed(1)}% of budget used`);
  console.log(`Current: $${currentCost.toFixed(2)} / $${budgetLimit}`);
});

// Requests throw error when budget exceeded
try {
  await provider.complete({ ... });
} catch (error) {
  if (error.code === 402) {
    console.log('Budget limit exceeded');
  }
}

Circuit Breaker#

Availability Check#

typescript
if (provider.isAvailable()) {
  const response = await provider.complete({ ... });
} else {
  console.log('Provider unavailable, circuit is open');
}

Circuit Breaker States#

State Description
closed Normal operation
open Rejecting requests after failures
half-open Testing with single request

Circuit Breaker Events#

typescript
provider.on('circuit-open', ({ provider, error }) => {
  console.log(`Circuit opened for ${provider}: ${error}`);
  // Alert monitoring system
});

Configuration#

typescript
// 5 failures opens circuit
maxFailures: 5;

// 60 second reset timeout
resetTimeout: 60000;

State Transitions#

text
CLOSED → 5 failures → OPEN
    ↓
OPEN (60 second wait)
    ↓
HALF-OPEN (test request)
    ↓
    ├─ Success → CLOSED
    │
    └─ Failure → OPEN

Retry Logic#

Exponential Backoff#

Requests are automatically retried with exponential backoff:

typescript
// Retry delays: 1s, 2s, 4s, 8s... (max 10s)
const backoffMs = Math.min(1000 * Math.pow(2, attempt - 1), 10000);

Retryable Errors#

Status Description Retried
429 Rate limited Yes
500+ Server errors Yes
4xx Client errors No

Health Check#

typescript
const healthy = await provider.healthCheck();
// true if API responds within 5 seconds

Events#

The provider emits events for monitoring:

Event Data Description
request-complete { latency, model, tokenUsage } Request completed
circuit-open { provider, error } Circuit breaker opened
budget-warning { currentCost, budgetLimit, percentUsed } Budget threshold reached
fallback-triggered { failedModel, fallbackModel, error } Model failover occurred

Event Listening#

typescript
provider.on('request-complete', ({ latency, model, tokenUsage }) => {
  console.log(`${model}: ${tokenUsage.total_tokens} tokens, ${latency}ms`);
});

provider.on('circuit-open', ({ provider, error }) => {
  alerting.send(`OpenRouter circuit open: ${error}`);
});

Error Handling#

OpenRouterError#

typescript
class OpenRouterError extends Error {
  code: number; // HTTP status code
  providerName?: string; // Upstream provider name
}

Error Codes#

Code Description
400 Bad request (invalid parameters)
401 Invalid API key
402 Budget limit exceeded
429 Rate limited
500 Internal server error
503 Circuit breaker open

Error Handling Example#

typescript
try {
  const response = await provider.complete({ ... });
} catch (error) {
  if (error instanceof OpenRouterError) {
    switch (error.code) {
      case 402:
        console.log('Budget exceeded');
        break;
      case 429:
        console.log('Rate limited, retry after cooldown');
        break;
      case 503:
        console.log('Service unavailable');
        break;
      default:
        console.log(`Error ${error.code}: ${error.message}`);
    }
  }
}

Cleanup#

typescript
// Shutdown provider and release resources
provider.shutdown();