Eve library

Cost Optimization Guide

Guide

Revision 1

On this page

This guide covers strategies for optimizing AI costs across OpenRouter, ElevenLabs, and GPU compute.

Overview#

AI costs come from three main sources:

  1. Language Models (OpenRouter): Token-based pricing
  2. Text-to-Speech (ElevenLabs): Character-based pricing
  3. GPU Compute (RunPod): Time-based pricing

OpenRouter Cost Optimization#

Token Cost Reference#

Model Input (per 1M) Output (per 1M) Relative
gpt-4o-mini $0.15 $0.60 $
claude-sonnet-4.5 $3.00 $15.00 $$$
gpt-4o $2.50 $10.00 $$$
gemini-2.5-pro $1.25 $5.00 $$
llama-3.3-70b ~$0.90 ~$0.90 $

Strategy 1: Use Tiered Models#

Route tasks to appropriate cost tiers:

typescript
class ModelRouter {
  selectModel(task: Task): string {
    switch (task.complexity) {
      case 'simple':
        return 'openai/gpt-4o-mini';        // $0.15/M input
      case 'medium':
        return 'meta-llama/llama-3.3-70b-instruct';  // $0.90/M
      case 'complex':
        return 'anthropic/claude-sonnet-4.5';  // $3.00/M input
    }
  }
}

Cost Impact: 80-95% savings on simple tasks.

Strategy 2: Optimize Token Usage#

Reduce Input Tokens#

typescript
// Bad: Verbose prompt
const badPrompt = `
  I would like you to please generate a meditation script
  for me. The meditation should be about mindfulness and
  should be suitable for beginners. Please make it calming
  and peaceful. It should be around 5 minutes long.
`;

// Good: Concise prompt
const goodPrompt = `
  Generate a 5-minute beginner mindfulness meditation.
  Style: calm, peaceful.
`;

Savings: 50-70% on input tokens.

Limit Output Tokens#

typescript
// Set appropriate limits
const quote = await provider.complete({
  user_prompt: 'Generate a meditation quote',
  max_tokens: 50  // Quotes are short
});

const script = await provider.complete({
  user_prompt: 'Generate 5-minute meditation',
  max_tokens: 1500  // ~10 minutes of speech
});

Strategy 3: Batch Similar Requests#

typescript
// Bad: Individual requests
for (const theme of themes) {
  await generateQuote(theme);  // 10 API calls
}

// Good: Batch request
const quotes = await provider.complete({
  user_prompt: `Generate 10 quotes, one for each theme: ${themes.join(', ')}`,
  response_format: { type: 'json_object' }
});

Savings: 60-80% overhead reduction.

Strategy 4: Cache Results#

typescript
class ContentCache {
  private cache = new Map<string, CachedContent>();

  async getOrGenerate(key: string, generator: () => Promise<string>): Promise<string> {
    const cached = this.cache.get(key);
    if (cached && !this.isExpired(cached)) {
      return cached.content;  // Free!
    }

    const content = await generator();
    this.cache.set(key, { content, timestamp: Date.now() });
    return content;
  }
}

Savings: 100% for cached content.

Strategy 5: Use Fallback Chains#

typescript
// Start cheap, escalate if needed
async function generateWithFallback(prompt: string): Promise<string> {
  try {
    // Try cheap model first
    return await provider.complete({
      model: 'openai/gpt-4o-mini',
      user_prompt: prompt
    });
  } catch {
    // Fall back to premium only if needed
    return await provider.complete({
      model: 'anthropic/claude-sonnet-4.5',
      user_prompt: prompt
    });
  }
}

Strategy 6: Reduce Embedding Dimensions#

typescript
// Standard: 1536 dimensions
const standard = await provider.createEmbedding({
  input: texts,
  model: 'openai/text-embedding-3-small'
});

// Optimized: 512 dimensions (67% less storage, faster search)
const optimized = await provider.createEmbedding({
  input: texts,
  model: 'openai/text-embedding-3-small',
  dimensions: 512
});

Savings: 67% storage, faster vector search.

Budget Controls#

typescript
// Set hard budget limits
provider.setBudgetLimit(100);  // $100 limit
provider.setBudgetWarningThreshold(0.7);  // Warn at 70%

provider.on('budget-warning', (data) => {
  console.warn(`Budget ${data.percentUsed}% used`);
  // Switch to cheaper models
  switchToCostOptimizedMode();
});

ElevenLabs Cost Optimization#

Character Cost Reference#

Model Credits per Char Relative
eleven_flash_v2_5 0.5 $
eleven_turbo_v2_5 1.0 $$
eleven_multilingual_v2 1.0 $$
eleven_v3 1.0 $$

Strategy 1: Choose Right Model#

typescript
// Real-time: Use Flash (50% cheaper)
const realtime = await provider.synthesize({
  text: response,
  model_id: 'eleven_flash_v2_5'
});

// Pre-recorded: Use v3 (better quality, same price as turbo)
const prerecorded = await provider.synthesize({
  text: script,
  model_id: 'eleven_v3'
});

Strategy 2: Optimize Text Length#

typescript
// Remove unnecessary words
function optimizeForTTS(text: string): string {
  return text
    .replace(/\s+/g, ' ')           // Remove extra whitespace
    .replace(/\.{3,}/g, '...')      // Limit ellipsis
    .trim();
}

Strategy 3: Use Audio Tags Instead of Text#

typescript
// Bad: Text-based pauses
const bad = 'Breathe in. ... ... ... Breathe out.';

// Good: Audio tag pauses (no extra characters)
const good = 'Breathe in. <break time="3s" /> Breathe out.';

Savings: Audio tags don't count as characters.

Strategy 4: Cache Common Phrases#

typescript
class TTSCache {
  private audioCache = new Map<string, Buffer>();

  async synthesize(text: string): Promise<Buffer> {
    const cacheKey = this.getCacheKey(text);

    if (this.audioCache.has(cacheKey)) {
      return this.audioCache.get(cacheKey)!;
    }

    const audio = await this.provider.synthesize({ text });
    this.audioCache.set(cacheKey, audio);
    return audio;
  }
}

Strategy 5: Batch Pre-Generation#

typescript
// Pre-generate common content during off-peak
const commonPhrases = [
  'Welcome to your meditation practice.',
  'Find a comfortable position.',
  'Take a deep breath.',
  // ... more common phrases
];

// Generate and cache all at once
await preGenerateAudio(commonPhrases);

GPU Compute Cost Optimization (RunPod)#

GPU Cost Reference#

GPU Cost/Hour VRAM Best For
RTX A4000 $0.20 16GB Inference
RTX A5000 $0.35 24GB Training
RTX A6000 $0.79 48GB Large models
RTX 4090 $0.44 24GB Fast inference
H100 $4.69 80GB Training

Strategy 1: Right-Size GPU#

typescript
function selectGPU(task: GPUTask): GPUType {
  if (task.type === 'inference' && task.modelSize < 7) {
    return 'NVIDIA RTX A4000';  // $0.20/hr
  }
  if (task.type === 'inference' && task.modelSize < 30) {
    return 'NVIDIA GeForce RTX 4090';  // $0.44/hr
  }
  if (task.type === 'training') {
    return 'NVIDIA RTX A6000';  // $0.79/hr
  }
  return 'NVIDIA H100';  // Large scale
}

Strategy 2: Use Serverless for Bursty Workloads#

typescript
// Serverless scales to zero when idle
const config = {
  minWorkers: 0,      // Scale to zero
  maxWorkers: 10,
  idleTimeout: 60,    // Shut down after 1 minute idle
  flashBoot: true     // Fast cold starts
};

Savings: Only pay for actual compute time.

Strategy 3: Batch Processing#

typescript
// Process multiple items per GPU session
async function batchProcess(items: Item[]): Promise<Result[]> {
  const batchSize = 16;  // Process 16 at once
  const results: Result[] = [];

  for (let i = 0; i < items.length; i += batchSize) {
    const batch = items.slice(i, i + batchSize);
    const batchResults = await processGPUBatch(batch);
    results.push(...batchResults);
  }

  return results;
}

Strategy 4: Use Warm Pools Strategically#

typescript
// Keep warm during peak hours only
const warmPoolConfig = {
  enabled: true,
  poolSize: 2,
  schedule: {
    warmHours: [8, 9, 10, 11, 12, 13, 14, 15, 16, 17],  // Business hours
    timezone: 'America/New_York'
  }
};

Strategy 5: Network Volume Caching#

typescript
// Cache models on network volumes
const volumeConfig = {
  size: 100,  // 100GB
  preloadModels: [
    'stabilityai/stable-diffusion-xl-base-1.0',
    'whisper-large-v3'
  ]
};

Savings: Avoid model download time on cold starts.

Cost Monitoring Dashboard#

Track Costs by Service#

typescript
class CostTracker {
  private costs = {
    openrouter: 0,
    elevenlabs: 0,
    runpod: 0
  };

  recordOpenRouterUsage(tokens: number, model: string): void {
    const pricing = MODEL_PRICING[model];
    this.costs.openrouter += (tokens / 1000) * pricing.avgCost;
  }

  recordElevenLabsUsage(characters: number): void {
    this.costs.elevenlabs += (characters / 1000) * 0.30;  // ~$0.30/1K chars
  }

  recordRunPodUsage(gpuHours: number, gpuType: GPUType): void {
    this.costs.runpod += gpuHours * GPU_HOURLY_RATES[gpuType];
  }

  getDailyReport(): CostReport {
    return {
      total: Object.values(this.costs).reduce((a, b) => a + b, 0),
      breakdown: this.costs,
      projectedMonthly: this.calculateProjection()
    };
  }
}

Set Alerts#

typescript
const alertConfig = {
  hourly: 10,    // $10/hour max
  daily: 100,    // $100/day max
  monthly: 2000  // $2000/month max
};

tracker.on('threshold-exceeded', (alert) => {
  sendSlackNotification(`Cost alert: ${alert.period} exceeded ${alert.threshold}`);

  if (alert.severity === 'critical') {
    switchToEmergencyMode();  // Use cheapest models only
  }
});

Cost Comparison: Before vs After Optimization#

Meditation Script Generation#

Aspect Before After Savings
Model Claude Sonnet for all Tiered selection 60%
Tokens Verbose prompts Optimized prompts 40%
Caching None Hash-based cache 30%
TTS Full regeneration Phrase caching 50%
Total $0.50/script $0.12/script 76%

Daily Content Generation#

Aspect Before After Savings
Quotes Claude Sonnet GPT-4o-mini 95%
Batching Individual calls Batch requests 70%
Caching None 24-hour cache 80%
Total $5.00/day $0.50/day 90%

Monthly Cost Estimates#

Low Usage (Startup)#

Service Monthly Cost
OpenRouter $50-100
ElevenLabs $22 (Creator)
RunPod $50-100
Total $122-222

Medium Usage (Growing)#

Service Monthly Cost
OpenRouter $300-500
ElevenLabs $99 (Pro)
RunPod $300-500
Total $699-1,099

High Usage (Scale)#

Service Monthly Cost
OpenRouter $1,000-3,000
ElevenLabs $330 (Scale)
RunPod $1,000-2,000
Total $2,330-5,330

Best Practices Checklist#

  • Use GPT-4o-mini for simple tasks
  • Set budget limits and warnings
  • Implement response caching
  • Batch similar requests
  • Optimize prompt length
  • Use Eleven Flash for real-time
  • Cache common TTS phrases
  • Right-size GPU selection
  • Use serverless for variable load
  • Monitor costs daily
  • Set spending alerts
  • Review usage weekly

Emergency Cost Reduction#

When costs spike unexpectedly:

typescript
async function emergencyMode(): Promise<void> {
  // 1. Switch all LLM calls to cheapest model
  defaultModel = 'openai/gpt-4o-mini';

  // 2. Disable GPU-intensive features
  gpuFeatures.disable();

  // 3. Increase cache TTL
  cacheConfig.ttl = 86400;  // 24 hours

  // 4. Reduce generation frequency
  generationConfig.rate = 0.1;  // 10% of normal

  // 5. Enable strict budget enforcement
  budgetEnforcer.setStrict(true);
}