Eve library

Rate Limits

Guide

Revision 1

On this page

Reference documentation for the Lilith API rate limiting system, quotas, and throttling behavior.

Overview#

The Lilith API implements a multi-layer rate limiting system:

  • HTTP Rate Limiting: Request-based limits with sliding window algorithm
  • Operation-Specific Limits: Endpoint-specific protections for sensitive operations
  • Token Quotas: Usage-based limits for AI and compute-intensive operations
  • Redis-Backed Distributed Limiting: Consistent limits across all API instances

Rate Limit Headers#

All API responses include rate limit information in headers:

Standard Headers (RFC 6585)#

Header Description Example
RateLimit-Limit Maximum requests allowed in window 100
RateLimit-Remaining Remaining requests in current window 95
RateLimit-Reset Unix timestamp when limit resets 1704067200
RateLimit-Policy Policy string 100;w=900

Legacy Headers#

For backward compatibility:

Header Description
X-RateLimit-Limit Maximum requests allowed
X-RateLimit-Remaining Remaining requests
X-RateLimit-Reset Reset timestamp

Default Rate Limits#

Global API Limits#

Plan Requests/Minute Requests/Hour Requests/Day
Free 12 720 17,280
Pro 60 3,600 86,400
Enterprise 300 18,000 432,000

Authentication Endpoints#

Higher security, stricter limits to prevent abuse:

Operation Window Max Attempts Purpose
Login 15 minutes 5 Brute force protection
Registration 1 hour 5 Registration spam prevention
Password Reset 1 hour 3 Abuse prevention
Email Verification Resend 1 hour 3 Email spam prevention
MFA Verification 5 minutes 5 MFA brute force protection

SMS/OTP Limits#

Limit Type Value Description
OTP Validity 5 minutes Time before OTP expires
Max Verification Attempts 5 Attempts before OTP invalidation
Resend Cooldown 60 seconds Minimum time between sends
Max Sends Per Hour 5 Maximum OTPs per phone per hour
Max Sends Per Day 10 Maximum OTPs per phone per day

Token Quotas (AI Endpoints)#

AI-powered endpoints have additional token-based quotas:

Plan Daily Tokens Daily Requests Monthly Tokens Monthly Requests
Free 10,000 100 100,000 1,000
Pro 100,000 1,000 1,000,000 10,000
Enterprise Unlimited 50,000 Unlimited 500,000

Token Usage Response#

AI endpoints return quota information in the response:

json
{
  "data": { ... },
  "usage": {
    "prompt_tokens": 150,
    "completion_tokens": 350,
    "total_tokens": 500
  },
  "quota": {
    "daily_tokens": 100000,
    "daily_tokens_remaining": 95000,
    "daily_requests": 1000,
    "daily_requests_remaining": 980,
    "monthly_tokens": 1000000,
    "monthly_tokens_remaining": 950000
  }
}

Rate Limit Exceeded Response#

When rate limited, the API returns HTTP 429:

http
HTTP/1.1 429 Too Many Requests
Content-Type: application/json
RateLimit-Limit: 12
RateLimit-Remaining: 0
RateLimit-Reset: 1704067260
RateLimit-Policy: 12;w=60
Retry-After: 45

{
  "error": {
    "type": "https://docs.lilith.app/errors/rate_limit_exceeded",
    "title": "Too Many Requests",
    "status": 429,
    "detail": "Rate limit exceeded. Please retry after 45 seconds.",
    "code": "rate_limit_exceeded",
    "retryAfter": 45
  }
}

Endpoint-Specific Limits#

Chat & Conversation#

Endpoint Limit Window
POST /v1/chat/completions 60/min 1 minute
POST /v1/chat/stream 30/min 1 minute
GET /v1/threads 100/min 1 minute
POST /v1/threads 20/min 1 minute

Content#

Endpoint Limit Window
GET /v1/content/* 300/min 1 minute
POST /v1/content/search 60/min 1 minute
GET /v1/content/recommendations 30/min 1 minute

Media#

Endpoint Limit Window
POST /v1/media/upload 10/min 1 minute
GET /v1/media/stream/* 100/min 1 minute
POST /v1/media/generate 5/min 1 minute

Notifications#

Endpoint Limit Window
POST /v1/notifications/send 100/hour 1 hour
GET /v1/notifications 60/min 1 minute

Rate Limiting Algorithm#

Sliding Window#

The API uses a sliding window algorithm for accurate rate limiting:

text
Window Size: 60 seconds (configurable)
Algorithm: Sorted Set with timestamp scoring

For each request:
1. Remove expired entries (timestamp < now - window)
2. Count remaining entries
3. If count >= limit: reject (429)
4. If count < limit: add current timestamp, allow request

Key Generation#

Rate limit keys are generated per-user or per-IP:

text
Authenticated: rate_limit:{user_id}:{endpoint}
Anonymous: rate_limit:{client_ip}:{endpoint}

Best Practices#

1. Respect Rate Limit Headers#

Check headers before making subsequent requests:

typescript
async function makeRequest(url: string): Promise<Response> {
  const response = await fetch(url);

  const remaining = parseInt(
    response.headers.get('RateLimit-Remaining') || '0'
  );
  const resetTime = parseInt(response.headers.get('RateLimit-Reset') || '0');

  if (remaining === 0) {
    const waitTime = resetTime - Math.floor(Date.now() / 1000);
    console.log(`Rate limited. Waiting ${waitTime} seconds.`);
    await sleep(waitTime * 1000);
  }

  return response;
}

2. Implement Exponential Backoff#

typescript
async function requestWithBackoff(
  url: string,
  maxRetries = 5
): Promise<Response> {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    const response = await fetch(url);

    if (response.status !== 429) {
      return response;
    }

    const retryAfter = parseInt(
      response.headers.get('Retry-After') || String(Math.pow(2, attempt))
    );

    await sleep(retryAfter * 1000);
  }

  throw new Error('Max retries exceeded');
}

3. Use Request Batching#

Combine multiple operations into single requests:

typescript
// Instead of multiple requests:
// GET /v1/content/1
// GET /v1/content/2
// GET /v1/content/3

// Use batch endpoint:
POST /v1/aggregate
{
  "strategy": "parallel",
  "requests": [
    { "id": "content_1", "path": "/v1/content/1" },
    { "id": "content_2", "path": "/v1/content/2" },
    { "id": "content_3", "path": "/v1/content/3" }
  ]
}

4. Cache Responses#

Reduce API calls by caching:

typescript
const cache = new Map<string, { data: any; expires: number }>();

async function getCached(url: string, ttl = 300000): Promise<any> {
  const cached = cache.get(url);
  if (cached && cached.expires > Date.now()) {
    return cached.data;
  }

  const response = await fetch(url);
  const data = await response.json();

  cache.set(url, { data, expires: Date.now() + ttl });
  return data;
}

Enterprise Features#

Custom Rate Limits#

Enterprise plans can configure custom limits:

http
POST /v1/admin/rate-limits
Authorization: Bearer <admin_token>

{
  "scope": "organization",
  "organization_id": "org_abc123",
  "limits": {
    "chat_completions_per_minute": 200,
    "media_uploads_per_hour": 100,
    "api_requests_per_day": 500000
  }
}

Rate Limit Monitoring#

Enterprise customers can monitor usage:

http
GET /v1/admin/rate-limits/usage
Authorization: Bearer <admin_token>

Response:
{
  "period": "2025-01-15",
  "usage": {
    "total_requests": 45000,
    "rate_limited_requests": 150,
    "endpoints": {
      "/v1/chat/completions": 15000,
      "/v1/content/search": 8000
    }
  },
  "limits": {
    "requests_per_day": 500000,
    "remaining": 455000
  }
}

IP Allowlisting#

Bypass rate limits for trusted IPs:

http
POST /v1/admin/allowlist
Authorization: Bearer <admin_token>

{
  "ip_addresses": ["192.168.1.0/24"],
  "rate_limit_multiplier": 10,
  "reason": "Internal service traffic"
}

Development & Testing#

Localhost Exemption#

Requests from localhost (127.0.0.1, ::1) are exempt from rate limiting in development environments.

Test Environment#

Test environments have relaxed limits:

Environment Rate Limit Multiplier
Production 1x (standard)
Staging 2x
Development 10x
Test Unlimited

Monitor-Only Mode#

Enable monitoring without blocking:

bash
SECURITY_MONITOR_ONLY=true

In this mode, rate limit violations are logged but requests are allowed through.

Configuration#

Environment Variables#

Variable Default Description
RATE_LIMIT_WINDOW_MS 60000 Rate limit window in ms
RATE_LIMIT_MAX_REQUESTS 12 Max requests per window
SECURITY_MIDDLEWARE_ENABLED true Enable rate limiting
SECURITY_MONITOR_ONLY false Log without blocking

Redis Configuration#

For distributed rate limiting:

typescript
{
  host: 'redis.example.com',
  port: 6379,
  password: 'secret',
  db: 0,
  lazyConnect: true,
  maxRetriesPerRequest: 3
}

If Redis is unavailable, the system falls back to in-memory rate limiting per instance.