Eve library

Context Management API

Guide

Revision 1

On this page

This document provides comprehensive documentation for the Lilith Conversation Service Context Management API, including context window management, session stores, and conversation memory.

Overview#

The Context Management API provides:

  • Context Window Management: Token counting, truncation strategies, and summarization
  • Session Storage: In-memory and Redis-backed session persistence
  • Conversation Memory: Short-term and long-term conversation context
  • Horizontal Scaling: Pub/sub for cross-server session synchronization

Table of Contents#

  1. Context Window Management
  2. Session Storage
  3. Session Operations
  4. Configuration
  5. SDK Integration

Context Window Management#

Overview#

The context window management system automatically handles conversation history to fit within token limits, using configurable truncation strategies.

Configuration#

Environment Variable Default Description
CONTEXT_WINDOW_SIZE 4000 Maximum tokens in context
CONTEXT_TRUNCATION_STRATEGY sliding_window Truncation strategy

Truncation Strategies#

Sliding Window#

Keeps the most recent messages within the token limit while preserving all system messages.

text
Strategy: sliding_window

[system] | [old msg 1] | [old msg 2] | [msg 3] | [msg 4] | [msg 5]
                 ↓ truncation
[system] |                           | [msg 3] | [msg 4] | [msg 5]

Behavior:

  • System messages are always preserved
  • Older non-system messages are removed first
  • Maintains chronological order of remaining messages

Summary#

Generates a summary of older messages and keeps recent messages.

text
Strategy: summary

[system] | [old msg 1] | [old msg 2] | [msg 3] | [msg 4] | [msg 5]
                 ↓ summarization
[system] | [summary of old] | [msg 3] | [msg 4] | [msg 5]

Behavior:

  • 20% of token budget reserved for summary
  • Older messages are summarized into a system message
  • Recent messages preserved within remaining budget
  • Summary includes topic overview

Context Window Response#

Chat responses include context window information:

json
{
  "context_window_info": {
    "total_tokens": 3500,
    "messages_in_context": 8,
    "window_size_limit": 4000,
    "truncated": true,
    "truncation_applied": true,
    "messages_truncated": 3,
    "truncation_strategy": "sliding_window",
    "strategy_used": "sliding_window",
    "summary": null,
    "system_messages_preserved": true
  }
}

Fields#

Field Type Description
total_tokens integer Current context token count
messages_in_context integer Messages included in context
window_size_limit integer Maximum allowed tokens
truncated boolean Whether truncation was needed
truncation_applied boolean Whether truncation was applied
messages_truncated integer Number of messages removed
truncation_strategy string Configured strategy
strategy_used string Strategy actually used
summary string Summary text if summary strategy used
system_messages_preserved boolean Whether system messages were kept

Session Storage#

Overview#

Session storage manages WebSocket session state for connected clients. Two implementations are available:

Type Use Case Scaling
memory Development, single-server No
redis Production, horizontal scaling Yes

Session Object#

json
{
  "userId": "user_123",
  "sessionId": "sess_abc456",
  "serverId": "server-1",
  "deviceId": "device_xyz",
  "userAgent": "Mozilla/5.0...",
  "ipAddress": "192.168.1.1",
  "connectedAt": "2024-01-01T00:00:00.000Z",
  "lastActivityAt": "2024-01-01T00:30:00.000Z",
  "metadata": {
    "conversationId": "conv_123",
    "persona": "meditation_guide"
  }
}

Session Query#

Filter sessions with query parameters:

json
{
  "userId": "user_123",
  "serverId": "server-1",
  "connectedAfter": "2024-01-01T00:00:00.000Z",
  "connectedBefore": "2024-01-02T00:00:00.000Z",
  "activeWithin": 3600
}
Field Type Description
userId string Filter by user ID
serverId string Filter by server ID
connectedAfter Date Sessions connected after this time
connectedBefore Date Sessions connected before this time
activeWithin integer Sessions active within N seconds

Session Stats#

json
{
  "totalSessions": 150,
  "uniqueUsers": 100,
  "sessionsByServer": {
    "server-1": 75,
    "server-2": 75
  },
  "oldestSession": "2024-01-01T00:00:00.000Z",
  "newestSession": "2024-01-01T12:00:00.000Z"
}

Session Operations#

Create Session#

typescript
const session = await store.createSession({
  userId: 'user_123',
  sessionId: 'sess_abc456',
  serverId: 'server-1',
  deviceId: 'device_xyz',
  userAgent: 'Mozilla/5.0...',
  ipAddress: '192.168.1.1',
  metadata: {
    conversationId: 'conv_123',
  },
});

Get Session#

typescript
const session = await store.getSession('sess_abc456');

Get User Sessions#

typescript
const sessions = await store.getUserSessions('user_123');

Get Server Sessions#

typescript
const sessions = await store.getServerSessions('server-1');

Update Activity#

typescript
await store.updateActivity('sess_abc456');

Update Metadata#

typescript
await store.updateMetadata('sess_abc456', {
  currentConversation: 'conv_456',
});

Delete Session#

typescript
await store.deleteSession('sess_abc456');

Delete User Sessions#

typescript
const count = await store.deleteUserSessions('user_123');

Delete Server Sessions#

typescript
const count = await store.deleteServerSessions('server-1');

List Sessions#

typescript
const sessions = await store.listSessions({
  activeWithin: 3600,
  serverId: 'server-1',
});

Count Sessions#

typescript
const count = await store.countSessions({
  userId: 'user_123',
});

Get Stats#

typescript
const stats = await store.getStats();

Session Events#

Event Types#

Type Description
session_created New session connected
session_closed Session disconnected
session_activity Activity on session
broadcast_message Message broadcast to sessions

Subscribe to Events#

typescript
await store.subscribe('session_events', (event) => {
  switch (event.type) {
    case 'session_created':
      console.log(`User ${event.userId} connected`);
      break;
    case 'session_closed':
      console.log(`Session ${event.sessionId} closed`);
      break;
  }
});

Publish Events#

typescript
await store.publish('session_events', {
  type: 'broadcast_message',
  timestamp: new Date(),
  data: { message: 'System update in 5 minutes' },
});

Unsubscribe#

typescript
await store.unsubscribe('session_events');

Configuration#

Environment Variables#

Variable Default Description
SESSION_STORE_TYPE memory Store type: memory or redis
REDIS_HOST - Redis server hostname
REDIS_PORT 6379 Redis server port
REDIS_PASSWORD - Redis password (optional)
REDIS_DB 0 Redis database number
REDIS_KEY_PREFIX lilith:session: Key prefix
REDIS_TLS false Enable TLS
SESSION_TTL 86400 Session TTL in seconds (24h)
CONTEXT_WINDOW_SIZE 4000 Max tokens in context
CONTEXT_TRUNCATION_STRATEGY sliding_window Truncation strategy

Programmatic Configuration#

typescript
import { createSessionStore } from './session/index.js';

// In-memory store
const memoryStore = createSessionStore({
  type: 'memory',
  sessionTTL: 86400,
  cleanupInterval: 300,
});

// Redis store
const redisStore = createSessionStore({
  type: 'redis',
  redis: {
    host: 'localhost',
    port: 6379,
    password: 'secret',
    db: 0,
    keyPrefix: 'lilith:session:',
    tls: false,
  },
  sessionTTL: 86400,
});

await redisStore.connect();

Create from Environment#

typescript
import { createSessionStoreFromEnv } from './session/index.js';

const store = createSessionStoreFromEnv();
await store.connect();

SDK Integration#

TypeScript/JavaScript#

typescript
import { LilithClient } from '@lilith/sdk';

const client = new LilithClient({
  apiKey: process.env.LILITH_API_KEY,
});

// Chat with context window info
const response = await client.conversation.chat({
  text: 'Continue our discussion',
  conversationId: 'conv_123',
  persona: 'meditation_guide',
});

// Access context window info
const { context_window_info } = response;
if (context_window_info.truncated) {
  console.log(`${context_window_info.messages_truncated} messages truncated`);
  console.log(`Strategy: ${context_window_info.strategy_used}`);
}

// Get conversation metrics including context
const metrics = await client.conversation.getMetrics('conv_123');
console.log(
  `Token usage: ${metrics.total.tokens_in + metrics.total.tokens_out}`
);

// Session management (admin)
const sessions = await client.conversation.admin.listSessions({
  activeWithin: 3600,
});

const stats = await client.conversation.admin.getSessionStats();
console.log(`Active users: ${stats.uniqueUsers}`);


Questions?#

  • Technical Support: api-support@lilith.app
  • Documentation: GitHub Issues