This document provides comprehensive documentation for the Lilith Conversation Service Context Management API, including context window management, session stores, and conversation memory.
Overview#
The Context Management API provides:
- Context Window Management: Token counting, truncation strategies, and summarization
- Session Storage: In-memory and Redis-backed session persistence
- Conversation Memory: Short-term and long-term conversation context
- Horizontal Scaling: Pub/sub for cross-server session synchronization
Table of Contents#
Context Window Management#
Overview#
The context window management system automatically handles conversation history to fit within token limits, using configurable truncation strategies.
Configuration#
| Environment Variable | Default | Description |
|---|---|---|
CONTEXT_WINDOW_SIZE |
4000 | Maximum tokens in context |
CONTEXT_TRUNCATION_STRATEGY |
sliding_window |
Truncation strategy |
Truncation Strategies#
Sliding Window#
Keeps the most recent messages within the token limit while preserving all system messages.
text
Strategy: sliding_window
[system] | [old msg 1] | [old msg 2] | [msg 3] | [msg 4] | [msg 5]
↓ truncation
[system] | | [msg 3] | [msg 4] | [msg 5]
Behavior:
- System messages are always preserved
- Older non-system messages are removed first
- Maintains chronological order of remaining messages
Summary#
Generates a summary of older messages and keeps recent messages.
text
Strategy: summary
[system] | [old msg 1] | [old msg 2] | [msg 3] | [msg 4] | [msg 5]
↓ summarization
[system] | [summary of old] | [msg 3] | [msg 4] | [msg 5]
Behavior:
- 20% of token budget reserved for summary
- Older messages are summarized into a system message
- Recent messages preserved within remaining budget
- Summary includes topic overview
Context Window Response#
Chat responses include context window information:
json
{
"context_window_info": {
"total_tokens": 3500,
"messages_in_context": 8,
"window_size_limit": 4000,
"truncated": true,
"truncation_applied": true,
"messages_truncated": 3,
"truncation_strategy": "sliding_window",
"strategy_used": "sliding_window",
"summary": null,
"system_messages_preserved": true
}
}
Fields#
| Field | Type | Description |
|---|---|---|
total_tokens |
integer | Current context token count |
messages_in_context |
integer | Messages included in context |
window_size_limit |
integer | Maximum allowed tokens |
truncated |
boolean | Whether truncation was needed |
truncation_applied |
boolean | Whether truncation was applied |
messages_truncated |
integer | Number of messages removed |
truncation_strategy |
string | Configured strategy |
strategy_used |
string | Strategy actually used |
summary |
string | Summary text if summary strategy used |
system_messages_preserved |
boolean | Whether system messages were kept |
Session Storage#
Overview#
Session storage manages WebSocket session state for connected clients. Two implementations are available:
| Type | Use Case | Scaling |
|---|---|---|
memory |
Development, single-server | No |
redis |
Production, horizontal scaling | Yes |
Session Object#
json
{
"userId": "user_123",
"sessionId": "sess_abc456",
"serverId": "server-1",
"deviceId": "device_xyz",
"userAgent": "Mozilla/5.0...",
"ipAddress": "192.168.1.1",
"connectedAt": "2024-01-01T00:00:00.000Z",
"lastActivityAt": "2024-01-01T00:30:00.000Z",
"metadata": {
"conversationId": "conv_123",
"persona": "meditation_guide"
}
}
Session Query#
Filter sessions with query parameters:
json
{
"userId": "user_123",
"serverId": "server-1",
"connectedAfter": "2024-01-01T00:00:00.000Z",
"connectedBefore": "2024-01-02T00:00:00.000Z",
"activeWithin": 3600
}
| Field | Type | Description |
|---|---|---|
userId |
string | Filter by user ID |
serverId |
string | Filter by server ID |
connectedAfter |
Date | Sessions connected after this time |
connectedBefore |
Date | Sessions connected before this time |
activeWithin |
integer | Sessions active within N seconds |
Session Stats#
json
{
"totalSessions": 150,
"uniqueUsers": 100,
"sessionsByServer": {
"server-1": 75,
"server-2": 75
},
"oldestSession": "2024-01-01T00:00:00.000Z",
"newestSession": "2024-01-01T12:00:00.000Z"
}
Session Operations#
Create Session#
typescript
const session = await store.createSession({
userId: 'user_123',
sessionId: 'sess_abc456',
serverId: 'server-1',
deviceId: 'device_xyz',
userAgent: 'Mozilla/5.0...',
ipAddress: '192.168.1.1',
metadata: {
conversationId: 'conv_123',
},
});
Get Session#
typescript
const session = await store.getSession('sess_abc456');
Get User Sessions#
typescript
const sessions = await store.getUserSessions('user_123');
Get Server Sessions#
typescript
const sessions = await store.getServerSessions('server-1');
Update Activity#
typescript
await store.updateActivity('sess_abc456');
Update Metadata#
typescript
await store.updateMetadata('sess_abc456', {
currentConversation: 'conv_456',
});
Delete Session#
typescript
await store.deleteSession('sess_abc456');
Delete User Sessions#
typescript
const count = await store.deleteUserSessions('user_123');
Delete Server Sessions#
typescript
const count = await store.deleteServerSessions('server-1');
List Sessions#
typescript
const sessions = await store.listSessions({
activeWithin: 3600,
serverId: 'server-1',
});
Count Sessions#
typescript
const count = await store.countSessions({
userId: 'user_123',
});
Get Stats#
typescript
const stats = await store.getStats();
Session Events#
Event Types#
| Type | Description |
|---|---|
session_created |
New session connected |
session_closed |
Session disconnected |
session_activity |
Activity on session |
broadcast_message |
Message broadcast to sessions |
Subscribe to Events#
typescript
await store.subscribe('session_events', (event) => {
switch (event.type) {
case 'session_created':
console.log(`User ${event.userId} connected`);
break;
case 'session_closed':
console.log(`Session ${event.sessionId} closed`);
break;
}
});
Publish Events#
typescript
await store.publish('session_events', {
type: 'broadcast_message',
timestamp: new Date(),
data: { message: 'System update in 5 minutes' },
});
Unsubscribe#
typescript
await store.unsubscribe('session_events');
Configuration#
Environment Variables#
| Variable | Default | Description |
|---|---|---|
SESSION_STORE_TYPE |
memory |
Store type: memory or redis |
REDIS_HOST |
- | Redis server hostname |
REDIS_PORT |
6379 |
Redis server port |
REDIS_PASSWORD |
- | Redis password (optional) |
REDIS_DB |
0 |
Redis database number |
REDIS_KEY_PREFIX |
lilith:session: |
Key prefix |
REDIS_TLS |
false |
Enable TLS |
SESSION_TTL |
86400 |
Session TTL in seconds (24h) |
CONTEXT_WINDOW_SIZE |
4000 |
Max tokens in context |
CONTEXT_TRUNCATION_STRATEGY |
sliding_window |
Truncation strategy |
Programmatic Configuration#
typescript
import { createSessionStore } from './session/index.js';
// In-memory store
const memoryStore = createSessionStore({
type: 'memory',
sessionTTL: 86400,
cleanupInterval: 300,
});
// Redis store
const redisStore = createSessionStore({
type: 'redis',
redis: {
host: 'localhost',
port: 6379,
password: 'secret',
db: 0,
keyPrefix: 'lilith:session:',
tls: false,
},
sessionTTL: 86400,
});
await redisStore.connect();
Create from Environment#
typescript
import { createSessionStoreFromEnv } from './session/index.js';
const store = createSessionStoreFromEnv();
await store.connect();
SDK Integration#
TypeScript/JavaScript#
typescript
import { LilithClient } from '@lilith/sdk';
const client = new LilithClient({
apiKey: process.env.LILITH_API_KEY,
});
// Chat with context window info
const response = await client.conversation.chat({
text: 'Continue our discussion',
conversationId: 'conv_123',
persona: 'meditation_guide',
});
// Access context window info
const { context_window_info } = response;
if (context_window_info.truncated) {
console.log(`${context_window_info.messages_truncated} messages truncated`);
console.log(`Strategy: ${context_window_info.strategy_used}`);
}
// Get conversation metrics including context
const metrics = await client.conversation.getMetrics('conv_123');
console.log(
`Token usage: ${metrics.total.tokens_in + metrics.total.tokens_out}`
);
// Session management (admin)
const sessions = await client.conversation.admin.listSessions({
activeWithin: 3600,
});
const stats = await client.conversation.admin.getSessionStats();
console.log(`Active users: ${stats.uniqueUsers}`);
Related Documentation#
- Thread API - Thread and message management
- Streaming API - Real-time streaming
- Chat API - Chat completion API
- Authentication - API authentication
Questions?#
- Technical Support: api-support@lilith.app
- Documentation: GitHub Issues