This document provides comprehensive documentation for the Lilith AI Prompt Management system, including versioned prompts, feature flags, and A/B testing capabilities.
Overview#
The Prompt Management API enables enterprise-grade management of AI system prompts with:
- Versioned Prompts: Semantic versioning with automatic incrementing
- Feature Flags: Gradual rollout and A/B testing capabilities
- Template Variables: Dynamic prompt customization with
{{variable}}syntax - Per-Persona Configuration: Each AI persona maintains independent prompt versions
Table of Contents#
Feature Flags#
Feature flags control access to prompt versions and enable gradual rollout of new AI capabilities.
Default Feature Flags#
| Flag Name | Default Enabled | Rollout | Description |
|---|---|---|---|
enhanced_personas |
true | 100% | Enhanced persona system with advanced prompts |
prompt_versioning |
true | 100% | Versioned system prompts with A/B testing |
advanced_reasoning |
false | 10% | Advanced reasoning capabilities for complex queries |
multi_modal_prompts |
false | 5% | Multi-modal prompt support with images |
custom_personas |
true | 80% | User-defined custom persona creation |
prompt_templates |
true | 90% | Templated prompt system with variables |
persona_memory |
false | 20% | Long-term persona memory and preferences |
safety_override |
true | 100% | Enhanced safety checks for persona boundaries |
List Feature Flags#
Retrieve all feature flags with optional user-specific enablement status.
Endpoint: GET /v1/ai/feature-flags
Query Parameters:
| Parameter | Type | Description |
|---|---|---|
user_id |
string | Optional user ID for personalized rollout calculation |
Request:
curl -X GET "https://api.lilith.app/v1/ai/feature-flags?user_id=user_12345" \
-H "Authorization: Bearer <access_token>"
Response:
{
"feature_flags": [
{
"name": "enhanced_personas",
"enabled": true,
"rollout": 100,
"description": "Enhanced persona system with advanced prompts",
"user_enabled": true
},
{
"name": "prompt_versioning",
"enabled": true,
"rollout": 100,
"description": "Versioned system prompts with A/B testing",
"user_enabled": true
},
{
"name": "advanced_reasoning",
"enabled": false,
"rollout": 10,
"description": "Advanced reasoning capabilities for complex queries",
"user_enabled": false
}
],
"user_id": "user_12345"
}
Update Feature Flag#
Modify feature flag configuration including enabled state, rollout percentage, and description.
Endpoint: PUT /v1/ai/feature-flags/{flagName}
Path Parameters:
| Parameter | Type | Description |
|---|---|---|
flagName |
string | Name of the feature flag to update |
Request Body:
| Field | Type | Description |
|---|---|---|
enabled |
boolean | Whether the feature is enabled |
rollout |
number | Percentage of users who have access (0-100) |
description |
string | Human-readable description |
Request:
curl -X PUT "https://api.lilith.app/v1/ai/feature-flags/advanced_reasoning" \
-H "Authorization: Bearer <access_token>" \
-H "Content-Type: application/json" \
-d '{
"enabled": true,
"rollout": 25,
"description": "Updated advanced reasoning feature"
}'
Response:
{
"flag_name": "advanced_reasoning",
"new_config": {
"enabled": true,
"rollout": 25,
"description": "Updated advanced reasoning feature"
},
"updated": true
}
Rollout Logic#
User access to feature flags is determined by a consistent hashing algorithm:
- A hash is computed from
userId + flagName - The hash is converted to a percentile (0-99)
- Access is granted if the user's percentile is below the rollout percentage
This ensures:
- Consistency: Same user always gets the same result for a given flag
- Gradual Rollout: Increasing the rollout percentage adds new users without removing existing ones
- A/B Testing: Users can be consistently assigned to feature groups
Versioned Prompts#
Create Versioned Prompt#
Create a new versioned system prompt for a persona. Version numbers can be specified or auto-incremented.
Endpoint: POST /v1/ai/personas/{personaKey}/prompts
Path Parameters:
| Parameter | Type | Description |
|---|---|---|
personaKey |
string | Unique identifier for the persona (e.g., meditation_teacher) |
Request Body:
| Field | Type | Required | Description |
|---|---|---|---|
system_prompt |
string | Yes | The system prompt content |
examples |
string[] | No | Example prompts for template variable extraction |
version |
string | No | Semantic version (auto-generated if not provided) |
feature_flags |
string[] | No | Feature flags required for this prompt version |
is_default |
boolean | No | Mark as the default version for this persona |
description |
string | No | Human-readable description of this version |
Request:
curl -X POST "https://api.lilith.app/v1/ai/personas/meditation_teacher/prompts" \
-H "Authorization: Bearer <access_token>" \
-H "Content-Type: application/json" \
-d '{
"system_prompt": "You are a wise meditation teacher who guides students with compassion and clarity.",
"examples": [
"Guide the user through a {{duration}} minute breathing meditation",
"Explain {{concept}} in simple terms suitable for beginners"
],
"version": "2.1.0",
"feature_flags": ["enhanced_personas", "prompt_templates"],
"is_default": true,
"description": "Enhanced meditation teacher prompt with templating support"
}'
Response:
{
"prompt_id": "prompt_1704067200000_abc123",
"persona_key": "meditation_teacher",
"type": "system",
"content": "You are a wise meditation teacher who guides students with compassion and clarity.",
"system_prompt": "You are a wise meditation teacher who guides students with compassion and clarity.",
"examples": [
"Guide the user through a {{duration}} minute breathing meditation",
"Explain {{concept}} in simple terms suitable for beginners"
],
"version": "2.1.0",
"feature_flags": ["enhanced_personas", "prompt_templates"],
"is_default": true,
"variables": ["duration", "concept"],
"created_at": "2024-01-01T00:00:00.000Z",
"updated_at": "2024-01-01T00:00:00.000Z"
}
Auto-Generated Versions#
When no version is specified, the system automatically increments the patch version:
First prompt: 1.0.0
Second prompt: 1.0.1
Third prompt: 1.0.2
...
To create a major or minor version, specify it explicitly:
{
"system_prompt": "Major update to persona",
"version": "2.0.0"
}
List Prompt Versions#
Retrieve all prompt versions for a persona, sorted by version (newest first).
Endpoint: GET /v1/ai/personas/{personaKey}/prompts
Query Parameters:
| Parameter | Type | Description |
|---|---|---|
version |
string | Optional specific version to retrieve |
type |
string | Optional filter by prompt type |
Request:
curl -X GET "https://api.lilith.app/v1/ai/personas/meditation_teacher/prompts" \
-H "Authorization: Bearer <access_token>"
Response:
{
"persona_key": "meditation_teacher",
"prompts": [
{
"prompt_id": "prompt_1704153600000_xyz789",
"version": "2.0.0",
"system_prompt": "Major update to persona",
"is_default": true,
"created_at": "2024-01-02T00:00:00.000Z"
},
{
"prompt_id": "prompt_1704067200000_abc123",
"version": "1.1.0",
"system_prompt": "Minor update",
"feature_flags": ["enhanced_personas"],
"created_at": "2024-01-01T12:00:00.000Z"
},
{
"prompt_id": "prompt_1704000000000_def456",
"version": "1.0.0",
"system_prompt": "Original prompt",
"created_at": "2024-01-01T00:00:00.000Z"
}
],
"total_versions": 3
}
Get Specific Version#
Retrieve a specific prompt version by passing the version query parameter:
curl -X GET "https://api.lilith.app/v1/ai/personas/meditation_teacher/prompts?version=1.1.0" \
-H "Authorization: Bearer <access_token>"
Active Prompt Resolution#
The active prompt endpoint resolves which prompt version should be used for a given persona and user, based on feature flags and default settings.
Resolution Flow#
┌─────────────────────────────────────────────────────────────────┐
│ Active Prompt Resolution │
├─────────────────────────────────────────────────────────────────┤
│ │
│ 1. Check if enhanced_personas flag is enabled │
│ └── If disabled → Return fallback prompt │
│ │
│ 2. Find prompts with feature flags user has access to │
│ └── If found → Return matching prompt │
│ │
│ 3. Find default prompt without feature flags │
│ └── If found → Return default prompt │
│ │
│ 4. Find latest version without feature flags │
│ └── If found → Return latest prompt │
│ │
│ 5. No accessible prompt found │
│ └── Return fallback_no_access prompt │
│ │
└─────────────────────────────────────────────────────────────────┘
Get Active Prompt#
Retrieve the active prompt for a persona based on feature flag access.
Endpoint: GET /v1/ai/personas/{personaKey}/active-prompt
Query Parameters:
| Parameter | Type | Description |
|---|---|---|
user_id |
string | Optional user ID for feature flag resolution |
Request:
curl -X GET "https://api.lilith.app/v1/ai/personas/meditation_teacher/active-prompt?user_id=user_12345" \
-H "Authorization: Bearer <access_token>"
Response (from versioned storage):
{
"prompt_id": "prompt_1704067200000_abc123",
"persona_key": "meditation_teacher",
"system_prompt": "You are a wise meditation teacher who guides students with compassion and clarity.",
"version": "2.1.0",
"feature_flags": ["enhanced_personas", "prompt_templates"],
"is_default": true,
"source": "versioned_storage",
"user_feature_access": {
"enhanced_personas": true,
"prompt_templates": true
}
}
Response (fallback when enhanced_personas disabled):
{
"system_prompt": "helpful meditation teacher assistant",
"version": "fallback",
"source": "fallback",
"persona_key": "meditation_teacher"
}
Response (fallback when user lacks feature access):
{
"system_prompt": "helpful meditation teacher assistant",
"version": "fallback",
"source": "fallback_no_access",
"persona_key": "meditation_teacher"
}
Source Types#
| Source | Description |
|---|---|
versioned_storage |
Prompt retrieved from versioned prompt storage |
fallback |
Feature flag enhanced_personas is disabled |
fallback_no_access |
User lacks access to all feature-gated prompts |
Template Variables#
Template variables allow dynamic customization of prompts using {{variable}}
syntax.
Variable Extraction#
When creating a prompt with examples, variables are automatically extracted:
Input:
{
"system_prompt": "You are a meditation teacher.",
"examples": [
"Guide the user through a {{duration}} minute breathing meditation",
"Explain {{concept}} in {{complexity}} terms"
]
}
Extracted Variables:
{
"variables": ["duration", "concept", "complexity"]
}
Using Variables#
Variables can be substituted at runtime when invoking the prompt:
curl -X POST "https://api.lilith.app/v1/ai/chat" \
-H "Authorization: Bearer <access_token>" \
-H "Content-Type: application/json" \
-d '{
"persona": "meditation_teacher",
"message": "Guide me through breathing",
"template_variables": {
"duration": "10",
"concept": "mindfulness",
"complexity": "beginner"
}
}'
Error Handling#
Error Codes#
| Code | HTTP Status | Description |
|---|---|---|
ai.feature_disabled |
403 | Prompt versioning feature is not enabled |
ai.missing_system_prompt |
400 | Required system_prompt field is missing |
ai.invalid_prompt_type |
400 | Invalid prompt type (must be: system, user, template) |
ai.empty_prompt_content |
400 | Prompt content cannot be empty |
ai.invalid_persona_key |
400 | Persona key cannot be empty |
ai.feature_flag_not_found |
404 | Feature flag does not exist |
ai.prompt_not_found |
404 | Prompt version not found for persona |
ai.no_active_prompt |
404 | No active prompt found for persona |
ai.add_prompt_failed |
500 | Failed to add persona prompt |
ai.feature_flags_failed |
500 | Failed to retrieve feature flags |
Example Error Response#
{
"error": "system_prompt is required",
"code": "ai.missing_system_prompt"
}
Best Practices#
- Always specify feature flags for experimental prompts to control rollout
- Use semantic versioning to track prompt evolution (major.minor.patch)
- Mark one version as default to ensure consistent fallback behavior
- Test with specific user IDs to verify feature flag rollout
- Monitor active prompt sources to detect fallback usage
SDK Integration#
TypeScript/JavaScript#
import { LilithClient } from '@lilith/sdk';
const client = new LilithClient({
apiKey: process.env.LILITH_API_KEY,
});
// List feature flags for a user
const flags = await client.ai.getFeatureFlags({ userId: 'user_12345' });
console.log(flags.feature_flags);
// Create a new prompt version
const prompt = await client.ai.createPrompt('meditation_teacher', {
system_prompt: 'You are a wise meditation teacher...',
examples: ['Guide the user through a {{duration}} minute meditation'],
version: '2.0.0',
feature_flags: ['enhanced_personas'],
is_default: true,
});
// Get active prompt for a user
const activePrompt = await client.ai.getActivePrompt('meditation_teacher', {
userId: 'user_12345',
});
console.log(activePrompt.system_prompt);
console.log(activePrompt.source); // 'versioned_storage' | 'fallback' | 'fallback_no_access'
// Update feature flag rollout
await client.ai.updateFeatureFlag('advanced_reasoning', {
enabled: true,
rollout: 50,
});
Related Documentation#
- Authentication - API authentication
- Error Codes - Complete error reference
- Rate Limits - Rate limiting policies
- Chat API - Chat integration with prompts
Questions?#
- Technical Support: api-support@lilith.app
- Documentation: GitHub Issues