Eve library

Prompt Management API

Guide

Revision 1

On this page

This document provides comprehensive documentation for the Lilith AI Prompt Management system, including versioned prompts, feature flags, and A/B testing capabilities.

Overview#

The Prompt Management API enables enterprise-grade management of AI system prompts with:

  • Versioned Prompts: Semantic versioning with automatic incrementing
  • Feature Flags: Gradual rollout and A/B testing capabilities
  • Template Variables: Dynamic prompt customization with {{variable}} syntax
  • Per-Persona Configuration: Each AI persona maintains independent prompt versions

Table of Contents#

  1. Feature Flags
  2. Versioned Prompts
  3. Active Prompt Resolution
  4. Template Variables
  5. Error Handling

Feature Flags#

Feature flags control access to prompt versions and enable gradual rollout of new AI capabilities.

Default Feature Flags#

Flag Name Default Enabled Rollout Description
enhanced_personas true 100% Enhanced persona system with advanced prompts
prompt_versioning true 100% Versioned system prompts with A/B testing
advanced_reasoning false 10% Advanced reasoning capabilities for complex queries
multi_modal_prompts false 5% Multi-modal prompt support with images
custom_personas true 80% User-defined custom persona creation
prompt_templates true 90% Templated prompt system with variables
persona_memory false 20% Long-term persona memory and preferences
safety_override true 100% Enhanced safety checks for persona boundaries

List Feature Flags#

Retrieve all feature flags with optional user-specific enablement status.

Endpoint: GET /v1/ai/feature-flags

Query Parameters:

Parameter Type Description
user_id string Optional user ID for personalized rollout calculation

Request:

bash
curl -X GET "https://api.lilith.app/v1/ai/feature-flags?user_id=user_12345" \
  -H "Authorization: Bearer <access_token>"

Response:

json
{
  "feature_flags": [
    {
      "name": "enhanced_personas",
      "enabled": true,
      "rollout": 100,
      "description": "Enhanced persona system with advanced prompts",
      "user_enabled": true
    },
    {
      "name": "prompt_versioning",
      "enabled": true,
      "rollout": 100,
      "description": "Versioned system prompts with A/B testing",
      "user_enabled": true
    },
    {
      "name": "advanced_reasoning",
      "enabled": false,
      "rollout": 10,
      "description": "Advanced reasoning capabilities for complex queries",
      "user_enabled": false
    }
  ],
  "user_id": "user_12345"
}

Update Feature Flag#

Modify feature flag configuration including enabled state, rollout percentage, and description.

Endpoint: PUT /v1/ai/feature-flags/{flagName}

Path Parameters:

Parameter Type Description
flagName string Name of the feature flag to update

Request Body:

Field Type Description
enabled boolean Whether the feature is enabled
rollout number Percentage of users who have access (0-100)
description string Human-readable description

Request:

bash
curl -X PUT "https://api.lilith.app/v1/ai/feature-flags/advanced_reasoning" \
  -H "Authorization: Bearer <access_token>" \
  -H "Content-Type: application/json" \
  -d '{
    "enabled": true,
    "rollout": 25,
    "description": "Updated advanced reasoning feature"
  }'

Response:

json
{
  "flag_name": "advanced_reasoning",
  "new_config": {
    "enabled": true,
    "rollout": 25,
    "description": "Updated advanced reasoning feature"
  },
  "updated": true
}

Rollout Logic#

User access to feature flags is determined by a consistent hashing algorithm:

  1. A hash is computed from userId + flagName
  2. The hash is converted to a percentile (0-99)
  3. Access is granted if the user's percentile is below the rollout percentage

This ensures:

  • Consistency: Same user always gets the same result for a given flag
  • Gradual Rollout: Increasing the rollout percentage adds new users without removing existing ones
  • A/B Testing: Users can be consistently assigned to feature groups

Versioned Prompts#

Create Versioned Prompt#

Create a new versioned system prompt for a persona. Version numbers can be specified or auto-incremented.

Endpoint: POST /v1/ai/personas/{personaKey}/prompts

Path Parameters:

Parameter Type Description
personaKey string Unique identifier for the persona (e.g., meditation_teacher)

Request Body:

Field Type Required Description
system_prompt string Yes The system prompt content
examples string[] No Example prompts for template variable extraction
version string No Semantic version (auto-generated if not provided)
feature_flags string[] No Feature flags required for this prompt version
is_default boolean No Mark as the default version for this persona
description string No Human-readable description of this version

Request:

bash
curl -X POST "https://api.lilith.app/v1/ai/personas/meditation_teacher/prompts" \
  -H "Authorization: Bearer <access_token>" \
  -H "Content-Type: application/json" \
  -d '{
    "system_prompt": "You are a wise meditation teacher who guides students with compassion and clarity.",
    "examples": [
      "Guide the user through a {{duration}} minute breathing meditation",
      "Explain {{concept}} in simple terms suitable for beginners"
    ],
    "version": "2.1.0",
    "feature_flags": ["enhanced_personas", "prompt_templates"],
    "is_default": true,
    "description": "Enhanced meditation teacher prompt with templating support"
  }'

Response:

json
{
  "prompt_id": "prompt_1704067200000_abc123",
  "persona_key": "meditation_teacher",
  "type": "system",
  "content": "You are a wise meditation teacher who guides students with compassion and clarity.",
  "system_prompt": "You are a wise meditation teacher who guides students with compassion and clarity.",
  "examples": [
    "Guide the user through a {{duration}} minute breathing meditation",
    "Explain {{concept}} in simple terms suitable for beginners"
  ],
  "version": "2.1.0",
  "feature_flags": ["enhanced_personas", "prompt_templates"],
  "is_default": true,
  "variables": ["duration", "concept"],
  "created_at": "2024-01-01T00:00:00.000Z",
  "updated_at": "2024-01-01T00:00:00.000Z"
}

Auto-Generated Versions#

When no version is specified, the system automatically increments the patch version:

text
First prompt:  1.0.0
Second prompt: 1.0.1
Third prompt:  1.0.2
...

To create a major or minor version, specify it explicitly:

json
{
  "system_prompt": "Major update to persona",
  "version": "2.0.0"
}

List Prompt Versions#

Retrieve all prompt versions for a persona, sorted by version (newest first).

Endpoint: GET /v1/ai/personas/{personaKey}/prompts

Query Parameters:

Parameter Type Description
version string Optional specific version to retrieve
type string Optional filter by prompt type

Request:

bash
curl -X GET "https://api.lilith.app/v1/ai/personas/meditation_teacher/prompts" \
  -H "Authorization: Bearer <access_token>"

Response:

json
{
  "persona_key": "meditation_teacher",
  "prompts": [
    {
      "prompt_id": "prompt_1704153600000_xyz789",
      "version": "2.0.0",
      "system_prompt": "Major update to persona",
      "is_default": true,
      "created_at": "2024-01-02T00:00:00.000Z"
    },
    {
      "prompt_id": "prompt_1704067200000_abc123",
      "version": "1.1.0",
      "system_prompt": "Minor update",
      "feature_flags": ["enhanced_personas"],
      "created_at": "2024-01-01T12:00:00.000Z"
    },
    {
      "prompt_id": "prompt_1704000000000_def456",
      "version": "1.0.0",
      "system_prompt": "Original prompt",
      "created_at": "2024-01-01T00:00:00.000Z"
    }
  ],
  "total_versions": 3
}

Get Specific Version#

Retrieve a specific prompt version by passing the version query parameter:

bash
curl -X GET "https://api.lilith.app/v1/ai/personas/meditation_teacher/prompts?version=1.1.0" \
  -H "Authorization: Bearer <access_token>"

Active Prompt Resolution#

The active prompt endpoint resolves which prompt version should be used for a given persona and user, based on feature flags and default settings.

Resolution Flow#

text
┌─────────────────────────────────────────────────────────────────┐
│                    Active Prompt Resolution                      │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  1. Check if enhanced_personas flag is enabled                  │
│     └── If disabled → Return fallback prompt                    │
│                                                                  │
│  2. Find prompts with feature flags user has access to          │
│     └── If found → Return matching prompt                       │
│                                                                  │
│  3. Find default prompt without feature flags                   │
│     └── If found → Return default prompt                        │
│                                                                  │
│  4. Find latest version without feature flags                   │
│     └── If found → Return latest prompt                         │
│                                                                  │
│  5. No accessible prompt found                                  │
│     └── Return fallback_no_access prompt                        │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

Get Active Prompt#

Retrieve the active prompt for a persona based on feature flag access.

Endpoint: GET /v1/ai/personas/{personaKey}/active-prompt

Query Parameters:

Parameter Type Description
user_id string Optional user ID for feature flag resolution

Request:

bash
curl -X GET "https://api.lilith.app/v1/ai/personas/meditation_teacher/active-prompt?user_id=user_12345" \
  -H "Authorization: Bearer <access_token>"

Response (from versioned storage):

json
{
  "prompt_id": "prompt_1704067200000_abc123",
  "persona_key": "meditation_teacher",
  "system_prompt": "You are a wise meditation teacher who guides students with compassion and clarity.",
  "version": "2.1.0",
  "feature_flags": ["enhanced_personas", "prompt_templates"],
  "is_default": true,
  "source": "versioned_storage",
  "user_feature_access": {
    "enhanced_personas": true,
    "prompt_templates": true
  }
}

Response (fallback when enhanced_personas disabled):

json
{
  "system_prompt": "helpful meditation teacher assistant",
  "version": "fallback",
  "source": "fallback",
  "persona_key": "meditation_teacher"
}

Response (fallback when user lacks feature access):

json
{
  "system_prompt": "helpful meditation teacher assistant",
  "version": "fallback",
  "source": "fallback_no_access",
  "persona_key": "meditation_teacher"
}

Source Types#

Source Description
versioned_storage Prompt retrieved from versioned prompt storage
fallback Feature flag enhanced_personas is disabled
fallback_no_access User lacks access to all feature-gated prompts

Template Variables#

Template variables allow dynamic customization of prompts using {{variable}} syntax.

Variable Extraction#

When creating a prompt with examples, variables are automatically extracted:

Input:

json
{
  "system_prompt": "You are a meditation teacher.",
  "examples": [
    "Guide the user through a {{duration}} minute breathing meditation",
    "Explain {{concept}} in {{complexity}} terms"
  ]
}

Extracted Variables:

json
{
  "variables": ["duration", "concept", "complexity"]
}

Using Variables#

Variables can be substituted at runtime when invoking the prompt:

bash
curl -X POST "https://api.lilith.app/v1/ai/chat" \
  -H "Authorization: Bearer <access_token>" \
  -H "Content-Type: application/json" \
  -d '{
    "persona": "meditation_teacher",
    "message": "Guide me through breathing",
    "template_variables": {
      "duration": "10",
      "concept": "mindfulness",
      "complexity": "beginner"
    }
  }'

Error Handling#

Error Codes#

Code HTTP Status Description
ai.feature_disabled 403 Prompt versioning feature is not enabled
ai.missing_system_prompt 400 Required system_prompt field is missing
ai.invalid_prompt_type 400 Invalid prompt type (must be: system, user, template)
ai.empty_prompt_content 400 Prompt content cannot be empty
ai.invalid_persona_key 400 Persona key cannot be empty
ai.feature_flag_not_found 404 Feature flag does not exist
ai.prompt_not_found 404 Prompt version not found for persona
ai.no_active_prompt 404 No active prompt found for persona
ai.add_prompt_failed 500 Failed to add persona prompt
ai.feature_flags_failed 500 Failed to retrieve feature flags

Example Error Response#

json
{
  "error": "system_prompt is required",
  "code": "ai.missing_system_prompt"
}

Best Practices#

  1. Always specify feature flags for experimental prompts to control rollout
  2. Use semantic versioning to track prompt evolution (major.minor.patch)
  3. Mark one version as default to ensure consistent fallback behavior
  4. Test with specific user IDs to verify feature flag rollout
  5. Monitor active prompt sources to detect fallback usage

SDK Integration#

TypeScript/JavaScript#

typescript
import { LilithClient } from '@lilith/sdk';

const client = new LilithClient({
  apiKey: process.env.LILITH_API_KEY,
});

// List feature flags for a user
const flags = await client.ai.getFeatureFlags({ userId: 'user_12345' });
console.log(flags.feature_flags);

// Create a new prompt version
const prompt = await client.ai.createPrompt('meditation_teacher', {
  system_prompt: 'You are a wise meditation teacher...',
  examples: ['Guide the user through a {{duration}} minute meditation'],
  version: '2.0.0',
  feature_flags: ['enhanced_personas'],
  is_default: true,
});

// Get active prompt for a user
const activePrompt = await client.ai.getActivePrompt('meditation_teacher', {
  userId: 'user_12345',
});
console.log(activePrompt.system_prompt);
console.log(activePrompt.source); // 'versioned_storage' | 'fallback' | 'fallback_no_access'

// Update feature flag rollout
await client.ai.updateFeatureFlag('advanced_reasoning', {
  enabled: true,
  rollout: 50,
});


Questions?#

  • Technical Support: api-support@lilith.app
  • Documentation: GitHub Issues