Document Version: 1.1 Date: 2025-10-03 Companion Document: THREAT_MODEL.md
This document provides detailed data flow diagrams for the Lilith platform to support threat modeling and security analysis.
Diagram Legend#
text
┌─────────┐ Process/Service
│ Process │
└─────────┘
┌─────────┐ External Entity
│ Entity │◄
└─────────┘
╔═════════╗ Data Store
║ Store ║
╚═════════╝
────────► Data Flow
Label
═══════► Trusted Boundary Crossing
Level 0: Context Diagram#
text
┌─────────────────┐
│ End Users │
│ (Web/Mobile) │
└────────┬────────┘
│
Auth Requests
Chat Messages
Content Requests
│
▼
┌─────────────┐ ┌─────────────────┐ ┌─────────────┐
│ Admins │◄───│ │───►│ AI Providers│
│ (Console) │ │ Lilith AI │ │ (OpenAI, │
└─────────────┘ │ Platform │ │ Anthropic) │
▲ │ │ └─────────────┘
│ └─────────┬───────┘ ▲
Admin Commands │ │
System Metrics │ AI Completions
│ ▼ │
│ ┌─────────────────┐ │
└───────────│ Data Stores │──────────┘
│ (DB, S3, Cache) │
└─────────────────┘
▲
│
Conversation Storage
Content Storage
User Data
flowchart TB
Users(("End Users"))
Admins(("Admins"))
Platform["Lilith AI Platform"]
Providers[("AI Providers")]
Stores[("Data Stores")]
Users -- "Auth / Chat / Content" --> Platform
Admins -- "Admin Commands" --> Platform
Platform -- "Operational Metrics" --> Admins
Platform -- "Prompts / Completions" --> Providers
Providers -- "AI Responses" --> Platform
Platform -- "Conversation / Content / User Data" --> Stores
Stores -- "Context & Metadata" --> Platform
Level 1: System Decomposition#
text
┌─────────────────┐
│ End Users │
└────────┬────────┘
│
┌───────▼────────┐
│ BFF │ ◄──── API Gateway
│ (Aggregator) │ Rate Limiting
└───────┬────────┘ Auth Check
│
┌───────────────┼───────────────┐
│ │ │
▼ ▼ ▼
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Auth │ │ Conversation │ │ Content │
│ Service │ │ Service │ │ Service │
└──────┬───────┘ └──────┬───────┘ └──────┬───────┘
│ │ │
│ ▼ ▼
│ ┌──────────────┐ ┌──────────────┐
│ │ AI Service │ │Media Service │
│ │ (LLM Proxy) │ │ (S3/CDN) │
│ └──────┬───────┘ └──────────────┘
│ │
│ ▼
│ ┌──────────────┐
│ │ Moderation │
│ │ Service │
│ └──────────────┘
│
▼
╔══════════════╗
║ User Store ║
║ (PostgreSQL) ║
╚══════════════╝
Level 2: Authentication Flow#
text
┌─────────────┐ ┌─────────────────┐
│ Client │ 1. Login Request │ BFF │
│ Application │ ───────────────────► (Auth Proxy) │
└─────────────┘ └─────────┬───────┘
▲ │
│ 2. Forward to Auth
│ ▼
│ ┌─────────────────┐
│ │ Auth Service │
│ └─────────┬───────┘
│ │
│ 8. Return JWT │ 3. Validate Input
│ ▼
│ ┌─────────────────┐
│ │ Password Hasher │
│ └─────────┬───────┘
│ │
│ 4. Hash & Compare
│ ▼
│ ╔═════════════════╗
│ ║ User Store ║
│ 7. Session Created ║ (PostgreSQL) ║
│ ╚═════════┬═══════╝
│ │
│ 5. User Found
│ ▼
│ ┌─────────────────┐
│ │ JWT Generator │
│ └─────────┬───────┘
│ │
│ 6. Create Session
│ ▼
│ ╔═════════════════╗
└─────────────────────────── ║ Session Store ║
║ (Redis) ║
╚═════════════════╝
Level 2: Chat Message Processing#
text
┌─────────────┐ ┌─────────────────┐
│ Client │ 1. Chat Message │ BFF │
│ (WebSocket) │ ───────────────────► (Stream Handler) │
└─────────────┘ └─────────┬───────┘
▲ │
│ 2. Validate Token
│ 10. Stream Response │
│ ▼
│ ┌─────────────────┐
│ │ Conversation │ 3. Store Message
│ │ Service │ ──────────────────┐
│ └─────────┬───────┘ │
│ │ ▼
│ 4. Forward to AI ╔═══════════╗
│ ▼ ║ Messages ║
│ ┌─────────────────┐ ║ DB ║
│ │ AI Service │ ╚═══════════╝
│ │ (Orchestrator) │
│ └─────────┬───────┘
│ │
│ 9. Stream Back 5. Get System Prompt
│ ▼
│ ╔═════════════════╗
│ ║ Persona/Prompt ║
│ ║ Store ║
│ ╚═════════┬═══════╝
│ │
│ 6. Build Full Prompt
│ ▼
│ ┌─────────────────┐ 7. API Call ┌──────────────┐
│ │ AI Provider │ ─────────────► │ OpenAI │
│ │ Adapter │ ◄───────────── │ Anthropic │
│ └─────────┬───────┘ 8. Response │ (External) │
│ │ └──────────────┘
└─────────────────────────────────────┘
Level 2: Content Delivery Flow#
text
┌─────────────┐ ┌─────────────────┐
│ Client │ 1. Content Request │ BFF │
│ (Browser) │ ───────────────────► (Content Proxy) │
└─────────────┘ └─────────┬───────┘
▲ │
│ 2. Route Request
│ 8. Return Content ▼
│ ┌─────────────────┐
│ │ Content Service │
│ └─────────┬───────┘
│ │
│ 3. Check Permissions
│ ▼
│ ╔═════════════════╗
│ ║ Content Catalog ║
│ ║ (PostgreSQL) ║
│ ╚═════════┬═══════╝
│ │
│ 4. Get Metadata
│ ▼
│ ┌─────────────────┐
│ │ Media Service │
│ 7. Deliver Content └─────────┬───────┘
│ │
│ 5. Generate Signed URL
│ ▼
│ ╔═════════════════╗
│ ║ File Store ║
│ ║ (S3/MinIO) ║
│ ╚═════════┬═══════╝
│ │
│ 6. Retrieve File
└─────────────────────────────────────┘
Level 2: Notification Flow#
text
┌─────────────┐ ┌─────────────────┐
│ System │ 1. Trigger Event │ Notification │
│ Services │ ───────────────────► Service │
└─────────────┘ └─────────┬───────┘
│
2. Process Event
▼
┌─────────────────┐
│ Template Engine │
└─────────┬───────┘
│
3. Render Template
▼
╔═════════════════╗
║ Template Store ║
║ (File System) ║
╚═════════┬═══════╝
│
4. Get User Prefs
▼
╔═════════════════╗
║ User Preferences║
║ (PostgreSQL) ║
╚═════════┬═══════╝
│
5. Check Delivery Rules
▼
┌─────────────────┐
│ Delivery Router │
└─────────┬───────┘
│
┌────────────────────────────────┼────────────────────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Email Provider │ │ Push Gateway │ │ SMS Gateway │
│ (External) │ │ (APNS/FCM) │ │ (Twilio) │
└─────────────────┘ └─────────────────┘ └─────────────────┘
Level 2: Media Processing Flow#
text
┌─────────────┐ ┌─────────────────┐
│ Content │ 1. Upload Request │ Media Service │
│ Manager │ ───────────────────► (File Handler) │
└─────────────┘ └─────────┬───────┘
│
2. Validate File
▼
┌─────────────────┐
│ File Validator │
│ (Type/Size/Sec) │
└─────────┬───────┘
│
3. Generate UUID
▼
┌─────────────────┐
│ File Processor │
└─────────┬───────┘
│
┌────────────────────────────────┼────────────────────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Image Processor │ │ Audio Processor │ │ Document Parser │
│ (Thumbnails) │ │ (Transcription) │ │ (Text Extract) │
└─────────┬───────┘ └─────────┬───────┘ └─────────┬───────┘
│ │ │
└────────────────────────────────┼────────────────────────────────┘
│
4. Store Metadata
▼
╔═════════════════╗
║ Media Metadata ║
║ (PostgreSQL) ║
╚═════════┬═══════╝
│
5. Store File
▼
╔═════════════════╗
║ File Store ║
║ (S3/MinIO) ║
╚═════════════════╝
Trust Boundaries & Security Zones#
Zone 1: Public Internet#
- Components: User browsers, mobile apps, bots
- Trust Level: Untrusted
- Security Controls: WAF, DDoS protection, rate limiting
- Data Classification: Public only
Zone 2: DMZ/Edge Layer#
- Components: Load balancer, API gateway, CDN
- Trust Level: Low trust
- Security Controls: Authentication, SSL termination, request filtering
- Data Classification: Public + authenticated user data
Zone 3: Application Layer#
- Components: BFF, core services
- Trust Level: Medium trust
- Security Controls: mTLS, service authentication, input validation
- Data Classification: User data, business logic
Zone 4: Data Layer#
- Components: Databases, cache, file storage
- Trust Level: High trust
- Security Controls: Encryption at rest, access controls, audit logging
- Data Classification: All data including PII
Zone 5: Management Layer#
- Components: Admin interfaces, monitoring, CI/CD
- Trust Level: High trust
- Security Controls: VPN, MFA, privileged access management
- Data Classification: System credentials, configuration
Data Classification & Flow Security#
Data Types & Security Requirements#
| Data Type | Classification | Encryption Required | Access Controls | Retention |
|---|---|---|---|---|
| User Credentials | Restricted | Yes (Hash+Salt) | Admin only | Until account deletion |
| Conversation History | Confidential | Yes | User + authorized systems | User configurable |
| AI Prompts/Responses | Confidential | Yes | Service-to-service only | 90 days |
| Personal Information | Confidential | Yes | User + authorized systems | GDPR compliance |
| System Logs | Internal | Yes | DevOps team | 1 year |
| Public Content | Public | No | Read-only public | Indefinite |
| API Keys/Secrets | Restricted | Yes | Service accounts only | Rotation schedule |
| Metrics/Analytics | Internal | No | Authorized systems | 2 years |
Cross-Boundary Data Flow Rules#
-
Internet → DMZ
- All traffic must be HTTPS
- Authentication required for protected resources
- Rate limiting enforced
-
DMZ → Application Layer
- mTLS required for service communication
- Token-based authorization
- Request/response validation
-
Application → Data Layer
- Encrypted connections (TLS)
- Database user authentication
- Query parameter validation
-
Internal Service Communication
- Service mesh with automatic mTLS
- Service identity verification
- Request tracing and logging
Security Control Points#
Input Validation Points#
- Client-side validation (first line of defense)
- API gateway validation (schema enforcement)
- Service-level validation (business logic)
- Database validation (constraints and triggers)
Authentication Points#
- User authentication (JWT tokens)
- Service authentication (mTLS certificates)
- Database authentication (connection credentials)
- External service authentication (API keys)
Authorization Points#
- API endpoint authorization (role-based)
- Service-to-service authorization (service identity)
- Data access authorization (user ownership)
- Admin operation authorization (privileged access)
Encryption Points#
- Data in transit (TLS/mTLS)
- Data at rest (KMS encryption)
- Application secrets (encrypted configuration)
- Client-side sensitive data (local encryption)
Monitoring & Logging Points#
Security Event Collection#
- Authentication events (login/logout/failures)
- Authorization events (access grants/denials)
- Data access events (read/write operations)
- Administrative events (configuration changes)
Network Monitoring#
- Traffic flow analysis (unusual patterns)
- Connection monitoring (failed connections)
- Bandwidth monitoring (potential exfiltration)
- Geographic analysis (suspicious locations)
Application Monitoring#
- Error rate monitoring (potential attacks)
- Response time monitoring (DoS indicators)
- Business logic monitoring (policy violations)
- Content monitoring (inappropriate content)
Integration Notes:
- These DFDs support the threat analysis in THREAT_MODEL.md
- Security controls referenced should be implemented per security requirements
- Regular review and updates required as system evolves
- Use these diagrams for security architecture reviews and penetration testing scope