Eve library

Data Flow Diagrams (DFDs) - Lilith Platform

Guide

Revision 1

On this page

Document Version: 1.1 Date: 2025-10-03 Companion Document: THREAT_MODEL.md

This document provides detailed data flow diagrams for the Lilith platform to support threat modeling and security analysis.

Diagram Legend#

text
┌─────────┐    Process/Service
│ Process │
└─────────┘

┌─────────┐    External Entity
│ Entity  │◄
└─────────┘

╔═════════╗    Data Store
║ Store   ║
╚═════════╝

────────►       Data Flow
   Label

═══════►        Trusted Boundary Crossing

Level 0: Context Diagram#

text
                           ┌─────────────────┐
                           │    End Users    │
                           │  (Web/Mobile)   │
                           └────────┬────────┘
                                   │
                              Auth Requests
                              Chat Messages
                              Content Requests
                                   │
                                   ▼
        ┌─────────────┐    ┌─────────────────┐    ┌─────────────┐
        │   Admins    │◄───│                 │───►│ AI Providers│
        │ (Console)   │    │  Lilith AI      │    │ (OpenAI,    │
        └─────────────┘    │  Platform       │    │ Anthropic)  │
              ▲            │                 │    └─────────────┘
              │            └─────────┬───────┘           ▲
         Admin Commands             │                   │
         System Metrics            │              AI Completions
              │                    ▼                   │
              │           ┌─────────────────┐          │
              └───────────│  Data Stores    │──────────┘
                          │ (DB, S3, Cache) │
                          └─────────────────┘
                                  ▲
                                  │
                           Conversation Storage
                           Content Storage
                           User Data
flowchart TB Users(("End Users")) Admins(("Admins")) Platform["Lilith AI Platform"] Providers[("AI Providers")] Stores[("Data Stores")] Users -- "Auth / Chat / Content" --> Platform Admins -- "Admin Commands" --> Platform Platform -- "Operational Metrics" --> Admins Platform -- "Prompts / Completions" --> Providers Providers -- "AI Responses" --> Platform Platform -- "Conversation / Content / User Data" --> Stores Stores -- "Context & Metadata" --> Platform

Level 1: System Decomposition#

text
                    ┌─────────────────┐
                    │    End Users    │
                    └────────┬────────┘
                            │
                    ┌───────▼────────┐
                    │      BFF       │ ◄──── API Gateway
                    │  (Aggregator)  │       Rate Limiting
                    └───────┬────────┘       Auth Check
                            │
            ┌───────────────┼───────────────┐
            │               │               │
            ▼               ▼               ▼
    ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
    │     Auth     │ │ Conversation │ │   Content    │
    │   Service    │ │   Service    │ │   Service    │
    └──────┬───────┘ └──────┬───────┘ └──────┬───────┘
           │                │                │
           │                ▼                ▼
           │        ┌──────────────┐ ┌──────────────┐
           │        │  AI Service  │ │Media Service │
           │        │ (LLM Proxy)  │ │   (S3/CDN)   │
           │        └──────┬───────┘ └──────────────┘
           │               │
           │               ▼
           │        ┌──────────────┐
           │        │ Moderation   │
           │        │   Service    │
           │        └──────────────┘
           │
           ▼
    ╔══════════════╗
    ║  User Store  ║
    ║ (PostgreSQL) ║
    ╚══════════════╝

Level 2: Authentication Flow#

text
┌─────────────┐                    ┌─────────────────┐
│   Client    │  1. Login Request  │      BFF        │
│ Application │ ───────────────────► (Auth Proxy)    │
└─────────────┘                    └─────────┬───────┘
      ▲                                     │
      │                              2. Forward to Auth
      │                                     ▼
      │                            ┌─────────────────┐
      │                            │  Auth Service   │
      │                            └─────────┬───────┘
      │                                     │
      │ 8. Return JWT                      │ 3. Validate Input
      │                                     ▼
      │                            ┌─────────────────┐
      │                            │ Password Hasher │
      │                            └─────────┬───────┘
      │                                     │
      │                              4. Hash & Compare
      │                                     ▼
      │                            ╔═════════════════╗
      │                            ║   User Store    ║
      │ 7. Session Created         ║  (PostgreSQL)   ║
      │                            ╚═════════┬═══════╝
      │                                     │
      │                              5. User Found
      │                                     ▼
      │                            ┌─────────────────┐
      │                            │  JWT Generator  │
      │                            └─────────┬───────┘
      │                                     │
      │                              6. Create Session
      │                                     ▼
      │                            ╔═════════════════╗
      └─────────────────────────── ║ Session Store   ║
                                   ║    (Redis)      ║
                                   ╚═════════════════╝

Level 2: Chat Message Processing#

text
┌─────────────┐                    ┌─────────────────┐
│   Client    │  1. Chat Message   │      BFF        │
│ (WebSocket) │ ───────────────────► (Stream Handler) │
└─────────────┘                    └─────────┬───────┘
      ▲                                     │
      │                              2. Validate Token
      │ 10. Stream Response                │
      │                                     ▼
      │                            ┌─────────────────┐
      │                            │  Conversation   │  3. Store Message
      │                            │    Service      │ ──────────────────┐
      │                            └─────────┬───────┘                   │
      │                                     │                           ▼
      │                              4. Forward to AI               ╔═══════════╗
      │                                     ▼                       ║ Messages  ║
      │                            ┌─────────────────┐               ║    DB     ║
      │                            │   AI Service    │               ╚═══════════╝
      │                            │ (Orchestrator)  │
      │                            └─────────┬───────┘
      │                                     │
      │ 9. Stream Back              5. Get System Prompt
      │                                     ▼
      │                            ╔═════════════════╗
      │                            ║ Persona/Prompt  ║
      │                            ║     Store       ║
      │                            ╚═════════┬═══════╝
      │                                     │
      │                              6. Build Full Prompt
      │                                     ▼
      │                            ┌─────────────────┐  7. API Call  ┌──────────────┐
      │                            │  AI Provider    │ ─────────────► │   OpenAI     │
      │                            │    Adapter      │ ◄───────────── │  Anthropic   │
      │                            └─────────┬───────┘  8. Response  │   (External) │
      │                                     │                       └──────────────┘
      └─────────────────────────────────────┘

Level 2: Content Delivery Flow#

text
┌─────────────┐                    ┌─────────────────┐
│   Client    │ 1. Content Request │      BFF        │
│ (Browser)   │ ───────────────────► (Content Proxy) │
└─────────────┘                    └─────────┬───────┘
      ▲                                     │
      │                              2. Route Request
      │ 8. Return Content                   ▼
      │                            ┌─────────────────┐
      │                            │ Content Service │
      │                            └─────────┬───────┘
      │                                     │
      │                              3. Check Permissions
      │                                     ▼
      │                            ╔═════════════════╗
      │                            ║ Content Catalog ║
      │                            ║  (PostgreSQL)   ║
      │                            ╚═════════┬═══════╝
      │                                     │
      │                              4. Get Metadata
      │                                     ▼
      │                            ┌─────────────────┐
      │                            │ Media Service   │
      │ 7. Deliver Content         └─────────┬───────┘
      │                                     │
      │                              5. Generate Signed URL
      │                                     ▼
      │                            ╔═════════════════╗
      │                            ║   File Store    ║
      │                            ║  (S3/MinIO)     ║
      │                            ╚═════════┬═══════╝
      │                                     │
      │                              6. Retrieve File
      └─────────────────────────────────────┘

Level 2: Notification Flow#

text
┌─────────────┐                    ┌─────────────────┐
│   System    │  1. Trigger Event  │ Notification    │
│  Services   │ ───────────────────► Service         │
└─────────────┘                    └─────────┬───────┘
                                             │
                                     2. Process Event
                                             ▼
                                   ┌─────────────────┐
                                   │ Template Engine │
                                   └─────────┬───────┘
                                             │
                                     3. Render Template
                                             ▼
                                   ╔═════════════════╗
                                   ║ Template Store  ║
                                   ║  (File System)  ║
                                   ╚═════════┬═══════╝
                                             │
                                     4. Get User Prefs
                                             ▼
                                   ╔═════════════════╗
                                   ║ User Preferences║
                                   ║  (PostgreSQL)   ║
                                   ╚═════════┬═══════╝
                                             │
                                     5. Check Delivery Rules
                                             ▼
                                   ┌─────────────────┐
                                   │ Delivery Router │
                                   └─────────┬───────┘
                                             │
            ┌────────────────────────────────┼────────────────────────────────┐
            │                               │                               │
            ▼                               ▼                               ▼
  ┌─────────────────┐            ┌─────────────────┐            ┌─────────────────┐
  │ Email Provider  │            │  Push Gateway   │            │  SMS Gateway    │
  │   (External)    │            │   (APNS/FCM)    │            │   (Twilio)      │
  └─────────────────┘            └─────────────────┘            └─────────────────┘

Level 2: Media Processing Flow#

text
┌─────────────┐                    ┌─────────────────┐
│   Content   │  1. Upload Request │ Media Service   │
│  Manager    │ ───────────────────► (File Handler)  │
└─────────────┘                    └─────────┬───────┘
                                             │
                                     2. Validate File
                                             ▼
                                   ┌─────────────────┐
                                   │ File Validator  │
                                   │ (Type/Size/Sec) │
                                   └─────────┬───────┘
                                             │
                                     3. Generate UUID
                                             ▼
                                   ┌─────────────────┐
                                   │  File Processor │
                                   └─────────┬───────┘
                                             │
            ┌────────────────────────────────┼────────────────────────────────┐
            │                               │                               │
            ▼                               ▼                               ▼
  ┌─────────────────┐            ┌─────────────────┐            ┌─────────────────┐
  │ Image Processor │            │ Audio Processor │            │ Document Parser │
  │  (Thumbnails)   │            │ (Transcription) │            │  (Text Extract) │
  └─────────┬───────┘            └─────────┬───────┘            └─────────┬───────┘
            │                               │                               │
            └────────────────────────────────┼────────────────────────────────┘
                                             │
                                     4. Store Metadata
                                             ▼
                                   ╔═════════════════╗
                                   ║ Media Metadata  ║
                                   ║  (PostgreSQL)   ║
                                   ╚═════════┬═══════╝
                                             │
                                     5. Store File
                                             ▼
                                   ╔═════════════════╗
                                   ║   File Store    ║
                                   ║   (S3/MinIO)    ║
                                   ╚═════════════════╝

Trust Boundaries & Security Zones#

Zone 1: Public Internet#

  • Components: User browsers, mobile apps, bots
  • Trust Level: Untrusted
  • Security Controls: WAF, DDoS protection, rate limiting
  • Data Classification: Public only

Zone 2: DMZ/Edge Layer#

  • Components: Load balancer, API gateway, CDN
  • Trust Level: Low trust
  • Security Controls: Authentication, SSL termination, request filtering
  • Data Classification: Public + authenticated user data

Zone 3: Application Layer#

  • Components: BFF, core services
  • Trust Level: Medium trust
  • Security Controls: mTLS, service authentication, input validation
  • Data Classification: User data, business logic

Zone 4: Data Layer#

  • Components: Databases, cache, file storage
  • Trust Level: High trust
  • Security Controls: Encryption at rest, access controls, audit logging
  • Data Classification: All data including PII

Zone 5: Management Layer#

  • Components: Admin interfaces, monitoring, CI/CD
  • Trust Level: High trust
  • Security Controls: VPN, MFA, privileged access management
  • Data Classification: System credentials, configuration

Data Classification & Flow Security#

Data Types & Security Requirements#

Data Type Classification Encryption Required Access Controls Retention
User Credentials Restricted Yes (Hash+Salt) Admin only Until account deletion
Conversation History Confidential Yes User + authorized systems User configurable
AI Prompts/Responses Confidential Yes Service-to-service only 90 days
Personal Information Confidential Yes User + authorized systems GDPR compliance
System Logs Internal Yes DevOps team 1 year
Public Content Public No Read-only public Indefinite
API Keys/Secrets Restricted Yes Service accounts only Rotation schedule
Metrics/Analytics Internal No Authorized systems 2 years

Cross-Boundary Data Flow Rules#

  1. Internet → DMZ

    • All traffic must be HTTPS
    • Authentication required for protected resources
    • Rate limiting enforced
  2. DMZ → Application Layer

    • mTLS required for service communication
    • Token-based authorization
    • Request/response validation
  3. Application → Data Layer

    • Encrypted connections (TLS)
    • Database user authentication
    • Query parameter validation
  4. Internal Service Communication

    • Service mesh with automatic mTLS
    • Service identity verification
    • Request tracing and logging

Security Control Points#

Input Validation Points#

  • Client-side validation (first line of defense)
  • API gateway validation (schema enforcement)
  • Service-level validation (business logic)
  • Database validation (constraints and triggers)

Authentication Points#

  • User authentication (JWT tokens)
  • Service authentication (mTLS certificates)
  • Database authentication (connection credentials)
  • External service authentication (API keys)

Authorization Points#

  • API endpoint authorization (role-based)
  • Service-to-service authorization (service identity)
  • Data access authorization (user ownership)
  • Admin operation authorization (privileged access)

Encryption Points#

  • Data in transit (TLS/mTLS)
  • Data at rest (KMS encryption)
  • Application secrets (encrypted configuration)
  • Client-side sensitive data (local encryption)

Monitoring & Logging Points#

Security Event Collection#

  • Authentication events (login/logout/failures)
  • Authorization events (access grants/denials)
  • Data access events (read/write operations)
  • Administrative events (configuration changes)

Network Monitoring#

  • Traffic flow analysis (unusual patterns)
  • Connection monitoring (failed connections)
  • Bandwidth monitoring (potential exfiltration)
  • Geographic analysis (suspicious locations)

Application Monitoring#

  • Error rate monitoring (potential attacks)
  • Response time monitoring (DoS indicators)
  • Business logic monitoring (policy violations)
  • Content monitoring (inappropriate content)

Integration Notes:

  • These DFDs support the threat analysis in THREAT_MODEL.md
  • Security controls referenced should be implemented per security requirements
  • Regular review and updates required as system evolves
  • Use these diagrams for security architecture reviews and penetration testing scope