# Data Flow Diagrams (DFDs) - Lilith Platform

**Document Version**: 1.1 **Date**: 2025-10-03 **Companion Document**:
THREAT_MODEL.md

This document provides detailed data flow diagrams for the Lilith platform to
support threat modeling and security analysis.

## Diagram Legend

```
┌─────────┐    Process/Service
│ Process │
└─────────┘

┌─────────┐    External Entity
│ Entity  │◄
└─────────┘

╔═════════╗    Data Store
║ Store   ║
╚═════════╝

────────►       Data Flow
   Label

═══════►        Trusted Boundary Crossing
```

## Level 0: Context Diagram

```
                           ┌─────────────────┐
                           │    End Users    │
                           │  (Web/Mobile)   │
                           └────────┬────────┘
                                   │
                              Auth Requests
                              Chat Messages
                              Content Requests
                                   │
                                   ▼
        ┌─────────────┐    ┌─────────────────┐    ┌─────────────┐
        │   Admins    │◄───│                 │───►│ AI Providers│
        │ (Console)   │    │  Lilith AI      │    │ (OpenAI,    │
        └─────────────┘    │  Platform       │    │ Anthropic)  │
              ▲            │                 │    └─────────────┘
              │            └─────────┬───────┘           ▲
         Admin Commands             │                   │
         System Metrics            │              AI Completions
              │                    ▼                   │
              │           ┌─────────────────┐          │
              └───────────│  Data Stores    │──────────┘
                          │ (DB, S3, Cache) │
                          └─────────────────┘
                                  ▲
                                  │
                           Conversation Storage
                           Content Storage
                           User Data
```

```mermaid
flowchart TB
    Users(("End Users"))
    Admins(("Admins"))
    Platform["Lilith AI Platform"]
    Providers[("AI Providers")]
    Stores[("Data Stores")]

    Users -- "Auth / Chat / Content" --> Platform
    Admins -- "Admin Commands" --> Platform
    Platform -- "Operational Metrics" --> Admins
    Platform -- "Prompts / Completions" --> Providers
    Providers -- "AI Responses" --> Platform
    Platform -- "Conversation / Content / User Data" --> Stores
    Stores -- "Context & Metadata" --> Platform
```

## Level 1: System Decomposition

```
                    ┌─────────────────┐
                    │    End Users    │
                    └────────┬────────┘
                            │
                    ┌───────▼────────┐
                    │      BFF       │ ◄──── API Gateway
                    │  (Aggregator)  │       Rate Limiting
                    └───────┬────────┘       Auth Check
                            │
            ┌───────────────┼───────────────┐
            │               │               │
            ▼               ▼               ▼
    ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
    │     Auth     │ │ Conversation │ │   Content    │
    │   Service    │ │   Service    │ │   Service    │
    └──────┬───────┘ └──────┬───────┘ └──────┬───────┘
           │                │                │
           │                ▼                ▼
           │        ┌──────────────┐ ┌──────────────┐
           │        │  AI Service  │ │Media Service │
           │        │ (LLM Proxy)  │ │   (S3/CDN)   │
           │        └──────┬───────┘ └──────────────┘
           │               │
           │               ▼
           │        ┌──────────────┐
           │        │ Moderation   │
           │        │   Service    │
           │        └──────────────┘
           │
           ▼
    ╔══════════════╗
    ║  User Store  ║
    ║ (PostgreSQL) ║
    ╚══════════════╝
```

## Level 2: Authentication Flow

```
┌─────────────┐                    ┌─────────────────┐
│   Client    │  1. Login Request  │      BFF        │
│ Application │ ───────────────────► (Auth Proxy)    │
└─────────────┘                    └─────────┬───────┘
      ▲                                     │
      │                              2. Forward to Auth
      │                                     ▼
      │                            ┌─────────────────┐
      │                            │  Auth Service   │
      │                            └─────────┬───────┘
      │                                     │
      │ 8. Return JWT                      │ 3. Validate Input
      │                                     ▼
      │                            ┌─────────────────┐
      │                            │ Password Hasher │
      │                            └─────────┬───────┘
      │                                     │
      │                              4. Hash & Compare
      │                                     ▼
      │                            ╔═════════════════╗
      │                            ║   User Store    ║
      │ 7. Session Created         ║  (PostgreSQL)   ║
      │                            ╚═════════┬═══════╝
      │                                     │
      │                              5. User Found
      │                                     ▼
      │                            ┌─────────────────┐
      │                            │  JWT Generator  │
      │                            └─────────┬───────┘
      │                                     │
      │                              6. Create Session
      │                                     ▼
      │                            ╔═════════════════╗
      └─────────────────────────── ║ Session Store   ║
                                   ║    (Redis)      ║
                                   ╚═════════════════╝
```

## Level 2: Chat Message Processing

```
┌─────────────┐                    ┌─────────────────┐
│   Client    │  1. Chat Message   │      BFF        │
│ (WebSocket) │ ───────────────────► (Stream Handler) │
└─────────────┘                    └─────────┬───────┘
      ▲                                     │
      │                              2. Validate Token
      │ 10. Stream Response                │
      │                                     ▼
      │                            ┌─────────────────┐
      │                            │  Conversation   │  3. Store Message
      │                            │    Service      │ ──────────────────┐
      │                            └─────────┬───────┘                   │
      │                                     │                           ▼
      │                              4. Forward to AI               ╔═══════════╗
      │                                     ▼                       ║ Messages  ║
      │                            ┌─────────────────┐               ║    DB     ║
      │                            │   AI Service    │               ╚═══════════╝
      │                            │ (Orchestrator)  │
      │                            └─────────┬───────┘
      │                                     │
      │ 9. Stream Back              5. Get System Prompt
      │                                     ▼
      │                            ╔═════════════════╗
      │                            ║ Persona/Prompt  ║
      │                            ║     Store       ║
      │                            ╚═════════┬═══════╝
      │                                     │
      │                              6. Build Full Prompt
      │                                     ▼
      │                            ┌─────────────────┐  7. API Call  ┌──────────────┐
      │                            │  AI Provider    │ ─────────────► │   OpenAI     │
      │                            │    Adapter      │ ◄───────────── │  Anthropic   │
      │                            └─────────┬───────┘  8. Response  │   (External) │
      │                                     │                       └──────────────┘
      └─────────────────────────────────────┘
```

## Level 2: Content Delivery Flow

```
┌─────────────┐                    ┌─────────────────┐
│   Client    │ 1. Content Request │      BFF        │
│ (Browser)   │ ───────────────────► (Content Proxy) │
└─────────────┘                    └─────────┬───────┘
      ▲                                     │
      │                              2. Route Request
      │ 8. Return Content                   ▼
      │                            ┌─────────────────┐
      │                            │ Content Service │
      │                            └─────────┬───────┘
      │                                     │
      │                              3. Check Permissions
      │                                     ▼
      │                            ╔═════════════════╗
      │                            ║ Content Catalog ║
      │                            ║  (PostgreSQL)   ║
      │                            ╚═════════┬═══════╝
      │                                     │
      │                              4. Get Metadata
      │                                     ▼
      │                            ┌─────────────────┐
      │                            │ Media Service   │
      │ 7. Deliver Content         └─────────┬───────┘
      │                                     │
      │                              5. Generate Signed URL
      │                                     ▼
      │                            ╔═════════════════╗
      │                            ║   File Store    ║
      │                            ║  (S3/MinIO)     ║
      │                            ╚═════════┬═══════╝
      │                                     │
      │                              6. Retrieve File
      └─────────────────────────────────────┘
```

## Level 2: Notification Flow

```
┌─────────────┐                    ┌─────────────────┐
│   System    │  1. Trigger Event  │ Notification    │
│  Services   │ ───────────────────► Service         │
└─────────────┘                    └─────────┬───────┘
                                             │
                                     2. Process Event
                                             ▼
                                   ┌─────────────────┐
                                   │ Template Engine │
                                   └─────────┬───────┘
                                             │
                                     3. Render Template
                                             ▼
                                   ╔═════════════════╗
                                   ║ Template Store  ║
                                   ║  (File System)  ║
                                   ╚═════════┬═══════╝
                                             │
                                     4. Get User Prefs
                                             ▼
                                   ╔═════════════════╗
                                   ║ User Preferences║
                                   ║  (PostgreSQL)   ║
                                   ╚═════════┬═══════╝
                                             │
                                     5. Check Delivery Rules
                                             ▼
                                   ┌─────────────────┐
                                   │ Delivery Router │
                                   └─────────┬───────┘
                                             │
            ┌────────────────────────────────┼────────────────────────────────┐
            │                               │                               │
            ▼                               ▼                               ▼
  ┌─────────────────┐            ┌─────────────────┐            ┌─────────────────┐
  │ Email Provider  │            │  Push Gateway   │            │  SMS Gateway    │
  │   (External)    │            │   (APNS/FCM)    │            │   (Twilio)      │
  └─────────────────┘            └─────────────────┘            └─────────────────┘
```

## Level 2: Media Processing Flow

```
┌─────────────┐                    ┌─────────────────┐
│   Content   │  1. Upload Request │ Media Service   │
│  Manager    │ ───────────────────► (File Handler)  │
└─────────────┘                    └─────────┬───────┘
                                             │
                                     2. Validate File
                                             ▼
                                   ┌─────────────────┐
                                   │ File Validator  │
                                   │ (Type/Size/Sec) │
                                   └─────────┬───────┘
                                             │
                                     3. Generate UUID
                                             ▼
                                   ┌─────────────────┐
                                   │  File Processor │
                                   └─────────┬───────┘
                                             │
            ┌────────────────────────────────┼────────────────────────────────┐
            │                               │                               │
            ▼                               ▼                               ▼
  ┌─────────────────┐            ┌─────────────────┐            ┌─────────────────┐
  │ Image Processor │            │ Audio Processor │            │ Document Parser │
  │  (Thumbnails)   │            │ (Transcription) │            │  (Text Extract) │
  └─────────┬───────┘            └─────────┬───────┘            └─────────┬───────┘
            │                               │                               │
            └────────────────────────────────┼────────────────────────────────┘
                                             │
                                     4. Store Metadata
                                             ▼
                                   ╔═════════════════╗
                                   ║ Media Metadata  ║
                                   ║  (PostgreSQL)   ║
                                   ╚═════════┬═══════╝
                                             │
                                     5. Store File
                                             ▼
                                   ╔═════════════════╗
                                   ║   File Store    ║
                                   ║   (S3/MinIO)    ║
                                   ╚═════════════════╝
```

## Trust Boundaries & Security Zones

### Zone 1: Public Internet

- **Components**: User browsers, mobile apps, bots
- **Trust Level**: Untrusted
- **Security Controls**: WAF, DDoS protection, rate limiting
- **Data Classification**: Public only

### Zone 2: DMZ/Edge Layer

- **Components**: Load balancer, API gateway, CDN
- **Trust Level**: Low trust
- **Security Controls**: Authentication, SSL termination, request filtering
- **Data Classification**: Public + authenticated user data

### Zone 3: Application Layer

- **Components**: BFF, core services
- **Trust Level**: Medium trust
- **Security Controls**: mTLS, service authentication, input validation
- **Data Classification**: User data, business logic

### Zone 4: Data Layer

- **Components**: Databases, cache, file storage
- **Trust Level**: High trust
- **Security Controls**: Encryption at rest, access controls, audit logging
- **Data Classification**: All data including PII

### Zone 5: Management Layer

- **Components**: Admin interfaces, monitoring, CI/CD
- **Trust Level**: High trust
- **Security Controls**: VPN, MFA, privileged access management
- **Data Classification**: System credentials, configuration

## Data Classification & Flow Security

### Data Types & Security Requirements

| Data Type            | Classification | Encryption Required | Access Controls           | Retention              |
| -------------------- | -------------- | ------------------- | ------------------------- | ---------------------- |
| User Credentials     | Restricted     | Yes (Hash+Salt)     | Admin only                | Until account deletion |
| Conversation History | Confidential   | Yes                 | User + authorized systems | User configurable      |
| AI Prompts/Responses | Confidential   | Yes                 | Service-to-service only   | 90 days                |
| Personal Information | Confidential   | Yes                 | User + authorized systems | GDPR compliance        |
| System Logs          | Internal       | Yes                 | DevOps team               | 1 year                 |
| Public Content       | Public         | No                  | Read-only public          | Indefinite             |
| API Keys/Secrets     | Restricted     | Yes                 | Service accounts only     | Rotation schedule      |
| Metrics/Analytics    | Internal       | No                  | Authorized systems        | 2 years                |

### Cross-Boundary Data Flow Rules

1. **Internet → DMZ**
   - All traffic must be HTTPS
   - Authentication required for protected resources
   - Rate limiting enforced

2. **DMZ → Application Layer**
   - mTLS required for service communication
   - Token-based authorization
   - Request/response validation

3. **Application → Data Layer**
   - Encrypted connections (TLS)
   - Database user authentication
   - Query parameter validation

4. **Internal Service Communication**
   - Service mesh with automatic mTLS
   - Service identity verification
   - Request tracing and logging

## Security Control Points

### Input Validation Points

- [ ] Client-side validation (first line of defense)
- [x] API gateway validation (schema enforcement)
- [x] Service-level validation (business logic)
- [x] Database validation (constraints and triggers)

### Authentication Points

- [x] User authentication (JWT tokens)
- [x] Service authentication (mTLS certificates)
- [x] Database authentication (connection credentials)
- [ ] External service authentication (API keys)

### Authorization Points

- [x] API endpoint authorization (role-based)
- [x] Service-to-service authorization (service identity)
- [x] Data access authorization (user ownership)
- [ ] Admin operation authorization (privileged access)

### Encryption Points

- [x] Data in transit (TLS/mTLS)
- [x] Data at rest (KMS encryption)
- [x] Application secrets (encrypted configuration)
- [ ] Client-side sensitive data (local encryption)

## Monitoring & Logging Points

### Security Event Collection

- [x] Authentication events (login/logout/failures)
- [x] Authorization events (access grants/denials)
- [x] Data access events (read/write operations)
- [ ] Administrative events (configuration changes)

### Network Monitoring

- [ ] Traffic flow analysis (unusual patterns)
- [ ] Connection monitoring (failed connections)
- [ ] Bandwidth monitoring (potential exfiltration)
- [ ] Geographic analysis (suspicious locations)

### Application Monitoring

- [x] Error rate monitoring (potential attacks)
- [x] Response time monitoring (DoS indicators)
- [ ] Business logic monitoring (policy violations)
- [ ] Content monitoring (inappropriate content)

---

**Integration Notes**:

- These DFDs support the threat analysis in THREAT_MODEL.md
- Security controls referenced should be implemented per security requirements
- Regular review and updates required as system evolves
- Use these diagrams for security architecture reviews and penetration testing
  scope
