# Data Governance & Privacy Framework

**Version:** 1.0.0 **Last Updated:** October 16, 2025 **Maintained By:**
Platform Team **Review Frequency:** Quarterly

---

## Table of Contents

1. [Overview](#overview)
2. [Data Classification Taxonomy](#data-classification-taxonomy)
3. [Data Inventory Management](#data-inventory-management)
4. [Retention Schedules](#retention-schedules)
5. [Regional Data Residency Rules](#regional-data-residency-rules)
6. [Data Lifecycle Management](#data-lifecycle-management)
7. [Compliance & Reporting](#compliance--reporting)
8. [Data Subject Rights (DSR)](#data-subject-rights-dsr)
9. [API Reference](#api-reference)
10. [Operational Procedures](#operational-procedures)

---

## Overview

### Purpose

The Lilith Data Governance Framework provides comprehensive controls for
managing user data throughout its lifecycle, ensuring compliance with global
privacy regulations including GDPR (EU), CCPA/CPRA (California), PIPEDA
(Canada), and other jurisdictional requirements.

### Core Principles

1. **Privacy by Design**: Data protection integrated into system architecture
2. **Data Minimization**: Collect only necessary data for stated purposes
3. **Purpose Limitation**: Use data only for explicitly stated, legitimate
   purposes
4. **Transparency**: Clear communication about data collection and usage
5. **User Control**: Empower users with rights over their personal data
6. **Security**: Implement appropriate technical and organizational measures
7. **Accountability**: Maintain audit trails and compliance records

### Service Architecture

The data-governance microservice (`services/data-governance`) provides
centralized governance capabilities:

- **Base URL**: `http://data-governance-service:8080/v1`
- **Health Check**: `GET /v1/health`
- **Authentication**: Service-to-service JWT tokens
- **Audit Logging**: All operations logged for compliance

---

## Data Classification Taxonomy

### Classification Levels

#### 1. PII_HIGH (High Sensitivity)

**Description**: Highly sensitive personally identifiable information

**Examples**:

- Email addresses
- Phone numbers
- Government-issued IDs (SSN, passport numbers)
- Biometric data (voice patterns, fingerprints)
- Financial information (credit cards, bank accounts)
- Health information

**Security Requirements**:

- ✅ Encryption at rest (AES-256)
- ✅ Encryption in transit (TLS 1.3)
- ✅ Anonymization required after retention period
- ✅ Regional data localization enforced
- ✅ Access logging mandatory
- ✅ MFA required for access

**Default Retention**: 7 years (2,555 days)

#### 2. PII_MEDIUM (Medium Sensitivity)

**Description**: Moderately sensitive personal information

**Examples**:

- Display names / usernames
- User preferences
- Usage patterns
- Device information
- IP addresses (non-persistent)

**Security Requirements**:

- ✅ Encryption at rest
- ✅ Encryption in transit
- ⚠️ Anonymization optional
- ⚠️ Regional restrictions optional
- ✅ Access logging recommended

**Default Retention**: 3 years (1,095 days)

#### 3. PII_LOW (Low Sensitivity)

**Description**: Low sensitivity personal data or anonymized information

**Examples**:

- Anonymized analytics
- Aggregated metrics
- Public profile information
- Anonymous survey responses

**Security Requirements**:

- ⚠️ Encryption optional (recommended for consistency)
- ✅ Encryption in transit
- ❌ Anonymization not required
- ❌ No regional restrictions

**Default Retention**: 1 year (365 days)

#### 4. CONTENT (User-Generated Content)

**Description**: User-created content and interactions

**Examples**:

- Chat messages with AI mentors
- Audio transcriptions
- Course progress and completions
- Journal entries
- Meditation session logs

**Security Requirements**:

- ✅ Encryption at rest
- ✅ Encryption in transit
- ⚠️ User-controlled deletion
- ⚠️ Anonymization on user request
- ✅ Access logging

**Default Retention**: 6 years (2,190 days)

#### 5. METADATA (System Metadata)

**Description**: System operational data and metadata

**Examples**:

- Timestamps
- Version numbers
- Status flags
- System events
- Performance metrics

**Security Requirements**:

- ⚠️ Encryption optional
- ✅ Encryption in transit
- ❌ No special handling required

**Default Retention**: 2 years (730 days)

#### 6. SECURITY (Security-Related Data)

**Description**: Authentication, authorization, and security event data

**Examples**:

- Password hashes (bcrypt/Argon2)
- Session tokens
- API keys (hashed)
- Audit logs
- Security incident records
- MFA credentials

**Security Requirements**:

- ✅ Encryption at rest (AES-256)
- ✅ Encryption in transit (TLS 1.3)
- ✅ Anonymization required after retention
- ✅ Regional data localization
- ✅ Immutable audit trails
- ✅ Access logging with MFA

**Default Retention**: 7 years (2,555 days)

---

## Data Inventory Management

### Purpose

The Data Inventory tracks all data assets across the platform, enabling:

- **Visibility**: Complete catalog of what data exists
- **Compliance**: Proof of data handling practices
- **Lifecycle Management**: Automated retention and deletion
- **Incident Response**: Rapid identification of affected data

### Inventory Entry Structure

```json
{
  "id": "inv-123",
  "service_name": "conversation-service",
  "data_type": "chat_messages",
  "classification": "CONTENT",
  "classification_rules": {
    "level": "MEDIUM",
    "retention_days": 2190,
    "encryption_required": true,
    "anonymization_required": false
  },
  "location": "us-west-2-postgres-primary",
  "user_id": "user-abc123",
  "metadata": {
    "consent_obtained": true,
    "consent_date": "2025-01-15T10:30:00Z",
    "purpose": "ai_mentoring"
  },
  "schema_definition": {
    "table": "messages",
    "columns": ["id", "user_id", "content", "timestamp"]
  },
  "sensitivity_level": "MEDIUM",
  "regional_scope": ["US"],
  "retention_days": 2190,
  "retention_date": "2031-01-15T10:30:00Z",
  "encryption_required": true,
  "anonymization_required": false,
  "created_at": "2025-01-15T10:30:00Z",
  "updated_at": "2025-01-15T10:30:00Z",
  "last_accessed": "2025-10-15T14:22:00Z",
  "access_count": 47,
  "data_lineage": [],
  "compliance_status": "COMPLIANT",
  "lifecycle_stage": "ACTIVE"
}
```

### Registration Process

**When to Register**:

- ✅ When creating new data storage (tables, collections, buckets)
- ✅ When collecting new user data types
- ✅ Before processing personal data
- ✅ When data crosses service boundaries

**How to Register**:

```bash
POST /v1/data-inventory
Content-Type: application/json

{
  "service_name": "conversation-service",
  "data_type": "chat_messages",
  "classification": "CONTENT",
  "location": "us-west-2-postgres-primary",
  "user_id": "user-abc123",
  "metadata": {
    "consent_obtained": true,
    "purpose": "ai_mentoring"
  },
  "schema_definition": {
    "table": "messages",
    "columns": ["id", "user_id", "content", "timestamp"]
  },
  "regional_scope": ["US"]
}
```

**Response**: Returns inventory entry with auto-assigned retention policies

### Auto-Classification

The system can automatically classify data based on field names:

```bash
POST /v1/data-classifications/auto-classify
Content-Type: application/json

{
  "field_names": ["email", "display_name", "created_at", "chat_content"],
  "service_context": "user-profile-service"
}
```

**Classification Rules**:

- `email`, `phone`, `ssn`, `password`, `biometric` → **PII_HIGH** (95%
  confidence)
- `name`, `address`, `preference`, `profile` → **PII_MEDIUM** (85% confidence)
- `message`, `content`, `transcript`, `conversation` → **CONTENT** (90%
  confidence)
- `session`, `token`, `auth`, `key` → **SECURITY** (90% confidence)
- `created_at`, `updated_at`, `timestamp`, `version` → **METADATA** (85%
  confidence)

---

## Retention Schedules

### Default Retention Periods

| Classification | Default Period       | GDPR EU              | CCPA CA  | PIPEDA CA |
| -------------- | -------------------- | -------------------- | -------- | --------- |
| **PII_HIGH**   | 7 years (2,555 days) | 3 years max          | No limit | 7 years   |
| **PII_MEDIUM** | 3 years (1,095 days) | 2 years max          | No limit | 3 years   |
| **PII_LOW**    | 1 year (365 days)    | 1 year max           | No limit | No limit  |
| **CONTENT**    | 6 years (2,190 days) | No specific limit    | No limit | No limit  |
| **METADATA**   | 2 years (730 days)   | No specific limit    | No limit | No limit  |
| **SECURITY**   | 7 years (2,555 days) | 3 years (audit logs) | No limit | 7 years   |

### Regional Override Logic

When data has multiple regional scopes, the **most restrictive** retention
period applies:

```
Example: PII_HIGH data with regional_scope = ["US", "EU"]
- Default: 2,555 days (7 years)
- EU GDPR Override: 1,095 days (3 years)
- **Applied Retention**: 1,095 days (most restrictive)
```

### Retention Policy Management

#### Create Custom Policy

```bash
POST /v1/retention-policies
Content-Type: application/json

{
  "name": "Chat Message Retention - GDPR Compliant",
  "description": "Retain chat messages for 2 years with automatic anonymization",
  "data_classifications": ["CONTENT"],
  "retention_days": 730,
  "auto_delete": false,
  "archive_before_delete": true,
  "regional_overrides": {
    "EU": 730,
    "US": 2190
  },
  "exceptions": [
    {
      "condition": "legal_hold",
      "action": "PRESERVE"
    }
  ]
}
```

#### Preview Policy Impact

```bash
GET /v1/retention-policies/policy-123/preview

Response:
{
  "policy": { ... },
  "affected_entries_count": 15420,
  "expired_entries_count": 230,
  "estimated_deletion_date": "2025-10-17T02:00:00Z",
  "sample_expired_entries": [ ... ]
}
```

### Lifecycle Stages

```
ACTIVE → APPROACHING_EXPIRY → ARCHIVED → ANONYMIZING → ANONYMIZED → DELETED
  |           (30 days)           |          |              |            |
  |                               |          |              |            |
  └─> Regular access allowed    ─┘          └─> Read-only  └─> Purged  └─> Permanent
```

**Stage Definitions**:

- **ACTIVE**: Normal operational use, full access
- **APPROACHING_EXPIRY**: <30 days until retention date, warning notifications
  sent
- **ARCHIVED**: Read-only, moved to cold storage, reduced access
- **ANONYMIZING**: PII removal in progress, no access
- **ANONYMIZED**: PII removed, aggregate data may remain
- **DELETED**: Permanent removal, unrecoverable

---

## Regional Data Residency Rules

### Supported Regions

#### 1. European Union (GDPR)

**Regulation**: General Data Protection Regulation (EU) 2016/679

**Requirements**:

- ✅ **Data Localization**: Personal data must be stored within EU borders
- ✅ **Encryption**: At rest (AES-256) and in transit (TLS 1.3+)
- ✅ **Right to be Forgotten**: Must support complete deletion within 30 days
- ✅ **Data Portability**: Machine-readable export in standard formats
- ✅ **Explicit Consent**: Required for processing personal data
- ✅ **Lawful Processing Basis**: One of 6 legal bases required

**Lawful Bases for Processing**:

1. **Consent**: User has given explicit consent
2. **Contract**: Processing necessary to fulfill contract
3. **Legal Obligation**: Required by law (e.g., tax records)
4. **Vital Interests**: Protecting life or health
5. **Public Task**: Performing official functions
6. **Legitimate Interests**: Business purposes (balanced against user rights)

**Retention Limits** (Most Restrictive):

- PII_HIGH: **3 years maximum** (1,095 days)
- PII_MEDIUM: **2 years maximum** (730 days)
- PII_LOW: **1 year maximum** (365 days)

**Data Transfer Rules**:

- ❌ No transfers outside EU without adequacy decision or safeguards
- ✅ Standard Contractual Clauses (SCCs) required for non-EU transfers
- ✅ Binding Corporate Rules (BCRs) for intra-company transfers

#### 2. United States (Federal)

**Regulations**: Varies by state, sector-specific laws (HIPAA, COPPA, GLBA)

**Requirements**:

- ⚠️ **Data Localization**: Not required federally (varies by state)
- ✅ **Encryption**: Industry best practice, required in transit
- ⚠️ **Right to be Forgotten**: Not federally mandated (CCPA/CPRA in CA)
- ⚠️ **Data Portability**: Not federally mandated
- ⚠️ **Consent**: Not federally required (notice & choice model)

**Notable State Laws**:

- **California**: CCPA/CPRA (see below)
- **Virginia**: VCDPA (Consumer Data Protection Act)
- **Colorado**: CPA (Privacy Act)
- **Connecticut**: CTDPA

**Retention**: No federal limits (business necessity standard)

#### 3. California (CCPA/CPRA)

**Regulation**: California Consumer Privacy Act / California Privacy Rights Act

**Requirements**:

- ⚠️ **Data Localization**: Not required
- ✅ **Encryption**: Required for personal information
- ✅ **Right to Delete**: Within 45 days (can extend 45 more days)
- ✅ **Data Portability**: Machine-readable format
- ⚠️ **Opt-Out**: "Do Not Sell" required, not explicit opt-in
- ✅ **Purpose Limitation**: Can't use data for unrelated purposes

**Consumer Rights**:

1. **Know**: What data is collected and how it's used
2. **Delete**: Request deletion of personal information
3. **Opt-Out**: Opt out of sale/sharing of personal information
4. **Non-Discrimination**: Equal service regardless of privacy choices
5. **Correction**: Correct inaccurate personal information (CPRA 2023+)
6. **Limit**: Limit use of sensitive personal information (CPRA 2023+)

**Retention**: Business necessity standard (reasonable retention periods)

#### 4. Canada (PIPEDA)

**Regulation**: Personal Information Protection and Electronic Documents Act

**Requirements**:

- ⚠️ **Data Localization**: Not required, but cross-border transfer rules apply
- ✅ **Encryption**: Safeguards appropriate to sensitivity
- ✅ **Right to be Forgotten**: Access and correction rights
- ✅ **Data Portability**: Access in understandable format
- ✅ **Meaningful Consent**: Clear, comprehensible, opt-in

**Key Principles** (10 Fair Information Principles):

1. Accountability
2. Identifying purposes
3. Consent
4. Limiting collection
5. Limiting use, disclosure, retention
6. Accuracy
7. Safeguards
8. Openness
9. Individual access
10. Challenging compliance

**Retention Limits**:

- PII_HIGH: **7 years** (tax/business records)
- PII_MEDIUM: **3 years** (reasonable period)
- Must destroy/anonymize when no longer needed for stated purpose

### Compliance Validation

#### Validate Regional Compliance

```bash
POST /v1/regional-compliance/validate
Content-Type: application/json

{
  "data_inventory_id": "inv-123",
  "target_regions": ["EU", "US", "CA"]
}

Response:
{
  "data_inventory_id": "inv-123",
  "validation_results": [
    {
      "region": "EU",
      "compliant": false,
      "violations": [
        "DATA_LOCALIZATION_REQUIRED",
        "RETENTION_LIMIT_EXCEEDED"
      ],
      "recommendations": [
        "Data must be stored within EU borders",
        "Retention period must not exceed 1095 days for PII_HIGH data in EU"
      ],
      "regional_rule": { ... }
    },
    {
      "region": "US",
      "compliant": true,
      "violations": [],
      "recommendations": []
    }
  ],
  "overall_compliant": false,
  "validated_at": "2025-10-16T10:30:00Z"
}
```

### Cross-Border Transfer Safeguards

When transferring data between regions:

1. **Assess**: Determine if destination has adequate protection
2. **Document**: Record lawful basis and safeguards
3. **Implement**: Use SCCs, BCRs, or other approved mechanisms
4. **Monitor**: Regular reviews of third-party compliance
5. **Notify**: Inform users of international transfers

---

## Data Lifecycle Management

### Lifecycle Automation Rules

Lifecycle rules automatically apply actions based on conditions:

```bash
POST /v1/lifecycle-rules
Content-Type: application/json

{
  "name": "Auto-Archive Old Chat Messages",
  "description": "Archive messages older than 2 years with zero access",
  "trigger_conditions": {
    "age_days": 730,
    "access_count_threshold": 0
  },
  "actions": ["ARCHIVE"],
  "data_classifications": ["CONTENT"],
  "schedule": {
    "frequency": "daily",
    "time": "02:00"
  },
  "enabled": true
}
```

### Available Actions

#### 1. ARCHIVE

**Purpose**: Move data to cold storage for compliance retention

**Effect**:

- Data becomes read-only
- Access requires elevated permissions
- Moved to lower-cost storage tier
- Search indexing disabled

**Typical Use Cases**:

- Chat messages > 2 years old with no access
- Completed course data after 1 year
- Inactive user accounts (90+ days no login)

#### 2. ANONYMIZE

**Purpose**: Remove PII while preserving aggregate/statistical value

**Effect**:

- User IDs replaced with anonymous identifiers
- Email/phone/name fields removed or hashed
- Behavioral data retained for analytics
- Irreversible process

**Typical Use Cases**:

- PII_HIGH data past retention period
- User deletion requests (Right to be Forgotten)
- Research datasets for model training

**Anonymization Techniques**:

- **Generalization**: Replace specific values with ranges (age 28 → 25-30)
- **Suppression**: Remove direct identifiers entirely
- **Perturbation**: Add statistical noise to values
- **Pseudonymization**: Replace IDs with irreversible tokens
- **K-Anonymity**: Ensure minimum group size of K
- **Differential Privacy**: Add calibrated noise for privacy guarantees

#### 3. DELETE

**Purpose**: Permanent, unrecoverable removal

**Effect**:

- Data purged from all systems (primary, backups, caches)
- Audit log entry created (non-reversible)
- Storage space reclaimed
- User receives deletion confirmation

**Typical Use Cases**:

- Data past legal retention requirement
- User account deletion (after grace period)
- Invalid/corrupted data
- Security incident (compromised data)

### Execute Lifecycle Rule

```bash
POST /v1/lifecycle-rules/rule-123/execute
Content-Type: application/json

{
  "dry_run": true
}

Response:
{
  "rule_id": "rule-123",
  "dry_run": true,
  "entries_processed": 1240,
  "execution_results": [
    {
      "inventory_id": "inv-456",
      "actions_applied": ["ARCHIVE"],
      "status": "SUCCESS",
      "errors": []
    },
    ...
  ],
  "executed_at": "2025-10-16T02:00:00Z"
}
```

**Best Practice**: Always run with `dry_run: true` first to preview impact

---

## Compliance & Reporting

### Report Types

#### 1. Data Inventory Summary

```bash
POST /v1/compliance-reports
Content-Type: application/json

{
  "report_type": "DATA_INVENTORY_SUMMARY",
  "scope": {
    "services": ["conversation", "content"],
    "date_range": {
      "start": "2025-01-01",
      "end": "2025-10-16"
    }
  },
  "include_recommendations": true
}

Response:
{
  "id": "report-789",
  "report_type": "DATA_INVENTORY_SUMMARY",
  "data": {
    "total_entries": 15420,
    "by_classification": {
      "PII_HIGH": 2340,
      "PII_MEDIUM": 5680,
      "CONTENT": 6200,
      "METADATA": 1200
    },
    "by_service": {
      "conversation-service": 8400,
      "content-service": 7020
    },
    "by_region": {
      "US": 10200,
      "EU": 5220
    },
    "by_lifecycle_stage": {
      "ACTIVE": 14100,
      "ARCHIVED": 1200,
      "DELETED": 120
    },
    "compliance_overview": {
      "compliant": 15300,
      "non_compliant": 120,
      "pending_review": 0
    }
  },
  "generated_at": "2025-10-16T10:30:00Z",
  "status": "COMPLETED"
}
```

#### 2. Retention Compliance Report

**Purpose**: Identify expired data requiring action

```json
{
  "report_type": "RETENTION_COMPLIANCE",
  "data": {
    "total_entries": 15420,
    "retention_status": {
      "within_policy": 14800,
      "approaching_expiry": 390,
      "expired": 230,
      "no_policy": 0
    },
    "upcoming_deletions": [
      {
        "inventory_id": "inv-234",
        "deletion_date": "2025-10-20",
        "days_remaining": 4
      }
    ],
    "policy_violations": [
      {
        "inventory_id": "inv-567",
        "violation_type": "RETENTION_EXPIRED",
        "days_overdue": 12
      }
    ]
  }
}
```

#### 3. Regional Compliance Report

**Purpose**: Verify adherence to regional regulations

```json
{
  "report_type": "REGIONAL_COMPLIANCE",
  "data": {
    "supported_regions": ["EU", "US", "CA"],
    "compliance_by_region": {
      "EU": {
        "total_entries": 5220,
        "compliant": 5100,
        "non_compliant": 120,
        "compliance_rate": 98
      },
      "US": {
        "total_entries": 10200,
        "compliant": 10200,
        "non_compliant": 0,
        "compliance_rate": 100
      }
    },
    "violations": [
      {
        "inventory_id": "inv-890",
        "region": "EU",
        "violations": ["DATA_LOCALIZATION_REQUIRED"]
      }
    ]
  }
}
```

---

## Data Subject Rights (DSR)

### Supported Request Types

#### 1. ACCESS (Right to Know)

User requests copy of all personal data held by the platform.

**Response Timeline**: 30 days (GDPR), 45 days (CCPA)

**Data Provided**:

- Profile information
- Chat history
- Course progress
- Usage analytics
- Account settings
- Third-party data sources

**Format**: Machine-readable JSON, CSV, or PDF

#### 2. PORTABILITY (Right to Data Portability)

User requests data in portable format to transfer to another service.

**Format**: JSON, CSV (structured data)

**Scope**: Only user-provided data and system-generated data derived from it

#### 3. RECTIFICATION (Right to Correction)

User requests correction of inaccurate personal data.

**Timeline**: 1 business day for simple corrections

**Process**:

1. User identifies incorrect data
2. Platform verifies identity
3. Correction applied
4. User receives confirmation

#### 4. ERASURE (Right to be Forgotten)

User requests complete deletion of all personal data.

**Timeline**: 30 days (includes grace period for account recovery)

**Process**:

1. **Verification**: Confirm identity (email + MFA)
2. **Grace Period**: 7-day window to cancel request
3. **Anonymization**: Remove PII from active data
4. **Deletion**: Purge from backups within 30 days
5. **Confirmation**: Email receipt of deletion

**Exceptions** (Data NOT Deleted):

- Legal obligation to retain (e.g., financial records)
- Ongoing dispute/claim (legal hold)
- Public health/scientific research (anonymized)
- Fraud prevention (hashed identifiers, no PII)

#### 5. RESTRICTION (Right to Restrict Processing)

User limits how data can be used while disputing accuracy or lawfulness.

**Effect**:

- Data marked as "restricted"
- Processing paused (except storage)
- User notified before restriction lifted

### DSR Request Flow

#### Create DSR

```bash
POST /v1/data-subject-requests
Content-Type: application/json

{
  "request_type": "ERASURE",
  "user_id": "user-abc123",
  "email": "user@example.com",
  "request_details": {
    "reason": "no_longer_using_service",
    "additional_notes": "Please delete all my data"
  },
  "verification_data": {
    "last_login": "2025-10-15",
    "account_creation_date": "2024-03-20"
  }
}

Response:
{
  "request_id": "dsr-123",
  "status": "PENDING_VERIFICATION",
  "estimated_completion_days": 30,
  "next_steps": "Identity verification required before processing"
}
```

#### Verify Identity

```bash
POST /v1/data-subject-requests/dsr-123/verify
Content-Type: application/json

{
  "verification_method": "email_mfa",
  "verification_proof": {
    "code": "123456",
    "timestamp": "2025-10-16T10:30:00Z"
  }
}

Response:
{
  "request_id": "dsr-123",
  "status": "VERIFIED",
  "verified_at": "2025-10-16T10:30:15Z",
  "next_steps": "Request processing will begin within 24 hours"
}
```

---

## API Reference

### Endpoints

```
GET    /v1/health                                    # Health check
POST   /v1/data-inventory                            # Register data
GET    /v1/data-inventory                            # List inventory
GET    /v1/data-inventory/:id                        # Get specific entry
GET    /v1/data-classifications                      # List classifications
POST   /v1/data-classifications/auto-classify        # Auto-classify data
POST   /v1/retention-policies                        # Create policy
GET    /v1/retention-policies                        # List policies
GET    /v1/retention-policies/:id/preview            # Preview policy impact
GET    /v1/regional-compliance                       # List regional rules
POST   /v1/regional-compliance/validate              # Validate compliance
POST   /v1/lifecycle-rules                           # Create lifecycle rule
POST   /v1/lifecycle-rules/:id/execute               # Execute rule
GET    /v1/audit-trails                              # Query audit logs
POST   /v1/compliance-reports                        # Generate report
GET    /v1/compliance-reports/:id                    # Get report
POST   /v1/data-subject-requests                     # Create DSR
POST   /v1/data-subject-requests/:id/verify          # Verify identity
```

### Authentication

All endpoints require service-to-service JWT authentication:

```
Authorization: Bearer <service-jwt-token>
```

**Token Claims**:

```json
{
  "iss": "auth-service",
  "sub": "conversation-service",
  "aud": "data-governance-service",
  "iat": 1697462400,
  "exp": 1697466000,
  "permissions": ["data:read", "data:write", "compliance:report"]
}
```

### Rate Limits

- **Standard Operations**: 1000 requests/minute
- **Report Generation**: 10 reports/minute
- **DSR Requests**: 100 requests/hour (anti-abuse)
- **Bulk Operations**: 10/minute

---

## Operational Procedures

### Daily Operations

#### 1. Monitor Compliance Dashboard

```bash
# Check overall compliance status
GET /v1/compliance-reports
```

**Key Metrics**:

- ✅ Data inventory compliance rate > 98%
- ✅ Zero expired data overdue > 7 days
- ✅ Regional compliance violations < 1%
- ✅ DSR completion rate > 95% within SLA

#### 2. Review Audit Logs

```bash
GET /v1/audit-trails?event_type=DATA_REGISTERED&limit=100
```

**Alert Conditions**:

- ⚠️ PII_HIGH data registered without encryption
- ⚠️ Data inventory entries without classification
- ⚠️ Regional compliance violations
- 🚨 Unauthorized access attempts

#### 3. Process DSRs

```bash
GET /v1/data-subject-requests?status=VERIFIED
```

**SLA Targets**:

- ACCESS requests: 30 days
- ERASURE requests: 30 days
- RECTIFICATION requests: 1 business day

### Weekly Operations

#### 1. Retention Policy Review

```bash
POST /v1/compliance-reports
{
  "report_type": "RETENTION_COMPLIANCE"
}
```

**Actions**:

- Review approaching expirations (next 30 days)
- Validate auto-deletion jobs
- Check for retention violations

#### 2. Regional Compliance Audit

```bash
POST /v1/compliance-reports
{
  "report_type": "REGIONAL_COMPLIANCE"
}
```

**Actions**:

- Address compliance violations
- Update data localization
- Review cross-border transfers

### Monthly Operations

#### 1. Comprehensive Data Inventory Audit

```bash
POST /v1/compliance-reports
{
  "report_type": "DATA_INVENTORY_SUMMARY"
}
```

**Actions**:

- Verify all services registered data
- Update classification if needed
- Review metadata for accuracy

#### 2. Lifecycle Management Review

```bash
POST /v1/compliance-reports
{
  "report_type": "LIFECYCLE_MANAGEMENT"
}
```

**Actions**:

- Optimize lifecycle rules
- Review anonymization jobs
- Verify deletion completeness

### Quarterly Operations

#### 1. Privacy Impact Assessment (PIA)

**Scope**: New features, services, or data types

**Process**:

1. Identify new data flows
2. Assess privacy risks
3. Implement mitigations
4. Document findings
5. Update data inventory

#### 2. Third-Party Audit Preparation

**Deliverables**:

- Data inventory export
- Compliance reports (all types)
- DSR fulfillment metrics
- Audit trail exports
- Security attestations

---

## Change Log

### Version 1.0.0 (October 16, 2025)

**Initial Release**:

- Comprehensive data classification taxonomy (6 levels)
- Multi-region compliance support (EU/GDPR, US/CCPA, CA/PIPEDA)
- Automated retention policies and lifecycle management
- DSR request processing (5 request types)
- Compliance reporting (4 report types)
- Audit trail infrastructure

---

## Related Documentation

- **[DSR Policy](./DSR_POLICY.md)**: Data Subject Rights implementation details
- **[Audit Logging](./AUDIT_LOGGING.md)**: Comprehensive audit trail
  specifications
- **[Threat Model](./THREAT_MODEL.md)**: Security risk analysis
- **[Privacy Policy](#)**: User-facing privacy commitments
- **[Security API Reference](./security-api-reference.md)**: Authentication and
  authorization

---

## Support & Escalation

**Data Governance Team**: data-governance@lilith.platform **Privacy Officer**:
privacy@lilith.platform **Security Incidents**: security@lilith.platform
**On-Call**: PagerDuty escalation policy "Data-Governance-Oncall"

**Office Hours**: Monday-Friday 9:00-17:00 UTC **Emergency Response**: 24/7 via
PagerDuty
