# Audit Logging Specification

**Document Version**: 1.0  
**Last Updated**: 2025-10-03  
**Owners**: Platform Security Guild & Privacy Office

## Overview

Every Lilith service emits tamper-evident audit events through the shared
`AuditLogger`. Events capture who did what, from where, and whether the
operation succeeded. This specification documents the canonical event schema,
event taxonomy, redaction rules, and the retention policy enforced across
storage tiers.

## Event Schema

The canonical JSON Schema describing an audit event lives at
[`docs/schemas/audit-log.schema.json`](../schemas/audit-log.schema.json).
Tooling and automated validation suites compile against this schema to guarantee
compatibility. Key guarantees include:

- ISO 8601 UTC timestamps (`timestamp`).
- Immutable identifiers (`event_id`, ULID compatible UUIDs).
- Strict categorisation (`event_category`, `event_type`, `severity`).
- Privacy-safe user attribution (`user_id` is HMAC hashed when privacy mode is
  enabled).
- Redaction of sensitive fields at ingest (passwords, tokens, payment
  artefacts).
- Deterministic retention metadata (`metadata.retention_category`).

### Sample Event (Post Sanitisation)

```json
{
  "timestamp": "2025-10-03T04:12:19.421Z",
  "event_id": "01J9R6KHW7D2GVQ1C6TZX8X0H0",
  "event_type": "security.attack.detected",
  "event_category": "SECURITY",
  "severity": "CRITICAL",
  "service": "bff",
  "version": "2.5.1",
  "environment": "production",
  "correlation_id": "corr-742d",
  "session_id": "sess-b17c",
  "user_id": "a0b1c2d3e4f5a6b7",
  "source": {
    "ip_address": "198.51.100.42",
    "user_agent": "LilithClient/1.9.0",
    "location": "us-west-2"
  },
  "resource": {
    "type": "firewall-rule",
    "id": "waf-9be3",
    "name": "edge-shield"
  },
  "outcome": {
    "success": true,
    "status_code": 200,
    "message": "Request blocked by WAF"
  },
  "details": {
    "attack_vector": "sql_injection",
    "request_id": "req-31cb",
    "sensitive_field": "[REDACTED]"
  },
  "metadata": {
    "compliance_tags": ["soc2", "gdpr", "iso27001"],
    "retention_category": "long_term",
    "data_classification": "restricted",
    "service_region": "eu-central-1"
  }
}
```

## Event Taxonomy

| Category   | Typical Event Types                                                                  | Notes                                                   |
| ---------- | ------------------------------------------------------------------------------------ | ------------------------------------------------------- |
| `SECURITY` | `security.attack.detected`, `security.policy.updated`, `security.alert.acknowledged` | High value signals, feed SIEM & SOAR workflows.         |
| `AUTH`     | `auth.login.success`, `auth.login.failed`, `auth.mfa.challenge`                      | Authentication lifecycle with hashed user identifiers.  |
| `AUTHZ`    | `authz.permission.elevated`, `authz.role.assigned`, `authz.policy.changed`           | Access control and RBAC policy events.                  |
| `DATA`     | `data.export.requested`, `data.delete`, `data.retention.exception`                   | Data governance operations subject to GDPR/CCPA review. |
| `API`      | `api.request`, `api.rate-limit.triggered`, `api.key.rotated`                         | High-volume events retained on the standard tier.       |
| `ADMIN`    | `admin.config.changed`, `admin.user.deactivated`, `admin.sudo.session`               | Administrative actions require long-term retention.     |
| `SYSTEM`   | `system.startup`, `system.shutdown`, `system.health.degraded`                        | Operational visibility for on-call runbooks.            |

## Severity Ladder

- `CRITICAL`: Potential or confirmed breach, regulatory exposure, immediate SIRT
  escalation.
- `HIGH`: Security-impacting or privileged administrative activity.
- `MEDIUM`: User-visible failures, policy enforcement actions.
- `INFO`: Routine activity tracked for non-repudiation.
- `DEBUG`: Optional diagnostic context (never shipped to immutable stores).

## Retention Policy

| Category    | Description                                            | Duration | Storage Tier                                    |
| ----------- | ------------------------------------------------------ | -------- | ----------------------------------------------- |
| long_term   | Critical security and privileged administrative events | 7 years  | Glacier / Archive object storage                |
| medium_term | Data governance and authorization changes              | 2 years  | Coldline bucket with quarterly integrity checks |
| auth_term   | Authentication lifecycle and session management        | 1 year   | Warm tier (standard IA)                         |
| standard    | API access, operational telemetry, low-risk events     | 90 days  | Hot tier (Elasticsearch + S3 Standard)          |

Retention enforcement is coordinated by the data lifecycle service and verified
nightly. Records are cryptographically shredded once outside the mandated
window. For GDPR compliance, user identifiers are HMAC hashed after two years
regardless of category.

## Storage & Processing Pipeline

1. **Ingest:** Services enqueue audit events via the `AuditLogger`, receiving an
   immutable event ID.
2. **Validation:** Events are validated against the JSON Schema and augmented
   with compliance metadata.
3. **Transport:** Events stream through Kafka into the security lake
   (Elasticsearch + S3 data lake partitioned by date).
4. **Archival:** Category-specific lifecycle policies copy objects into
   Glacier/Archive tiers according to the table above.
5. **Monitoring:** Real-time subscribers listen to the `audit_event` emitter for
   anomaly detection and alerting.
6. **Tamper Evidence:** Daily Merkle trees over each partition are notarised via
   Sigstore and stored in the compliance vault.

## Privacy & Redaction Controls

- PII fields (`password`, `token`, `api_key`, `auth_token`, `refresh_token`,
  `sensitive_field`, payment artefacts) are replaced with `[REDACTED]` at
  ingest.
- Optional user hashing uses HMAC-SHA256 with a Vault-managed key
  (`AUDIT_USER_HASH_KEY`).
- `details` payloads are truncated to 1 000 characters to prevent accidental
  blob storage.
- Consent and DSAR tooling reference the audit store to provide access reports
  with automatic redaction.

## Operational Runbook

- **Batch Flush:** Buffers flush every 5 seconds or when 100 events are queued
  (`batch_size` configurable per service).
- **Backpressure:** Failures emit `audit_error` events which are monitored by
  the on-call automation.
- **Shutdown:** Services call `await logger.shutdown()` during graceful exit to
  guarantee flush and close file handles.
- **Disaster Recovery:** Archived partitions mirror to the secondary region
  weekly; restoration procedures are documented in
  `docs/runbooks/security/audit-log-recovery.md`.

## Ownership & Review Cadence

- Quarterly audit by the Platform Security Guild with Privacy Office sign-off.
- Schema changes require version bump and backward compatibility review.
- Retention table is synchronised with legal/compliance policy updates and
  reflected in automated Playwright compliance tests
  (`tests/security.audit-logging.spec.ts`).
