Document Version: 1.0
Last Updated: 2025-10-03
Owners: Platform Security Guild & Privacy Office
Overview#
Every Lilith service emits tamper-evident audit events through the shared
AuditLogger. Events capture who did what, from where, and whether the
operation succeeded. This specification documents the canonical event schema,
event taxonomy, redaction rules, and the retention policy enforced across
storage tiers.
Event Schema#
The canonical JSON Schema describing an audit event lives at
docs/schemas/audit-log.schema.json.
Tooling and automated validation suites compile against this schema to guarantee
compatibility. Key guarantees include:
- ISO 8601 UTC timestamps (
timestamp). - Immutable identifiers (
event_id, ULID compatible UUIDs). - Strict categorisation (
event_category,event_type,severity). - Privacy-safe user attribution (
user_idis HMAC hashed when privacy mode is enabled). - Redaction of sensitive fields at ingest (passwords, tokens, payment artefacts).
- Deterministic retention metadata (
metadata.retention_category).
Sample Event (Post Sanitisation)#
{
"timestamp": "2025-10-03T04:12:19.421Z",
"event_id": "01J9R6KHW7D2GVQ1C6TZX8X0H0",
"event_type": "security.attack.detected",
"event_category": "SECURITY",
"severity": "CRITICAL",
"service": "bff",
"version": "2.5.1",
"environment": "production",
"correlation_id": "corr-742d",
"session_id": "sess-b17c",
"user_id": "a0b1c2d3e4f5a6b7",
"source": {
"ip_address": "198.51.100.42",
"user_agent": "LilithClient/1.9.0",
"location": "us-west-2"
},
"resource": {
"type": "firewall-rule",
"id": "waf-9be3",
"name": "edge-shield"
},
"outcome": {
"success": true,
"status_code": 200,
"message": "Request blocked by WAF"
},
"details": {
"attack_vector": "sql_injection",
"request_id": "req-31cb",
"sensitive_field": "[REDACTED]"
},
"metadata": {
"compliance_tags": ["soc2", "gdpr", "iso27001"],
"retention_category": "long_term",
"data_classification": "restricted",
"service_region": "eu-central-1"
}
}
Event Taxonomy#
| Category | Typical Event Types | Notes |
|---|---|---|
SECURITY |
security.attack.detected, security.policy.updated, security.alert.acknowledged |
High value signals, feed SIEM & SOAR workflows. |
AUTH |
auth.login.success, auth.login.failed, auth.mfa.challenge |
Authentication lifecycle with hashed user identifiers. |
AUTHZ |
authz.permission.elevated, authz.role.assigned, authz.policy.changed |
Access control and RBAC policy events. |
DATA |
data.export.requested, data.delete, data.retention.exception |
Data governance operations subject to GDPR/CCPA review. |
API |
api.request, api.rate-limit.triggered, api.key.rotated |
High-volume events retained on the standard tier. |
ADMIN |
admin.config.changed, admin.user.deactivated, admin.sudo.session |
Administrative actions require long-term retention. |
SYSTEM |
system.startup, system.shutdown, system.health.degraded |
Operational visibility for on-call runbooks. |
Severity Ladder#
CRITICAL: Potential or confirmed breach, regulatory exposure, immediate SIRT escalation.HIGH: Security-impacting or privileged administrative activity.MEDIUM: User-visible failures, policy enforcement actions.INFO: Routine activity tracked for non-repudiation.DEBUG: Optional diagnostic context (never shipped to immutable stores).
Retention Policy#
| Category | Description | Duration | Storage Tier |
|---|---|---|---|
| long_term | Critical security and privileged administrative events | 7 years | Glacier / Archive object storage |
| medium_term | Data governance and authorization changes | 2 years | Coldline bucket with quarterly integrity checks |
| auth_term | Authentication lifecycle and session management | 1 year | Warm tier (standard IA) |
| standard | API access, operational telemetry, low-risk events | 90 days | Hot tier (Elasticsearch + S3 Standard) |
Retention enforcement is coordinated by the data lifecycle service and verified nightly. Records are cryptographically shredded once outside the mandated window. For GDPR compliance, user identifiers are HMAC hashed after two years regardless of category.
Storage & Processing Pipeline#
- Ingest: Services enqueue audit events via the
AuditLogger, receiving an immutable event ID. - Validation: Events are validated against the JSON Schema and augmented with compliance metadata.
- Transport: Events stream through Kafka into the security lake (Elasticsearch + S3 data lake partitioned by date).
- Archival: Category-specific lifecycle policies copy objects into Glacier/Archive tiers according to the table above.
- Monitoring: Real-time subscribers listen to the
audit_eventemitter for anomaly detection and alerting. - Tamper Evidence: Daily Merkle trees over each partition are notarised via Sigstore and stored in the compliance vault.
Privacy & Redaction Controls#
- PII fields (
password,token,api_key,auth_token,refresh_token,sensitive_field, payment artefacts) are replaced with[REDACTED]at ingest. - Optional user hashing uses HMAC-SHA256 with a Vault-managed key
(
AUDIT_USER_HASH_KEY). detailspayloads are truncated to 1 000 characters to prevent accidental blob storage.- Consent and DSAR tooling reference the audit store to provide access reports with automatic redaction.
Operational Runbook#
- Batch Flush: Buffers flush every 5 seconds or when 100 events are queued
(
batch_sizeconfigurable per service). - Backpressure: Failures emit
audit_errorevents which are monitored by the on-call automation. - Shutdown: Services call
await logger.shutdown()during graceful exit to guarantee flush and close file handles. - Disaster Recovery: Archived partitions mirror to the secondary region
weekly; restoration procedures are documented in
docs/runbooks/security/audit-log-recovery.md.
Ownership & Review Cadence#
- Quarterly audit by the Platform Security Guild with Privacy Office sign-off.
- Schema changes require version bump and backward compatibility review.
- Retention table is synchronised with legal/compliance policy updates and
reflected in automated Playwright compliance tests
(
tests/security.audit-logging.spec.ts).