Eve library

Audit Logging Specification

Guide

Revision 1

On this page

Document Version: 1.0
Last Updated: 2025-10-03
Owners: Platform Security Guild & Privacy Office

Overview#

Every Lilith service emits tamper-evident audit events through the shared AuditLogger. Events capture who did what, from where, and whether the operation succeeded. This specification documents the canonical event schema, event taxonomy, redaction rules, and the retention policy enforced across storage tiers.

Event Schema#

The canonical JSON Schema describing an audit event lives at docs/schemas/audit-log.schema.json. Tooling and automated validation suites compile against this schema to guarantee compatibility. Key guarantees include:

  • ISO 8601 UTC timestamps (timestamp).
  • Immutable identifiers (event_id, ULID compatible UUIDs).
  • Strict categorisation (event_category, event_type, severity).
  • Privacy-safe user attribution (user_id is HMAC hashed when privacy mode is enabled).
  • Redaction of sensitive fields at ingest (passwords, tokens, payment artefacts).
  • Deterministic retention metadata (metadata.retention_category).

Sample Event (Post Sanitisation)#

json
{
  "timestamp": "2025-10-03T04:12:19.421Z",
  "event_id": "01J9R6KHW7D2GVQ1C6TZX8X0H0",
  "event_type": "security.attack.detected",
  "event_category": "SECURITY",
  "severity": "CRITICAL",
  "service": "bff",
  "version": "2.5.1",
  "environment": "production",
  "correlation_id": "corr-742d",
  "session_id": "sess-b17c",
  "user_id": "a0b1c2d3e4f5a6b7",
  "source": {
    "ip_address": "198.51.100.42",
    "user_agent": "LilithClient/1.9.0",
    "location": "us-west-2"
  },
  "resource": {
    "type": "firewall-rule",
    "id": "waf-9be3",
    "name": "edge-shield"
  },
  "outcome": {
    "success": true,
    "status_code": 200,
    "message": "Request blocked by WAF"
  },
  "details": {
    "attack_vector": "sql_injection",
    "request_id": "req-31cb",
    "sensitive_field": "[REDACTED]"
  },
  "metadata": {
    "compliance_tags": ["soc2", "gdpr", "iso27001"],
    "retention_category": "long_term",
    "data_classification": "restricted",
    "service_region": "eu-central-1"
  }
}

Event Taxonomy#

Category Typical Event Types Notes
SECURITY security.attack.detected, security.policy.updated, security.alert.acknowledged High value signals, feed SIEM & SOAR workflows.
AUTH auth.login.success, auth.login.failed, auth.mfa.challenge Authentication lifecycle with hashed user identifiers.
AUTHZ authz.permission.elevated, authz.role.assigned, authz.policy.changed Access control and RBAC policy events.
DATA data.export.requested, data.delete, data.retention.exception Data governance operations subject to GDPR/CCPA review.
API api.request, api.rate-limit.triggered, api.key.rotated High-volume events retained on the standard tier.
ADMIN admin.config.changed, admin.user.deactivated, admin.sudo.session Administrative actions require long-term retention.
SYSTEM system.startup, system.shutdown, system.health.degraded Operational visibility for on-call runbooks.

Severity Ladder#

  • CRITICAL: Potential or confirmed breach, regulatory exposure, immediate SIRT escalation.
  • HIGH: Security-impacting or privileged administrative activity.
  • MEDIUM: User-visible failures, policy enforcement actions.
  • INFO: Routine activity tracked for non-repudiation.
  • DEBUG: Optional diagnostic context (never shipped to immutable stores).

Retention Policy#

Category Description Duration Storage Tier
long_term Critical security and privileged administrative events 7 years Glacier / Archive object storage
medium_term Data governance and authorization changes 2 years Coldline bucket with quarterly integrity checks
auth_term Authentication lifecycle and session management 1 year Warm tier (standard IA)
standard API access, operational telemetry, low-risk events 90 days Hot tier (Elasticsearch + S3 Standard)

Retention enforcement is coordinated by the data lifecycle service and verified nightly. Records are cryptographically shredded once outside the mandated window. For GDPR compliance, user identifiers are HMAC hashed after two years regardless of category.

Storage & Processing Pipeline#

  1. Ingest: Services enqueue audit events via the AuditLogger, receiving an immutable event ID.
  2. Validation: Events are validated against the JSON Schema and augmented with compliance metadata.
  3. Transport: Events stream through Kafka into the security lake (Elasticsearch + S3 data lake partitioned by date).
  4. Archival: Category-specific lifecycle policies copy objects into Glacier/Archive tiers according to the table above.
  5. Monitoring: Real-time subscribers listen to the audit_event emitter for anomaly detection and alerting.
  6. Tamper Evidence: Daily Merkle trees over each partition are notarised via Sigstore and stored in the compliance vault.

Privacy & Redaction Controls#

  • PII fields (password, token, api_key, auth_token, refresh_token, sensitive_field, payment artefacts) are replaced with [REDACTED] at ingest.
  • Optional user hashing uses HMAC-SHA256 with a Vault-managed key (AUDIT_USER_HASH_KEY).
  • details payloads are truncated to 1 000 characters to prevent accidental blob storage.
  • Consent and DSAR tooling reference the audit store to provide access reports with automatic redaction.

Operational Runbook#

  • Batch Flush: Buffers flush every 5 seconds or when 100 events are queued (batch_size configurable per service).
  • Backpressure: Failures emit audit_error events which are monitored by the on-call automation.
  • Shutdown: Services call await logger.shutdown() during graceful exit to guarantee flush and close file handles.
  • Disaster Recovery: Archived partitions mirror to the secondary region weekly; restoration procedures are documented in docs/runbooks/security/audit-log-recovery.md.

Ownership & Review Cadence#

  • Quarterly audit by the Platform Security Guild with Privacy Office sign-off.
  • Schema changes require version bump and backward compatibility review.
  • Retention table is synchronised with legal/compliance policy updates and reflected in automated Playwright compliance tests (tests/security.audit-logging.spec.ts).