Disciplines · Decisions (ADRs)

ADR-0087: Eve trace and correlation continuity

Eve crosses synchronous calls, browser callbacks, stored confirmations, event-sourced work, leases, agent reports, watchers, channels, and artifact verification.

3sections3 minread
Markdown source
On this page
  • Status: Accepted
  • Date: 2026-09-14
  • Decision owner: SRE Lead
  • Independent verifier: Charter QA

Context#

Eve crosses synchronous calls, browser callbacks, stored confirmations, event-sourced work, leases, agent reports, watchers, channels, and artifact verification. Request-only logging cannot join those boundaries, while a generic telemetry attribute bag would create an easy path for prompts, tool payloads, or secrets to enter default observability data. Task 13.2 requires one causal identity contract before Task 13.3 adds standardized execution controls and before later tasks export or operate telemetry.

Decision#

Adopt @oshun/tracing's eve-trace-context.v1 contract as the only Eve propagation format.

  1. The carrier follows the W3C Trace Context Recommendation dated 2021-11-23. Semantic naming is pinned to the repository-resolved @opentelemetry/semantic-conventions@1.41.1; changing either pin requires a versioned decision and updated compatibility evidence.
  2. One server-minted correlation UUID and one W3C trace ID identify the whole causal journey. Each logical stage receives a fresh span ID. Synchronous work is a child; deferred work is represented as a link through originSpanId instead of pretending that it remains on a live call stack.
  3. The total graph is: invocation → session → turn → router → model → tool, with explicit tool branches to client-tool, MCP, A2A, and confirmation; confirmed work continues through ledger → queue → lease → agent, which branches to watcher → channel and artifact-verification.
  4. A valid version-00 external traceparent may be continued. Version ff, unsupported versions, and malformed values are rejected. An external x-correlation-id is never trusted or echoed; only explicitly trusted internal ingress may retain it. Malformed, wrong-stage, integrity-tampered, active-parent-substituted, unknown-field, and undeclared-edge carriers fail closed.
  5. The serializable envelope is closed and versioned. It can contain only version, stage, traceparent, correlationId, originSpanId, and an HMAC integrity tag. Production requires an OSHUN_EVE_TRACE_INTEGRITY_KEY of at least 32 bytes, shared by participating processes. External tracestate is deliberately discarded because its vendor value could be abused as a payload-smuggling field. The envelope has no generic attributes field. Prompts, messages, tool arguments/results, member or tenant identifiers, secrets, sensitive payloads, and error text are not representable in default propagation.
  6. Browser callbacks echo the opaque server-issued envelope. Confirmed work persists the derived queue carrier on the append-only work-item creation event, so a later lease resumes it without relying on process memory or a caller-supplied replacement; leases and agent reports then persist their bounded carriers. MCP, A2A, confirmation, watcher, channel, and artifact adapters validate the expected source stage before deriving the next one. Provider or channel integrations that do not participate in W3C propagation keep the local boundary context without placing identifiers into user content.
  7. An integrity-valid envelope proves only that a participating service minted the continuity metadata. It is not an authorization credential, does not bind a principal or tenant, and does not independently prevent replay. Authentication, authorization, tenant isolation, idempotency, and fencing remain the responsibility of their existing boundary contracts. When an in-process parent exists, an explicit carrier must exactly match it.
  8. verify-trace-context-propagation.ts owns the source binding inventory and a synthetic totality proof. Focused integration tests own each real adapter; the queue test crosses the actual confirmation bridge, append-only creation event, delayed lease, report, and persisted artifact-verification event. Negative controls include missing, malformed, integrity-tampered, severed, substituted, and payload-smuggling cases.

Consequences#

  • Support can join one Eve journey across synchronous and deferred boundaries without searching message content.
  • A missing stage or severed identity is a verification failure, not a newly minted trace that appears complete.
  • Default carriers stay low-cardinality and payload-free; richer telemetry must be explicitly classified and governed in later work.
  • Rotating the production integrity key invalidates outstanding deferred carriers. Operations must coordinate rotation with draining or expiring pending confirmations, leases, watcher outbox rows, and callbacks; silently falling back to a process-local key is forbidden in production.
  • This decision proves propagation logic and participating repository adapters. It does not claim a deployed collector, exported production spans, dashboard coverage, retention policy, or current SLO attainment. Those remain Tasks 13.3–13.7 and 14.3 obligations.
  • Eve does not currently expose live MCP or A2A routes from the BFF. Their shared platform adapters are propagation-ready and covered, but this task does not fabricate runtime traffic through endpoints that do not exist.