Disciplines · Decisions (ADRs)

ADR-0040: OSHUN Studio Metrics and Analytics Instrumentation

Studio now exposes many advanced governance surfaces, but metrics and analytics instrumentation behavior is still distributed and partially inconsistent across flows.

Accepted · 2026-02-23
10sections4 minread
Markdown source
On this page

Status: Accepted
Date: 2026-02-23
Authors: OSHUN Studio Architecture, OSHUN Web + BFF Engineering, OSHUN Data/Analytics Engineering
Reviewers: Domain Leads (Yemaya, Isis, Hathor, Aja, Bellona), Project Obsidian Program Design

Context and Problem Statement#

Studio now exposes many advanced governance surfaces, but metrics and analytics instrumentation behavior is still distributed and partially inconsistent across flows. Without a canonical metrics/instrumentation model, telemetry quality and cross-domain comparability degrade as the platform scales.

Without a unified metrics and analytics instrumentation architecture, OSHUN risks:

  • inconsistent KPI definitions across Studio surfaces
  • incomplete lineage between workflow state and emitted analytics events
  • slower incident/debug cycles due to weak telemetry-attribution context
  • governance gaps when audit and analytics contracts diverge

Decision Drivers#

  • Comparability: one metrics model across all Studio domains and workflows.
  • Determinism: instrumentation semantics must be stable and explicit.
  • Traceability: events must preserve actor/workspace/workflow attribution.
  • Governance: analytics and audit evidence must stay contract-aligned.
  • Operability: metrics must support alerting, diagnostics, and release gates.

Considered Options#

Option 1: Per-Surface Instrumentation Patterns#

Allow each Studio surface to define local metrics and event semantics.

Pros:

  • Fast local implementation for individual feature teams.

Cons:

  • Fragmented KPI taxonomy and difficult cross-domain analytics governance.
  • Increased schema drift and maintenance overhead.

Option 2: Backend-Only Aggregated Metrics#

Capture only server-side aggregate metrics and minimize UI instrumentation.

Pros:

  • Reduced frontend implementation complexity.

Cons:

  • Lower attribution fidelity for user-driven flows and policy outcomes.
  • Harder UX-quality diagnostics and remediation guidance.

Option 3: Canonical Studio Metrics and Instrumentation Contracts (Chosen)#

Adopt shared flow-span, audit-event, metric-definition, and KPI-composition contracts with explicit Studio attribution dimensions.

Pros:

  • Consistent, queryable metrics across all Studio workspaces.
  • Strong alignment between UX flows, telemetry, and governance evidence.
  • Better diagnostics and faster regression detection.

Cons:

  • Requires coordinated schema governance and contract version discipline.

Decision Outcome#

Chosen option: Option 3.

Metrics and analytics instrumentation requirements:

  1. Canonical metric-definition contract for counters, rates, and latencies.
  2. Flow-span contract for requested/completed lifecycle instrumentation.
  3. Audit-to-analytics linkage contract for policy and governance outcomes.
  4. KPI composition contract for success, reliability, latency, and adoption.
  5. Attribution and cardinality contract for actor/workspace/domain context.
  6. Release validity contract for telemetry completeness and schema integrity.

Normative Rules#

Canonical Metric Definition Contract#

  • Every metric must include metricId, metricType, unit, owner, aggregationWindow, and alertThreshold.
  • Metric semantics must be versioned and documented.
  • Undocumented metrics are not permitted in production dashboards.

Flow-Span Contract#

  • Every actionable flow must emit flow_requested and flow_completed events.
  • Completed events must include success, latencyMs, and error details when applicable.
  • Flow spans must be idempotent for duplicate completion protection.

Audit-to-Analytics Linkage Contract#

  • Policy/audit outcomes (allowed, denied, error, success) must emit corresponding analytics events.
  • Analytics payloads must include audit-context dimensions for attribution.
  • Missing linkage between audit and analytics events is a release-blocking defect.

KPI Composition Contract#

  • Studio KPI set must include success rate, p95 latency, policy-denial rate, recovery rate, and governance closure completeness.
  • KPI definitions must be reusable across Yemaya/Isis/Hathor/Aja/Bellona.
  • KPI computation logic must be deterministic and documented.

Attribution and Cardinality Contract#

  • Required dimensions: actorId, role, workspaceId, selectedEntityId, sourceDomain, and releaseChannel (or equivalent).
  • High-cardinality free text must be constrained and justified.
  • PII fields are disallowed in analytics payloads.

Release Validity Contract#

A metrics/instrumentation release is valid only when all are true:

  • metric-definition schema validation passes
  • flow-span and audit-linkage tests pass
  • KPI composition and threshold checks pass
  • telemetry completeness and null-rate checks pass
  • rollback and ownership handoff documentation exists

Architecture Implications#

  • Studio workspaces share one instrumentation vocabulary and payload structure.
  • BFF and web layers use compatible metric/event schemas.
  • Shared analytics library enforces typed payload contracts.
  • Observability layers consume deterministic KPI-ready telemetry streams.

Acceptance Criteria (OST-00217)#

OST-00217 is complete only when:

  1. ADR exists at docs/adr/ADR-0046-oshun-studio-metrics-and-analytics-instrumentation.md.
  2. ADR defines options, trade-offs, and selected strategy.
  3. ADR defines metric definitions, flow spans, audit linkage, KPI composition, attribution/cardinality, and release-validity contracts.
  4. ADR aligns with ADR-0007 through ADR-0039, especially ADR-0012, ADR-0038, and ADR-0039.
  5. ADR aligns with docs/releases/v1/design/ux-principles.md, libs/oshun/analytics, and libs/oshun/domain-registry.
  6. ADR explicitly covers Yemaya, Isis, Hathor, Aja, Bellona, and Project Obsidian.

Consequences#

Positive#

  • More reliable and comparable Studio metrics across all workflow surfaces.
  • Stronger analytics governance and faster telemetry-driven diagnostics.
  • Better release confidence via explicit instrumentation gates.

Negative#

  • Additional upfront implementation and schema-management overhead.
  • Requires sustained cross-team ownership of KPI definitions.
  • docs/adr/ADR-0013-oshun-shell-architecture-and-domain-adapters.md
  • docs/adr/ADR-0018-analytics-taxonomy-and-event-naming.md
  • docs/adr/ADR-0044-oshun-studio-background-jobs-and-progress-ux.md
  • docs/adr/ADR-0045-oshun-studio-resilience-and-error-ux.md

References#

  • libs/oshun/analytics/src/types.ts
  • libs/oshun/domain-registry/src/index.ts
  • docs/releases/v1/design/ux-principles.md
  • docs/domains/yemaya/features.md
  • docs/domains/isis/features.md
  • docs/domains/hathor/features.md
  • docs/domains/aja/features.md
  • docs/domains/bellona/features.md