Eve library

OSHUN V1 Assistant Quality and Disclosure Compliance Dashboards

Guide

Revision 1

On this page

Status: Canonical Date: 2026-04-26 Owners: OSHUN Assistant Product, OSHUN Trust & Safety, OSHUN Compliance, OSHUN Product Analytics Tracks: V1-OBS-008 from OSHUN_V1_TODOS.md Spec source: libs/oshun/analytics/src/dashboards-assistant.ts

This document is the canonical, human-readable companion to the machine-readable V1 assistant dashboard manifest in @oshun/analytics.

The machine-readable manifest is the single source of truth.


1. Scope#

This manifest defines the two V1 assistant dashboards mandated by V1-OBS-008:

  • oshun-v1-assistant-quality — assistant quality across turn success and latency, customer feedback, grounding application and fallback, low-confidence flags, safety refusals, persona-policy violations, and continuity-stage completion
  • oshun-v1-assistant-disclosure-compliance — ADR-0060 disclosure indicator visibility, customer dismissal/acknowledgement, the launch-blocking compliance-check failure metric, plus memory-scope consent transitions and memory-access denials

Pairs with:

  • the assistant taxonomy (assistant-taxonomy.ts, V1-OBS-003) — the source of every event a metric reads from
  • the customer KPI dashboard manifest (dashboards-customer-kpi.ts, V1-OBS-006) — the shared metric/panel/visualization shape

2. Manifest shape#

Assistant dashboards reuse the metric/panel shape from dashboards-customer-kpi.ts:

  • id: oshun-v1-assistant-quality | oshun-v1-assistant-disclosure-compliance
  • owner: assistant_product | trust_safety | compliance | product_analytics

3. Quality dashboard#

Metrics:

  • session and turn volume (oshun.assistant.session_started, oshun.assistant.turn_completed, oshun.assistant.turn_failed)
  • turn success rate, turn failure class distribution, turn latency p50/p90
  • feedback thumbs-up rate and signal distribution
  • grounding applied rate, grounding fallback by reason, low-confidence count
  • refusals by class, unsafe input/output by severity
  • persona policy violations by class
  • continuity handback rate, continuity stage duration p50/p90
  • citation follow rate, citation flag class distribution

Panels: session volume, turn success/failure/latency, feedback, grounding, low-confidence, refusals, safety, persona policy, continuity, citations.


4. Disclosure compliance dashboard#

Metrics:

  • assistant_disclosure.shown_by_indicator — oshun.disclosure.shown by indicator (one of the eight ADR-0060 indicators: ai_assistant, memory_off, memory_session, memory_profile, synthetic_voice, synthetic_avatar, synthetic_media, human_review_present)
  • assistant_disclosure.shown_by_placement — by render placement
  • assistant_disclosure.dismissed_rate — oshun.disclosure.dismissed / oshun.disclosure.shown
  • assistant_disclosure.acknowledged_rate — oshun.disclosure.acknowledged / oshun.disclosure.shown
  • assistant_disclosure.compliance_failure_count — oshun.disclosure.compliance_check_failed by missing_indicator (the launch-blocking signal)
  • assistant_disclosure.compliance_failure_severity — by severity
  • assistant_disclosure.memory_scope_consent_changes — oshun.memory_scope.consent_changed by state
  • assistant_disclosure.memory_access_denied_rate — oshun.memory.access_denied by denied_boundary

Panels: indicator volume, placement mix, dismissal rate, acknowledgement rate, compliance failures, severity mix, memory consent transitions, memory access denials.


5. KPI parity and integrity#

Tests in dashboards-assistant.test.ts enforce:

  • the two-dashboard manifest is published
  • every metric event reference resolves to the assistant taxonomy
  • every panel references only same-dashboard metrics
  • the quality dashboard exposes the core quality metrics (session, success, latency, feedback, grounding, low-confidence, refusals, persona, continuity, citations)
  • the disclosure dashboard surfaces every ADR-0060 indicator and tracks the launch-blocking compliance failure metric
  • the disclosure dashboard tracks the memory-scope consent transitions

6. Governance#

6.1 Amendments#

Adding or changing an assistant dashboard, metric, or panel requires:

  1. updating dashboards-assistant.ts
  2. updating tests (lock-style tests fail closed if metrics drift away from the assistant taxonomy)
  3. updating this document if the change is structurally significant
  4. signoff from the listed owner plus platform analytics

6.2 Versioning#

Same rules as the sibling dashboard manifests.


7. Out-of-scope#

This manifest does not implement an actual rendering pipeline; downstream analytics platforms compile the manifest into concrete dashboards.


8. Change log#

  • 2026-04-26 — Initial canonical V1 assistant dashboard manifest published alongside libs/oshun/analytics/src/dashboards-assistant.ts (V1-OBS-008).