Status: Canonical Date: 2026-04-26 Owners: OSHUN Assistant Product, OSHUN Trust
& Safety, OSHUN Compliance, OSHUN Product Analytics Tracks: V1-OBS-008 from
OSHUN_V1_TODOS.md Spec source:
libs/oshun/analytics/src/dashboards-assistant.ts
This document is the canonical, human-readable companion to the machine-readable
V1 assistant dashboard manifest in @oshun/analytics.
The machine-readable manifest is the single source of truth.
1. Scope#
This manifest defines the two V1 assistant dashboards mandated by V1-OBS-008:
oshun-v1-assistant-quality— assistant quality across turn success and latency, customer feedback, grounding application and fallback, low-confidence flags, safety refusals, persona-policy violations, and continuity-stage completionoshun-v1-assistant-disclosure-compliance— ADR-0060 disclosure indicator visibility, customer dismissal/acknowledgement, the launch-blocking compliance-check failure metric, plus memory-scope consent transitions and memory-access denials
Pairs with:
- the assistant taxonomy (
assistant-taxonomy.ts, V1-OBS-003) — the source of every event a metric reads from - the customer KPI dashboard manifest (
dashboards-customer-kpi.ts, V1-OBS-006) — the shared metric/panel/visualization shape
2. Manifest shape#
Assistant dashboards reuse the metric/panel shape from
dashboards-customer-kpi.ts:
id:oshun-v1-assistant-quality|oshun-v1-assistant-disclosure-complianceowner:assistant_product|trust_safety|compliance|product_analytics
3. Quality dashboard#
Metrics:
- session and turn volume (
oshun.assistant.session_started,oshun.assistant.turn_completed,oshun.assistant.turn_failed) - turn success rate, turn failure class distribution, turn latency p50/p90
- feedback thumbs-up rate and signal distribution
- grounding applied rate, grounding fallback by reason, low-confidence count
- refusals by class, unsafe input/output by severity
- persona policy violations by class
- continuity handback rate, continuity stage duration p50/p90
- citation follow rate, citation flag class distribution
Panels: session volume, turn success/failure/latency, feedback, grounding, low-confidence, refusals, safety, persona policy, continuity, citations.
4. Disclosure compliance dashboard#
Metrics:
assistant_disclosure.shown_by_indicator—oshun.disclosure.shownby indicator (one of the eight ADR-0060 indicators:ai_assistant,memory_off,memory_session,memory_profile,synthetic_voice,synthetic_avatar,synthetic_media,human_review_present)assistant_disclosure.shown_by_placement— by render placementassistant_disclosure.dismissed_rate—oshun.disclosure.dismissed / oshun.disclosure.shownassistant_disclosure.acknowledged_rate—oshun.disclosure.acknowledged / oshun.disclosure.shownassistant_disclosure.compliance_failure_count—oshun.disclosure.compliance_check_failedbymissing_indicator(the launch-blocking signal)assistant_disclosure.compliance_failure_severity— by severityassistant_disclosure.memory_scope_consent_changes—oshun.memory_scope.consent_changedby stateassistant_disclosure.memory_access_denied_rate—oshun.memory.access_deniedbydenied_boundary
Panels: indicator volume, placement mix, dismissal rate, acknowledgement rate, compliance failures, severity mix, memory consent transitions, memory access denials.
5. KPI parity and integrity#
Tests in dashboards-assistant.test.ts enforce:
- the two-dashboard manifest is published
- every metric event reference resolves to the assistant taxonomy
- every panel references only same-dashboard metrics
- the quality dashboard exposes the core quality metrics (session, success, latency, feedback, grounding, low-confidence, refusals, persona, continuity, citations)
- the disclosure dashboard surfaces every ADR-0060 indicator and tracks the launch-blocking compliance failure metric
- the disclosure dashboard tracks the memory-scope consent transitions
6. Governance#
6.1 Amendments#
Adding or changing an assistant dashboard, metric, or panel requires:
- updating
dashboards-assistant.ts - updating tests (lock-style tests fail closed if metrics drift away from the assistant taxonomy)
- updating this document if the change is structurally significant
- signoff from the listed
ownerplus platform analytics
6.2 Versioning#
Same rules as the sibling dashboard manifests.
7. Out-of-scope#
This manifest does not implement an actual rendering pipeline; downstream analytics platforms compile the manifest into concrete dashboards.
8. Change log#
- 2026-04-26 — Initial canonical V1 assistant dashboard manifest published
alongside
libs/oshun/analytics/src/dashboards-assistant.ts(V1-OBS-008).