V1 Web PWA · Surface walkthrough

Studio · Isis · User Feedback Loop

A per-surface walkthrough of the V1 Web PWA studio surface: layout, states, interactions, data, and cross-references.

walked
8sections3 minread
Markdown source
On this page

Context. surface studio · domain isis · route /studio/isis/user-feedback-loop · auth signed-in + studio entitlement (NOT AAA-gated) · source apps/oshun/web/src/app/studio/isis/user-feedback-loop/page.tsx

Last walked. 2026-07-21 focused Playwright runtime walk — authenticated empty/read/add/reload, duplicate submission, erased-subject, storage-unavailable, malformed-response, keyboard-control, and axe checks verified; live screen-reader, touch, offline, and telemetry-delivery checks still require a manual AT pass.

Purpose#

Admin lane console for diagnostic model-output evaluation. An authenticated operator submits an approved, de-identified internal review package (predicted model output vs. reference answer); the service scores it and aggregates mean BLEU and mean ROUGE-L across persisted samples. The metrics inform human review and do not automatically release, publish, or train a model. Customer feedback, personal data, and unapproved source material must not be entered. Reads and scores use /v1/admin/isis/feedback-eval; access and storage fail closed.

Entry points#

  • Sibling quick-action from /studio/isis/benchmarking
  • Studio index (/studio)
  • Direct URL / bookmark

Layout regions#

page.tsx mounts <ShellLayout active="studio"> (no breadcrumb panel) and three panels: the workspace, the Route Map, and sibling quick-actions.

  • Header: shell header
  • Workspace panel (data-isis-user-feedback-loop-workspace):
    • <h1> (WorkspaceHeading) "Isis User Feedback Loop Workspace"
    • Summary paragraph (data-feedback-summary)
    • Aggregate (data-feedback-aggregate) with data-feedback-mean-bleu, data-feedback-mean-rougel
    • Score form (data-feedback-form)
    • Scored samples (data-feedback-samples, data-feedback-sample=<id>, data-feedback-scores); empty state data-feedback-samples-empty
  • Route Map panel (data-isis-user-feedback-loop-route-map): <h2>Route Map</h2> over the route map (5 entries)
  • Sibling quick-actions panel: /studio/isis/benchmarking, /studio

States#

  • Loading — data-feedback-loading
  • Unauthorized — 401/403 admin-scope fail-closed; data-feedback-unauthorized
  • Storage unavailable — 503 persistence failure; data-feedback-error
  • Erased subject — 410 when the authenticated operator's permanent deletion fence rejects a new evaluation write; data-feedback-error
  • Malformed response — unexpected or internally inconsistent payload; data-feedback-error
  • Ready (form) — aggregate + score form render
  • Empty — no samples scored yet; data-feedback-samples-empty
  • Result — server-confirmed sample and aggregate in data-feedback-result (data-feedback-status) after scoring
  • Duplicate — 409 for an already-persisted sample id; form remains recoverable with a specific message

Interactions#

Score form (data-feedback-form)#

  • Sample id (data-feedback-sampleid, input)
  • Model output (predicted) (data-feedback-predicted, textarea)
  • Reference answer (expected) (data-feedback-expected, textarea)
  • Score (data-feedback-score, submit) — POST /v1/admin/isis/feedback-eval

The authenticated session supplies evaluator attribution; the request body does not accept evaluatedBy or createdBy. Text fields are limited to 16,000 characters and reject unsafe control characters. Sample ids are globally unique and idempotent across replicas.

Route Map#

  • Route-map articles ×5

Sibling quick-actions#

  • Open Isis Benchmarking workspace → /studio/isis/benchmarking
  • Back to Studio workspace index → /studio

Data & contracts#

  • Reads: GET /v1/admin/isis/feedback-eval (persisted samples + aggregate; no process-local seed or synthetic quality data)
  • Writes: POST /v1/admin/isis/feedback-eval (atomically stores a normalized review-package sample and returns its server-confirmed aggregate BLEU / ROUGE-L projection)
  • Realtime: none
  • Caching: client fetch on mount, cache: 'no-store', buildBffAuthHeaders()
  • Auth/role check: admin-scoped, fail-closed (401/403); route signed-in + studio entitlement (not AAA-gated)
  • Durability: PostgreSQL is required in production; records survive process restart and are visible across replicas
  • Privacy / deletion: evaluator identity is stored as a bound digest. Account erasure pseudonymizes the operator state and installs a permanent digest fence that rejects future evaluation writes for that identity
  • Retention: the canonical review_package class is retained for seven years; eligible rows are purged only after the retention boundary and are skipped while a legal hold or approved exception is active
  • Component sources:
    • apps/oshun/web/src/components/studio/StudioIsisUserFeedbackLoopWorkspace.tsx
    • apps/oshun/web/src/components/studio/StudioIsisUserFeedbackLoopRouteMap.ts

Cross-references#

Open questions / known gaps#

  • No breadcrumb on this route — navigation via sibling quick-actions
  • Route-map sub-paths are declared but not yet implemented as separate pages