---
path: /studio/isis/user-feedback-loop
surface: studio
domain: isis
auth: signed-in + studio entitlement (NOT AAA-gated)
source: apps/oshun/web/src/app/studio/isis/user-feedback-loop/page.tsx
status: walked
last_walked:
  '2026-07-21 focused Playwright runtime walk — authenticated
  empty/read/add/reload, duplicate submission, erased-subject,
  storage-unavailable, malformed-response, keyboard-control, and axe checks
  verified; live screen-reader, touch, offline, and telemetry-delivery checks
  still require a manual AT pass.'
---

# Studio · Isis · User Feedback Loop

## Purpose

Admin lane console for diagnostic model-output evaluation. An authenticated
operator submits an approved, de-identified internal review package (predicted
model output vs. reference answer); the service scores it and aggregates mean
BLEU and mean ROUGE-L across persisted samples. The metrics inform human review
and do not automatically release, publish, or train a model. Customer feedback,
personal data, and unapproved source material must not be entered. Reads and
scores use `/v1/admin/isis/feedback-eval`; access and storage fail closed.

## Entry points

- Sibling quick-action from `/studio/isis/benchmarking`
- Studio index (`/studio`)
- Direct URL / bookmark

## Layout regions

`page.tsx` mounts `<ShellLayout active="studio">` (no breadcrumb panel) and
three panels: the workspace, the Route Map, and sibling quick-actions.

- **Header**: shell header
- **Workspace panel** (`data-isis-user-feedback-loop-workspace`):
  - `<h1>` (`WorkspaceHeading`) "Isis User Feedback Loop Workspace"
  - Summary paragraph (`data-feedback-summary`)
  - Aggregate (`data-feedback-aggregate`) with `data-feedback-mean-bleu`,
    `data-feedback-mean-rougel`
  - Score form (`data-feedback-form`)
  - Scored samples (`data-feedback-samples`, `data-feedback-sample=<id>`,
    `data-feedback-scores`); empty state `data-feedback-samples-empty`
- **Route Map panel** (`data-isis-user-feedback-loop-route-map`):
  `<h2>Route Map</h2>` over the route map (5 entries)
- **Sibling quick-actions panel**: `/studio/isis/benchmarking`, `/studio`

## States

- [ ] **Loading** — `data-feedback-loading`
- [ ] **Unauthorized** — 401/403 admin-scope fail-closed;
      `data-feedback-unauthorized`
- [ ] **Storage unavailable** — 503 persistence failure; `data-feedback-error`
- [ ] **Erased subject** — 410 when the authenticated operator's permanent
      deletion fence rejects a new evaluation write; `data-feedback-error`
- [ ] **Malformed response** — unexpected or internally inconsistent payload;
      `data-feedback-error`
- [ ] **Ready (form)** — aggregate + score form render
- [ ] **Empty** — no samples scored yet; `data-feedback-samples-empty`
- [ ] **Result** — server-confirmed sample and aggregate in
      `data-feedback-result` (`data-feedback-status`) after scoring
- [ ] **Duplicate** — 409 for an already-persisted sample id; form remains
      recoverable with a specific message

## Interactions

### Score form (`data-feedback-form`)

- [ ] **Sample id** (`data-feedback-sampleid`, input)
- [ ] **Model output (predicted)** (`data-feedback-predicted`, textarea)
- [ ] **Reference answer (expected)** (`data-feedback-expected`, textarea)
- [ ] **Score** (`data-feedback-score`, submit) —
      `POST     /v1/admin/isis/feedback-eval`

The authenticated session supplies evaluator attribution; the request body does
not accept `evaluatedBy` or `createdBy`. Text fields are limited to 16,000
characters and reject unsafe control characters. Sample ids are globally unique
and idempotent across replicas.

### Route Map

- [ ] **Route-map articles** ×5

### Sibling quick-actions

- [ ] **Open Isis Benchmarking workspace** → `/studio/isis/benchmarking`
- [ ] **Back to Studio workspace index** → `/studio`

## Data & contracts

- **Reads**: `GET /v1/admin/isis/feedback-eval` (persisted samples + aggregate;
  no process-local seed or synthetic quality data)
- **Writes**: `POST /v1/admin/isis/feedback-eval` (atomically stores a
  normalized review-package sample and returns its server-confirmed aggregate
  BLEU / ROUGE-L projection)
- **Realtime**: none
- **Caching**: client fetch on mount, `cache: 'no-store'`,
  `buildBffAuthHeaders()`
- **Auth/role check**: admin-scoped, fail-closed (401/403); route signed-in +
  studio entitlement (not AAA-gated)
- **Durability**: PostgreSQL is required in production; records survive process
  restart and are visible across replicas
- **Privacy / deletion**: evaluator identity is stored as a bound digest.
  Account erasure pseudonymizes the operator state and installs a permanent
  digest fence that rejects future evaluation writes for that identity
- **Retention**: the canonical `review_package` class is retained for seven
  years; eligible rows are purged only after the retention boundary and are
  skipped while a legal hold or approved exception is active
- **Component sources**:
  - `apps/oshun/web/src/components/studio/StudioIsisUserFeedbackLoopWorkspace.tsx`
  - `apps/oshun/web/src/components/studio/StudioIsisUserFeedbackLoopRouteMap.ts`

## Cross-references

- Studio overview: [`../../studio-overview.md`](../../studio-overview.md)
- Sibling routes:
  - [`studio-isis-benchmarking.md`](./studio-isis-benchmarking.md)
  - [`studio-isis-automated-quality-scoring.md`](./studio-isis-automated-quality-scoring.md)

## Open questions / known gaps

- [ ] No breadcrumb on this route — navigation via sibling quick-actions
- [ ] Route-map sub-paths are declared but not yet implemented as separate pages