---
path: /studio/isis/benchmarking
surface: studio
domain: isis
auth:
  signed-in + studio entitlement (segment not in `AAA_ONLY_STUDIO_ROUTES` —
  renders as-is)
source: apps/oshun/web/src/app/studio/isis/benchmarking/page.tsx
status: walked
last_walked:
  '2026-05-29 automated runtime walk (Playwright headless) — render, /v1 data
  (2xx), console/page-errors, expected content, screenshot verified; live
  screen-reader, touch, offline, and telemetry-delivery checks pending a manual
  AT pass. Evidence: WALKTHROUGH/results/runtime-sweep-2026-05-29.md; body
  re-derived 2026-06-03 from current source (lane-console architecture)'
---

# Isis Benchmarking

## Purpose

Admin lane console for §24.9 quality benchmarking: it scores generated 3D outputs
against reference embeddings using the real 3d-generation-benchmarks
CLIP-similarity metric (clamped cosine) and gates each case on a per-case
threshold. All reads and evaluations flow through `/v1/admin/isis/benchmarking`;
it is admin-scoped and fails closed.

## Entry points

- Sibling quick-action from `/studio/isis/artifact-detection`
- Studio index (`/studio`)
- Direct URL / bookmark

## Layout regions

`page.tsx` mounts `<ShellLayout active="studio">` (no breadcrumb panel) and three
panels: the workspace, the Route Map, and sibling quick-actions.

- **Header**: shell header
- **Workspace panel** (`data-isis-benchmarking-workspace`):
  - `<h1>` (`WorkspaceHeading`) "Isis Benchmarking Workspace"
  - Summary paragraph (`data-benchmark-summary`)
  - Rollup pass-rate (`data-benchmark-rollup` / `data-benchmark-passrate` /
    `data-benchmark-passing`)
  - "Evaluate a benchmark case" form (`data-benchmark-form`)
  - Stored cases list (`data-benchmark-cases`), each `data-benchmark-case`; empty
    state `data-benchmark-cases-empty`
- **Route Map panel** (`data-isis-benchmarking-route-map`): `<h2>Route Map</h2>`
  over `STUDIO_ISIS_BENCHMARKING_ROUTE_MAP` (5 entries)
- **Sibling quick-actions panel**: `/studio/isis/artifact-detection`, `/studio`

## States

- [ ] **Loading** — `data-benchmark-loading`
- [ ] **Unauthorized** — 401/403 admin-scope fail-closed;
      `data-benchmark-unauthorized`
- [ ] **Error** — non-OK or malformed response; `data-benchmark-error`
- [ ] **Ready (form)** — response present; rollup + form + cases render
- [ ] **Result** — after a score POST, `data-benchmark-result` shows
      `data-benchmark-verdict`, `data-benchmark-clip` (CLIP similarity), and
      `data-benchmark-status`

## Interactions

### Evaluate a benchmark case (`data-benchmark-form`)

- [ ] **Case id** (`data-benchmark-caseid`, text input)
- [ ] **Label** (`data-benchmark-label`, text input)
- [ ] **Threshold (0..1)** (`data-benchmark-threshold`, number input,
      `min=0 max=1 step=0.01`)
- [ ] **Reference embedding (comma-separated numbers)**
      (`data-benchmark-reference`, text input)
- [ ] **Candidate embedding (comma-separated numbers)**
      (`data-benchmark-candidate`, text input)
- [ ] **Run benchmark** (`data-benchmark-run`, button; "Scoring…" while
      submitting) — `POST /v1/admin/isis/benchmarking`

### Route Map

- [ ] **Route-map articles** ×5

### Sibling quick-actions

- [ ] **Open Isis Artifact Detection workspace** →
      `/studio/isis/artifact-detection`
- [ ] **Back to Studio workspace index** → `/studio`

## Data & contracts

- **Reads**: `GET /v1/admin/isis/benchmarking` (cases + rollup pass rate)
- **Writes**: `POST /v1/admin/isis/benchmarking` (score a case; returns the CLIP
  similarity verdict and updated rollup)
- **Realtime**: none
- **Caching**: client fetch on mount, `cache: 'no-store'`, `buildBffAuthHeaders()`
- **Auth/role check**: admin-scoped, fail-closed (401/403); route gated on
  signed-in + studio entitlement
- **Component sources**:
  - `apps/oshun/web/src/components/studio/StudioIsisBenchmarkingWorkspace.tsx`
  - `apps/oshun/web/src/components/studio/StudioIsisBenchmarkingRouteMap.ts`

## Cross-references

- Studio overview: [`../../studio-overview.md`](../../studio-overview.md)
- Sibling routes:
  - [`studio-isis-artifact-detection.md`](./studio-isis-artifact-detection.md)
  - [`studio-isis-model-quality-assessment.md`](./studio-isis-model-quality-assessment.md)
  - [`studio-isis-automated-quality-scoring.md`](./studio-isis-automated-quality-scoring.md)

## Open questions / known gaps

- [ ] No breadcrumb on this route — navigation is via sibling quick-actions
- [ ] Embeddings are pasted as raw comma-separated vectors; confirm whether they
      will be sourced from real asset embeddings before V1