V1 Web PWA · Surface walkthrough

Isis Benchmarking

A per-surface walkthrough of the V1 Web PWA studio surface: layout, states, interactions, data, and cross-references.

walked
8sections2 minread
Markdown source
On this page

Context. surface studio · domain isis · route /studio/isis/benchmarking · auth signed-in + studio entitlement (segment not in AAA_ONLY_STUDIO_ROUTES — renders as-is) · source apps/oshun/web/src/app/studio/isis/benchmarking/page.tsx

Last walked. 2026-05-29 automated runtime walk (Playwright headless) — render, /v1 data (2xx), console/page-errors, expected content, screenshot verified; live screen-reader, touch, offline, and telemetry-delivery checks pending a manual AT pass. Evidence: WALKTHROUGH/results/runtime-sweep-2026-05-29.md; body re-derived 2026-06-03 from current source (lane-console architecture)

Purpose#

Admin lane console for §24.9 quality benchmarking: it scores generated 3D outputs against reference embeddings using the real 3d-generation-benchmarks CLIP-similarity metric (clamped cosine) and gates each case on a per-case threshold. All reads and evaluations flow through /v1/admin/isis/benchmarking; it is admin-scoped and fails closed.

Entry points#

  • Sibling quick-action from /studio/isis/artifact-detection
  • Studio index (/studio)
  • Direct URL / bookmark

Layout regions#

page.tsx mounts <ShellLayout active="studio"> (no breadcrumb panel) and three panels: the workspace, the Route Map, and sibling quick-actions.

  • Header: shell header
  • Workspace panel (data-isis-benchmarking-workspace):
    • <h1> (WorkspaceHeading) "Isis Benchmarking Workspace"
    • Summary paragraph (data-benchmark-summary)
    • Rollup pass-rate (data-benchmark-rollup / data-benchmark-passrate / data-benchmark-passing)
    • "Evaluate a benchmark case" form (data-benchmark-form)
    • Stored cases list (data-benchmark-cases), each data-benchmark-case; empty state data-benchmark-cases-empty
  • Route Map panel (data-isis-benchmarking-route-map): <h2>Route Map</h2> over STUDIO_ISIS_BENCHMARKING_ROUTE_MAP (5 entries)
  • Sibling quick-actions panel: /studio/isis/artifact-detection, /studio

States#

  • Loading — data-benchmark-loading
  • Unauthorized — 401/403 admin-scope fail-closed; data-benchmark-unauthorized
  • Error — non-OK or malformed response; data-benchmark-error
  • Ready (form) — response present; rollup + form + cases render
  • Result — after a score POST, data-benchmark-result shows data-benchmark-verdict, data-benchmark-clip (CLIP similarity), and data-benchmark-status

Interactions#

Evaluate a benchmark case (data-benchmark-form)#

  • Case id (data-benchmark-caseid, text input)
  • Label (data-benchmark-label, text input)
  • Threshold (0..1) (data-benchmark-threshold, number input, min=0 max=1 step=0.01)
  • Reference embedding (comma-separated numbers) (data-benchmark-reference, text input)
  • Candidate embedding (comma-separated numbers) (data-benchmark-candidate, text input)
  • Run benchmark (data-benchmark-run, button; "Scoring…" while submitting) — POST /v1/admin/isis/benchmarking

Route Map#

  • Route-map articles ×5

Sibling quick-actions#

  • Open Isis Artifact Detection workspace → /studio/isis/artifact-detection
  • Back to Studio workspace index → /studio

Data & contracts#

  • Reads: GET /v1/admin/isis/benchmarking (cases + rollup pass rate)
  • Writes: POST /v1/admin/isis/benchmarking (score a case; returns the CLIP similarity verdict and updated rollup)
  • Realtime: none
  • Caching: client fetch on mount, cache: 'no-store', buildBffAuthHeaders()
  • Auth/role check: admin-scoped, fail-closed (401/403); route gated on signed-in + studio entitlement
  • Component sources:
    • apps/oshun/web/src/components/studio/StudioIsisBenchmarkingWorkspace.tsx
    • apps/oshun/web/src/components/studio/StudioIsisBenchmarkingRouteMap.ts

Cross-references#

Open questions / known gaps#

  • No breadcrumb on this route — navigation is via sibling quick-actions
  • Embeddings are pasted as raw comma-separated vectors; confirm whether they will be sourced from real asset embeddings before V1