Admin Cockpit · Surface walkthrough

Isis ops · RunPod endpoints

A per-surface walkthrough of the Admin Cockpit admin surface: layout, states, interactions, data, and cross-references.

drafted
8sections2 minread
Markdown source
On this page

Context. surface admin · domain isis · route /isis/runpod-endpoints · auth signed-in (admin) · source apps/oshun/admin/src/app/isis/runpod-endpoints/page.tsx

Last walked. —

Purpose#

Per-endpoint queue depth, latency P50/P95/P99, error rate, cost per hour, region, GPU class, health badge, last-failover, and cold-start indicator (§24.4). Also exposes the cost & quota panel and a queue-inspector tabset. Backed by @isis/runpod-surface.

Entry points#

  • Direct URL — /isis/runpod-endpoints
  • External runbooks / deep links
  • Not in canonical IA — no sidebar entry

Layout regions#

Page does NOT wrap in AdminShell. Resolves a DashboardContext via binding (bindRunpodDashboardLoader) and renders <RunpodDashboardPanel {...ctx} />. Default binding yields empty rows / tenantWindows / envelopes.

  • RunpodDashboardPanel:
    • Tab bar — four tabs from tab state: endpoints / cost / queue / controls; default endpoints
    • Endpoints view — DashboardRow[] sorted worst-health-first; columns include latency / error / cost / region / GPU / health / last failover / cold-start
    • Cost view — TenantCostWindow[] and BudgetEnvelope[] with evaluateBudget + tenantFairnessView summaries
    • Queue view — QueueSnapshot rows; per-job priority change input + rationale textbox; applyQueueAction audits actions; computeEtaSeconds produces ETA values from avgRuntimeSeconds
    • Controls view — kill switches (killSwitches state), region-failover form (failoverDraft), secret-rotation form (rotationDraft); recordRegionFailover, startRotation, advanceRotation from @isis/runpod-surface are wired

States#

  • Anonymous → global middleware redirects
  • Binding unset → empty rows / tenant windows / envelopes; tabs render empty
  • tab === 'endpoints' → sorted endpoint rows (worst health first)
  • tab === 'cost' → cost & quota panel with envelope status
  • tab === 'queue' → queue jobs with priority change inputs
  • tab === 'controls' → kill switches, region failover, secret rotation forms
  • controlError non-null → surfaced in controls tab when an action fails validation
  • queueRationale empty when changing priority → action may be rejected (verify in full source)

Interactions#

  • Tab buttons (×4: endpoints / cost / queue / controls)
  • Endpoint row — read-only metrics (health badge, latency, error, cost)
  • Queue priority input (per job) — newPriorityByJob map
  • Queue rationale textarea — queueRationale
  • Queue submit — applyQueueAction produces a QueueAction record; appended to queueAudit
  • Kill switch toggle (per endpoint or region)
  • Region failover form — fields fromRegion, toRegion, rollbackSeconds, rationale; submit calls recordRegionFailover
  • Secret rotation form — fields endpointId, oldKeyId, newKeyId, rationale; calls startRotation → advanceRotation

Data & contracts#

  • Reads: binding() returns DashboardContext — rows, tenantWindows, envelopes, optional queueSnapshot, avgRuntimeSeconds, failoverEntries, secretRotations, operatorId, nowUnixSeconds
  • Writes: pure-function audits in-component (no BFF write visible). Production persistence presumably happens via a downstream binding
  • Realtime: None at component level. Metrics views would benefit from realtime but this snapshot is per nav
  • Auth/role check: only the global middleware; operator id falls back to operator-anon

Cross-references#

  • Shell: shell/02-routing-layouts.md
  • Sibling Isis routes: isis-lora-training.md (jobs originate there), isis-model-merging.md, isis-output-gallery.md
  • Related canonical workspace: ../workspaces/safety/incidents.md (endpoint outages declare incidents)
  • Library: @isis/runpod-surface — advanceRotation, applyQueueAction, computeEtaSeconds, evaluateBudget, recordRegionFailover, startRotation, tenantFairnessView, plus types BudgetEnvelope, DashboardRow, QueueAction, QueueSnapshot, RegionFailoverEntry, SecretRotationRecord, TenantCostWindow
  • Component source: apps/oshun/admin/src/app/isis/runpod-endpoints/RunpodDashboardPanel.tsx

Open questions / known gaps#

  • No session / scope check at the page level
  • Kill-switch persistence — they're tracked in component state only; verify there's a downstream BFF / agent that consumes killSwitches
  • Realtime is a good fit for the endpoints + queue views; document polling / WebSocket strategy