Eve library

Provider operations runbook

Guide

Revision 1

On this page

Phase 182, 182.C.39.04. Onboarding, live probing, benchmark, promotion, quarantine, cost, retention, and deletion — eight procedures, each with the module that enforces it and the mistake it exists to prevent.

1. Onboarding#

Onboarding a provider is mostly writing down what their API will never tell you: licensing, acceptable use, commercial-use rights, biometric handling, public figures, minors, retention, and deletion (human-video-provider-constraints.ts, human-video-model-licensing.ts).

That record is manual and it carries a date. An undated constraints record is worse than none, because it reads as current: terms change, and a record with no date cannot be told apart from one written this morning.

Separately, deployment readiness is a credential and region check that must not depend on whether a TypeScript module imports (human-video-provider-readiness.ts). A provider whose adapter compiles is not a provider that answers.

Done when: a dated constraints record exists, a readiness check passes against real credentials, and the model has a record of its own (human-video-model-record.ts).

2. Live probing#

A probe asks the model to do the thing the catalog advertises, and it binds to the dated slug it ran against. A probe of alibaba/wan-2.7-20260801 is not evidence about -20260901; it is evidence about a model that is no longer being served (human-video-lane-eligibility.ts).

  • Probes expire: DEFAULT_PROBE_MAX_AGE_DAYS is 90.
  • probe_inconclusive is not probe_failed. A probe that could not be read tells us nothing, and filing it as a failure hides a lane that may work.
  • region_unrecorded is not region_unavailable. Nobody has written down where this lane may serve, and treating silence as "allowed everywhere" is how a likeness ends up processed in a jurisdiction the authorization never covered. It blocks, and it says which of the two it is.

Never: accept advertised support as eligibility. Advertised capability is an invitation to probe, never a promise.

3. Benchmark#

A benchmark is a cohort claim, and 182.C.34.07 prices it: 299 clean renders per cohort to substantiate a 99% tier at 95% confidence, derived exactly rather than by the rule of three. At the measured $0.25 per delivered second and eight-second takes that is $598 per cohort, and six cohorts is $3,588 — in renders alone, before anybody is paid to look at them.

Split the corpus by subject and scenario, never by clip (human-video-corpus-splits.ts): one person in twelve clips, shuffled at clip level, puts eight in calibration and four in validation, and the threshold is then fitted on a face it is validated against. The number that comes back is always better than the truth.

4. Promotion#

Five rungs, none skippable (human-video-promotion-ladder.ts):

discovered → probed → shadow → canary → default

  • Shadow is the rung people skip, and it is the one that costs nothing to skip until it does. Twenty runs minimum, alongside the live route, output served to nobody. A shadow run whose output was served is not a shadow run, and that is recorded as a fact rather than left to convention.
  • Canary is 50 takes minimum — and 50 takes resolve no failure rate below about 5.8% (182.C.38.04). A canary abort threshold written at 1% is a threshold that never fires; the 299 takes it would need are six canaries.
  • Promotion is never automatic. Demotion is, because it is the safe direction: a bounded re-run can show the aggregate has fallen and cannot re-establish what a promotion asserted (human-video-auto-demotion.ts).

5. Quarantine — and the switch that is not a quarantine#

A quarantine is a consequence. A kill switch is a decision. They must never be read from the same field.

  • Quarantine follows automatically from a catalog snapshot diff: the model changed under a stable name, and 182.C.10.03 works out what that invalidated (human-video-catalog-snapshots.ts). Nobody signs it.
  • A kill switch has a person's name, a reason and a timestamp on it (human-video-kill-switch.ts). It blocks submission only: polling, downloading, validating, evaluating and promoting already-submitted jobs all continue, because each of those only reads something that already exists. A retry counts as a submission.

If the concern is safety rather than cost or reliability, the release gates stop the take — not this.

6. Cost#

Price from what you were billed, never from the catalog. The measured case is committed: alibaba/wan-2.7 prices duration_seconds at $0.10, the shortest render it sells is 2 s, so the predicted bill was $0.20. The billed amount was $0.50 — two and a half times the prediction (evidence/2026-09-11-wan-canary/README.md).

  • Read usage.cost from the provider response; do not compute from pricing_skus.
  • Model cost per releasable second, not per attempt (human-video-cost-attribution.ts): a mean over attempts looks healthy while containing the take that retried four times.
  • Reservations, per-job ceilings and daily budgets are separate controls from the alerting on the gap between reserved and billed.

7. Retention#

Retention is a consent dimension, not a footnote: what a provider keeps decides what a subject's authorization actually permitted (human-video-provider-constraints.ts, human-video-zdr-posture.ts, human-video-retention-gaps.ts).

A fallback that moves work to a provider with weaker retention is prohibited even when it is otherwise capable (human-video-fallback-safety.ts). The fallback is a capability decision; retention is a promise already made.

8. Deletion#

Deletion destroys what we hold: origin objects, derived renditions, evaluation frames, and the face embeddings — which outlive the images they came from and are what makes re-identification possible.

It does not reach provider-side copies, CDN edge caches, signed links already issued, or anything already downloaded. Zero-retention is the only provider class where the provider-side line is genuinely empty, which is why the class is part of consent (human-video-revocation.ts).

A deletion report that lists only successes reads as completion, and the person it is shown to is the one person entitled to know it is not.

Where this runbook is still theory#

No provider has been onboarded through it end to end. One route has been probed with real money — the $0.50 canary above — and no route has a benchmark, so nothing has climbed past probed, and the quarantine and kill switch procedures have never been exercised on a live incident (182.C.38.05 is where that would be rehearsed).