# ADR-0005: Slim Worker Image and Volume-Backed Models for RunPod Serverless

**Status**: Proposed. Written 2026-09-11 as accepted, with no approval record;
awaiting approval in Eve. "Current state (2026-10-01)" below corrects what has
changed since.  
**Date**: 2026-09-11  
**Authors**: Isis Runtime Engineering  
**Supersedes**: the per-flavor baked-model images described in
`docker/runpod/comfyui/Dockerfile` (`MODEL_FLAVOR=sd15|sdxl|…|flagship-image`)
for every new endpoint; existing flavor entries are retained in
`infra/runpod/endpoints/inventory/` as records only.

## Current state (2026-10-01)

The sections after this one record the decision as made on 2026-09-11. This
section describes the code on 2026-10-01 and governs wherever they differ.

1. **Cost (Decision 5, Consequences).** The two development endpoints still run
   with `workersMin: 0`, `workersMax: 1`, `idleTimeout: 10` and an execution
   timeout of 300 s or 900 s (`infra/runpod/endpoints/desired-state.json`). The
   generation API's daily ledger (`runpod-cost-ledger.service.ts`) is an
   estimate-based admission guard, not a strict budget. It refuses a job when
   today's recorded spend plus that job's estimate exceeds the daily budget, but
   it reserves nothing for jobs admitted and not yet finished. Concurrent
   submissions can therefore pass against the same remaining balance, and a job
   can cost more than its estimate. `workersMax` limits concurrency, not
   cumulative spend. "Idle costs nothing" holds only for GPU workers scaled to
   zero: the network volume is billed continuously
   (`infra/runpod/volumes/desired-state.json`: 400 GB, $28 a month). A strict
   budget remains unresolved. It would need reservations for outstanding jobs,
   execution ceilings, and reconciliation against actual cost.
2. **Build path (Decision 3, Alternatives).** GitHub Actions has been
   billing-blocked on this account since 2026-09-11. The endpoints therefore
   build from the repository through RunPod's GitHub integration
   (`"kind": "runpod-github"` in the endpoint desired state), with the
   repository root as build context. That is the only working path. The GHCR
   route of Decision 3 is not a usable fallback today, even with billing
   restored. Its workflow (`.github/workflows/runpod-comfyui-slim.yml`) builds
   with `docker/runpod/comfyui-slim` as context, but the Dockerfile now copies
   the workflow golden fixtures (`libs/isis/workflows/.../__golden__/`) and the
   volume manifest from the repository root, so that build would fail. Using it
   again needs Actions billing, a repository-root build context, a passing smoke
   test and a published private image.
3. **Models (Decision 2, Consequences).** The model library on Hetzner is the
   source of truth for weights. The RunPod network volume is a 400 GB
   least-recently-used cache of it (Isis ADR-0010, Eve document
   b3fb7aaa-7bec-83d8-b91c-a8cec377e1b2;
   `infra/runpod/volumes/isis-chroma-models.manifest.json` says the same). The
   pre-submit check compares a workflow's model names with the volume manifest;
   it does not prove a file is present. Residency is decided by cache admission
   (`apps/isis/generation-api/src/services/runpod-model-cache/admission.ts`).
   With `ISIS_RUNPOD_CACHE_AUTOSYNC` off (the default), a job whose model
   families are not cached is refused with `ModelsNotCachedError`. With it on,
   the job is held as `awaiting_cache` while the missing families sync, and the
   cached ones stay leased.
4. **Capabilities (Consequences).** The vendored SAM 3 nodes were removed
   (A.01.09). `SAM3Segment` and its sibling classes must not appear in the image
   (`docker/runpod/comfyui-slim/smoke-test.sh`). Text-prompted segmentation uses
   ComfyUI's core SAM 3.1 classes (`SAM3_Detect` in
   `libs/isis/workflows/src/workflows/chroma/sam3-segment.json`). RMBG-2.0 was
   dropped from the matting catalog (A.02.08).
5. **References.** The tracker is now the `isis-chroma-runpod` board family. The
   evidence ledger is Eve document fe1e2d43-646f-8c5c-be24-1ce5398263b4. Isis
   ADR-0003 and ADR-0004 are Eve documents 079a1759-7861-89b6-862b-5942e6843a53
   and 06493ace-dc51-88ce-8be8-fa12a5a8dd55. None of these is approved yet, so
   this ADR names them without pinning them. `docker/runpod/comfyui-slim/README.md`,
   which held the token-rotation steps, is no longer in the repository.

## Context and Problem Statement

Isis needed real image generation on RunPod serverless with the full-precision
Chroma stack plus SAM 3, BiRefNet and RMBG-2.0, usable from the Isis dashboard
the same day and with spend bounded by configuration.

Findings on 2026-09-11:

- The image lineage every existing endpoint assumes, `oshunai/runpod-comfyui:*`
  on top of `oshunai/runpod-base`, has **never been published** (Docker Hub
  returns 404 for both repositories). Nothing in `desired-state.json` has ever
  run.
- That lineage bakes a per-flavor model set into the image. The
  `comfyui-flagship-image` inventory lists 47 baked artifacts. Every new model
  means a multi-hour, multi-hundred-GB image rebuild, and the build cannot run
  on the developer machine (16 GB RAM).
- RunPod's stock `runpod/worker-comfyui` image already implements the serverless
  handler contract (API-format graph in, base64/S3 images out), reads models
  from a network volume at `/runpod-volume/models/...`, and supports custom node
  packs via a derived Dockerfile.
- The user's constraints: no new third-party accounts, no public image, no
  quantized weights, cost strictly bounded.

## Decision

1. **Worker image = ComfyUI + node packs only.** `docker/runpod/comfyui-slim/`
   derives from `runpod/worker-comfyui:5.10.0-base` (pinned by tag and digest)
   and adds ComfyUI-RMBG at a pinned commit. No model file is ever copied into
   an image.
2. **Models live on a RunPod network volume** (now a cache of the Hetzner
   library; Current state 3), seeded from a checked-in manifest
   (`infra/runpod/volumes/isis-chroma-models.manifest.json`: source, pinned
   revision or CivitAI file id, sha256, destination) by a stdlib-only loader
   running in a short-lived CPU pod. Adding a model is a manifest row plus a
   loader run. Only a new node pack rebuilds the image.
3. **The image is built by GitHub Actions and published to GHCR as a private
   package** (not usable today; Current state 2) (`ghcr.io/greychimp/oshun/runpod-comfyui-slim`), which the repo
   already uses for its other services. RunPod pulls it with a Container
   Registry Auth built from a read-only `read:packages` token.
4. **Every build runs an in-image smoke test before push**: ComfyUI boots on CPU
   and `/object_info` must list every node class the Isis Chroma catalog uses.
   The resulting `object_info.json` is the source of the generation API's
   pre-submit class check.
5. **Cost is bounded in desired state, not by attention** (the ledger is an
   estimate-based guard; Current state 1): `workersMax: 1`,
   `workersMin: 0`, `executionTimeout: 300 s`, `idleTimeout: 10 s`, plus the
   generation API's daily USD ledger (fails closed).
6. **The generation API renders catalog workflows to bare graphs itself**
   (`ISIS_RUNPOD_WORKER_KIND=worker-comfyui`), because the stock worker has no
   catalog renderer. The old handler contract remains selectable for the
   unpublished flavor images but is not the path forward.

## Alternatives Considered

- **Publish the baked flavor images.** Rejected: 50+ GB images, rebuild on every
  model change, cannot be built on the developer machine, and none of the
  flavors matched the Chroma stack anyway.
- **Docker Hub.** Rejected by the user (no new accounts). GHCR is already in
  use.
- **Public GHCR package** to avoid a pull credential. Rejected by the user; a
  read-only token is a one-time setup and keeps the package private.
- **RunPod's GitHub-integration build.** Would avoid a registry entirely, but
  requires connecting the GitHub account to RunPod and building the whole
  monorepo as context. Kept as a fallback here; it became the only working path
  (Current state 2).
- **Custom nodes on the volume instead of in the image.** Fragile (Python
  dependencies would need to be installed at cold start, on billed GPU seconds).
  Rejected.
- **Quantized Chroma (fp8/GGUF) to fit smaller GPUs.** Rejected by the user; the
  volume holds BF16/FP16 only and the manifest lint enforces it.

## Consequences

- Cold start now includes loading ~27 GB (checkpoint + T5) from the network
  volume, roughly 60–120 billed seconds per worker start. Warm images cost
  cents; idle GPU workers cost nothing (the volume is billed; Current state 1). The dashboard shows cold vs warm per job.
- A model referenced by a workflow but missing from the volume fails the
  pre-submit check (manifest cross-check), not at runtime. That check compares
  names only; cache admission decides residency (Current state 3).
- `groundingdino-py` is excluded from the ComfyUI-RMBG install (needs a compiler
  the runtime base lacks); the GroundingDINO-prompted `Segment` nodes are
  therefore unavailable. Text-prompted segmentation used `SAM3Segment`; it now uses core `SAM3_Detect`
  (Current state 4).
- RMBG-2.0 carries a non-commercial license (BRIA). It is on the volume for
  evaluation; commercial use needs a BRIA agreement or switching to BiRefNet.
  It has since been dropped from the matting catalog (Current state 4).
- Rotating the pull token is a documented three-step swap
  (`docker/runpod/comfyui-slim/README.md`, since removed; Current state 5).

## References

- Tracker: `ISIS_CHROMA_RUNPOD_MVP_TODOS_2026-09-11.md` (now the
  `isis-chroma-runpod` board family)
- Evidence: `docs/agents/isis-chroma-runpod-evidence.md` (now Eve document
  fe1e2d43-646f-8c5c-be24-1ce5398263b4)
- Provisioner: `scripts/isis/runpod-provision.mjs`
- Loader: `scripts/isis/runpod-volume-loader.py`
- ADR-0003 (serverless execution strategy), ADR-0004 (provisioning mechanism)

