# Composer production runbook

Operational handbook for running the `@isis/scene-from-image-composer` pipeline
against real upstream providers + persistence + observability. Pairs with
`manual-qa.md` (acceptance recipes for the deferred ISIS_GAPS items) and
`phase-summaries.md` (per-phase shipped work).

Closes P3-007.

---

## 1. Prerequisites

- Node 22.x, pnpm ≥9, TypeScript ≥5.
- Postgres 16.x with a database the operator can `prisma db push` into. The
  composer uses one schema + `pipeline_state_*` table prefix.
- Redis 7.x for `dispatchMode: 'job-envelope'` runs (in- process mode skips
  this).
- MinIO/S3-compatible blob storage for large artifact off-loading (block H
  persistence currently inlines all artifacts; the artifactUri path is reserved
  for a future cleanup).
- API keys for each upstream provider used:
  - World Labs Marble (`MARBLE_API_KEY`)
  - Hunyuan / Tripo / Meshy (per provider)
  - ElevenLabs SFX (`ELEVENLABS_API_KEY`)

## 2. Boot sequence

```bash
# 1. Bring up infra.
docker compose -f docker/docker-compose.dev.yml up -d postgres redis minio
docker compose -f docker/docker-compose.dev.yml ps  # verify all healthy

# 2. Install deps + sync schema.
pnpm install
cd libs/isis/database && \
  ISIS_DATABASE_URL='postgresql://oshun:oshun_dev@localhost:5432/isis' \
  npx prisma db push
cd -

# 3. Build packages the BFF will load.
pnpm nx run @isis/scene-from-image-composer:build
pnpm nx run @isis/operation-orchestrator:build
```

## 3. Composer call shape

The convenience entry point:

```ts
import {
  composeSceneFromImage,
  type SceneFromImageRequest,
} from '@isis/scene-from-image-composer';
import { createMarbleProvider } from '@isis/3d-generation/providers';
import { createHunyuanProvider } from '@isis/3d-generation/providers';
import { createElevenLabsSfxProvider } from '@isis/audio-generation/generation';

const result = await composeSceneFromImage({
  assetId: 'asset-12345',
  tenantId: 'tenant-abc',
  userId: 'user-xyz',
  sourceImage: readFileSync('input.png'),
  mimeType: 'image/png',
  providers: {
    marble: createMarbleProvider({
      apiKey: process.env.MARBLE_API_KEY,
      mode: 'environment',
      splatDensity: 'high',
      emitAmbientHint: true,
    }),
    mesh: createHunyuanProvider({ apiKey: process.env.HUNYUAN_API_KEY }),
    sfx: createElevenLabsSfxProvider({
      apiKey: process.env.ELEVENLABS_API_KEY,
    }),
  },
  objects: [
    {
      source: 'explicit',
      id: 'tree-1',
      prompt: 'cypress tree',
      anchor: { type: 'centre-ground' },
    },
  ],
  outputMode: 'baked', // produce composed .spz
  bakeOptions: { method: 'surface-sampling' }, // 'hybrid' for sharper edges
  ambient: { prompt: 'forest dawn ambient' },
  // Block H — persist run state to Postgres.
  repositoryConfig: { mode: 'postgres' },
  // Block I — wire observability + policy.
  policyContext: {
    budget: {
      caps: [{ maxCostCents: 5000 }],
      perPhaseEstimateCents: { environment: 300, objects: 500, bake: 200 },
      reportPhaseCostCents: (phase) => upstreamBillingClient.lastCost(phase),
    },
    license: {
      registry: orgLicenseRegistry,
      deploymentTopology: 'self-hosted-saas',
      commercialIntent: 'commercial-saas',
    },
    gallery: {
      tenantId: 'tenant-abc',
      userId: 'user-xyz',
      domain: 'sophia',
      entitlementTier: 'curated-creator',
      model: 'world-labs-marble',
      generatorKey: 'scene-from-image-composer/v1',
      locale: 'en-US',
      assetUrl: 'https://cdn.example.com/scenes/asset-12345.spz',
      thumbnailUrl: 'https://cdn.example.com/scenes/asset-12345.thumb.png',
      costCents: 0, // updated post-run from result.policyOutcomes.budget
    },
    anomaly: { zScoreThreshold: 3.0 },
  },
});
```

The result carries:

- `manifest` — the canonical `OutputManifest` to persist.
- `phaseOutputs` — per-phase outputs (env splat bytes, mesh GLBs, bake metrics,
  etc.).
- `policyOutcomes` — gallery `OutputRecord`, lineage graph, budget report,
  license assessment, anomaly stats.

## 4. Dispatch mode (separate worker process)

For BFF setups where the composer's heavy phases run on a GPU worker pool
instead of inline:

```ts
// In the BFF:
const result = await composeSceneFromImage({
  ...,
  dispatchMode: 'job-envelope',
  dispatchConfig: { redisUrl: 'redis://localhost:6379' },
});

// In the worker process (separate Node):
import { attachWorker } from '@isis/scene-from-image-composer/dispatch/attach-worker';
import { RedisJobChannel } from '@isis/scene-from-image-composer/dispatch/redis-job-channel';
import { createEnvironmentHandler, createMeshHandler, createBakeHandler } from '...';

const channel = new RedisJobChannel({ redisUrl: 'redis://localhost:6379' });
const envHandler = createEnvironmentHandler({ createMarbleProvider });
const meshHandler = createMeshHandler({ providers: { hunyuan, tripo, meshy } });
const bakeHandler = createBakeHandler();  // uses real bakeMesh by default

await attachWorker({ channel, handler: envHandler });
await attachWorker({ channel, handler: meshHandler });
await attachWorker({ channel, handler: bakeHandler });

console.log('composer worker ready');
```

The worker process is long-lived; the BFF dispatches per-job envelopes; the
worker handlers process them; responses flow back via per-jobId response
channels. See `libs/isis/scene-from-image-composer/src/dispatch/` for the full
wiring.

## 5. Resume after interruption

Composer runs can be 5–30 minutes. If the BFF restarts:

```ts
// 1. Look up unfinished pipelines from the repo:
const repo = new PostgresPipelineRepository({ prisma });
const orch = createPipelineOrchestrator(undefined, undefined, repo);
const awaiting = await repo.loadAwaitingCheckpoints();  // pending checkpoints
// (For RUNNING pipelines, the BFF should also track pipelineIds in its
// own state — the orchestrator doesn't surface a list-all method by design.)

// 2. Resume each:
const resumed = await composeSceneFromImage({
  ...,
  repositoryConfig: { mode: 'postgres' },
  resumeFromPipelineId: pipelineId,
});
```

Resume reads the snapshot, skips completed phases (hydrating their outputs from
artifacts), and runs the remaining phases. End state matches a non-interrupted
run; see composer.persistence.test.ts > SIGKILL-resume golden.

## 6. Observability hooks

The composer's `policyOutcomes` surface is the primary hook. For per-phase
events, subscribe to the orchestrator:

```ts
import { createPipelineOrchestrator } from '@isis/operation-orchestrator';

const orch = createPipelineOrchestrator(undefined, undefined, repo);
orch.on('pipeline_started', (e) =>
  metricsClient.gauge('pipeline.started', 1, { pipelineId: e.pipelineId })
);
orch.on('pipeline_completed', (e) =>
  metricsClient.gauge('pipeline.completed', 1, {
    pipelineId: e.pipelineId,
    durationMs: e.durationMs,
  })
);
orch.on('pipeline_error', (e) =>
  metricsClient.gauge('pipeline.failed', 1, {
    pipelineId: e.pipelineId,
    error: e.error,
  })
);
```

`policyOutcomes.anomaly.anomalies` carries any z-score threshold violations
across the run's tracked channels (object-count, grounding-recall,
grounding-novelty, coherence-score) — wire to PagerDuty / Slack as appropriate.

## 7. Cost reporting + budget enforcement

Pre-flight budget caps:

```ts
policyContext: {
  budget: {
    caps: [
      { name: 'per-pipeline-cap', maxCostCents: 5000 },
      { name: 'env-phase-cap', maxCostCents: 1000, maxCostPerOpCents: 1000 },
    ],
    perPhaseEstimateCents: { environment: 300, objects: 500, bake: 200 },
    reportPhaseCostCents: (phase) => upstreamBillingClient.lastCost(phase),
  },
},
```

If a pre-flight check fails (estimate + spent > cap), the composer throws
`ComposerError({ code: 'invalid-input' })` BEFORE running the phase. Caller
catches + reports.

## 8. License compliance

Per-artifact decisions:

```ts
policyContext: {
  license: {
    registry: orgLicenseRegistry,
    deploymentTopology: 'self-hosted-saas',
    commercialIntent: 'commercial-saas',
    failOnDeny: true,  // default; set false to audit without blocking
  },
},
```

A `deny` aggregate decision throws `ComposerError({ code: 'invalid-input' })`
after the objects phase (before bake- phase compute burns). Per-artifact
assessments land in `result.policyOutcomes.license.perArtifact` for audit logs.

## 9. Gallery + lineage

`policyOutcomes.gallery` is the `OutputRecord` the operator writes to the
gallery's storage tier (the composer doesn't own that write — the BFF or a
downstream consumer does).

`policyOutcomes.lineage` is the full `LineageGraph` — pipe into the gallery's
lineage indexer or store as a JSON blob alongside the manifest.

## 10. Troubleshooting

- **`ComposerError({ code: 'phase-failed', phase: 'environment' })`** — Marble
  API call failed. Check Marble API key + the upstream service's status page.
- **`ComposerError({ code: 'provider-missing', phase: 'objects' })`** — request
  had `objects: [...]` but no `providers.mesh`. Add a mesh provider.
- **`PipelineAlreadyClosedError`** — tried to resume a pipeline already marked
  COMPLETED or CANCELLED. Check `repo.loadPipeline(pipelineId).status` first.
- **`JobTimeoutError`** — dispatch mode hit response timeout. Increase
  `dispatchConfig.responseTimeoutMs` (default 30 min) or check worker process
  health.
- **`DuplicateJobIdError`** — the same jobId was dispatched twice on the same
  JobDispatcher instance. Caller should use unique jobIds or wait for the first
  dispatch to settle before re-dispatching.

## 11. Health checks

- Postgres: `SELECT 1` against the isis database.
- Redis (when dispatching): `PING` returns PONG.
- Composer wiring: run `composer.test.ts` against fixtures — if 13 tests pass,
  the composer's public API contract is green.
