Operational handbook for running the @isis/scene-from-image-composer pipeline
against real upstream providers + persistence + observability. Pairs with
manual-qa.md (acceptance recipes for the deferred ISIS_GAPS items) and
phase-summaries.md (per-phase shipped work).
Closes P3-007.
1. Prerequisites#
- Node 22.x, pnpm ≥9, TypeScript ≥5.
- Postgres 16.x with a database the operator can
prisma db pushinto. The composer uses one schema +pipeline_state_*table prefix. - Redis 7.x for
dispatchMode: 'job-envelope'runs (in- process mode skips this). - MinIO/S3-compatible blob storage for large artifact off-loading (block H persistence currently inlines all artifacts; the artifactUri path is reserved for a future cleanup).
- API keys for each upstream provider used:
- World Labs Marble (
MARBLE_API_KEY) - Hunyuan / Tripo / Meshy (per provider)
- ElevenLabs SFX (
ELEVENLABS_API_KEY)
- World Labs Marble (
2. Boot sequence#
# 1. Bring up infra.
docker compose -f docker/docker-compose.dev.yml up -d postgres redis minio
docker compose -f docker/docker-compose.dev.yml ps # verify all healthy
# 2. Install deps + sync schema.
pnpm install
cd libs/isis/database && \
ISIS_DATABASE_URL='postgresql://oshun:oshun_dev@localhost:5432/isis' \
npx prisma db push
cd -
# 3. Build packages the BFF will load.
pnpm nx run @isis/scene-from-image-composer:build
pnpm nx run @isis/operation-orchestrator:build
3. Composer call shape#
The convenience entry point:
import {
composeSceneFromImage,
type SceneFromImageRequest,
} from '@isis/scene-from-image-composer';
import { createMarbleProvider } from '@isis/3d-generation/providers';
import { createHunyuanProvider } from '@isis/3d-generation/providers';
import { createElevenLabsSfxProvider } from '@isis/audio-generation/generation';
const result = await composeSceneFromImage({
assetId: 'asset-12345',
tenantId: 'tenant-abc',
userId: 'user-xyz',
sourceImage: readFileSync('input.png'),
mimeType: 'image/png',
providers: {
marble: createMarbleProvider({
apiKey: process.env.MARBLE_API_KEY,
mode: 'environment',
splatDensity: 'high',
emitAmbientHint: true,
}),
mesh: createHunyuanProvider({ apiKey: process.env.HUNYUAN_API_KEY }),
sfx: createElevenLabsSfxProvider({
apiKey: process.env.ELEVENLABS_API_KEY,
}),
},
objects: [
{
source: 'explicit',
id: 'tree-1',
prompt: 'cypress tree',
anchor: { type: 'centre-ground' },
},
],
outputMode: 'baked', // produce composed .spz
bakeOptions: { method: 'surface-sampling' }, // 'hybrid' for sharper edges
ambient: { prompt: 'forest dawn ambient' },
// Block H — persist run state to Postgres.
repositoryConfig: { mode: 'postgres' },
// Block I — wire observability + policy.
policyContext: {
budget: {
caps: [{ maxCostCents: 5000 }],
perPhaseEstimateCents: { environment: 300, objects: 500, bake: 200 },
reportPhaseCostCents: (phase) => upstreamBillingClient.lastCost(phase),
},
license: {
registry: orgLicenseRegistry,
deploymentTopology: 'self-hosted-saas',
commercialIntent: 'commercial-saas',
},
gallery: {
tenantId: 'tenant-abc',
userId: 'user-xyz',
domain: 'sophia',
entitlementTier: 'curated-creator',
model: 'world-labs-marble',
generatorKey: 'scene-from-image-composer/v1',
locale: 'en-US',
assetUrl: 'https://cdn.example.com/scenes/asset-12345.spz',
thumbnailUrl: 'https://cdn.example.com/scenes/asset-12345.thumb.png',
costCents: 0, // updated post-run from result.policyOutcomes.budget
},
anomaly: { zScoreThreshold: 3.0 },
},
});
The result carries:
manifest— the canonicalOutputManifestto persist.phaseOutputs— per-phase outputs (env splat bytes, mesh GLBs, bake metrics, etc.).policyOutcomes— galleryOutputRecord, lineage graph, budget report, license assessment, anomaly stats.
4. Dispatch mode (separate worker process)#
For BFF setups where the composer's heavy phases run on a GPU worker pool instead of inline:
// In the BFF:
const result = await composeSceneFromImage({
...,
dispatchMode: 'job-envelope',
dispatchConfig: { redisUrl: 'redis://localhost:6379' },
});
// In the worker process (separate Node):
import { attachWorker } from '@isis/scene-from-image-composer/dispatch/attach-worker';
import { RedisJobChannel } from '@isis/scene-from-image-composer/dispatch/redis-job-channel';
import { createEnvironmentHandler, createMeshHandler, createBakeHandler } from '...';
const channel = new RedisJobChannel({ redisUrl: 'redis://localhost:6379' });
const envHandler = createEnvironmentHandler({ createMarbleProvider });
const meshHandler = createMeshHandler({ providers: { hunyuan, tripo, meshy } });
const bakeHandler = createBakeHandler(); // uses real bakeMesh by default
await attachWorker({ channel, handler: envHandler });
await attachWorker({ channel, handler: meshHandler });
await attachWorker({ channel, handler: bakeHandler });
console.log('composer worker ready');
The worker process is long-lived; the BFF dispatches per-job envelopes; the
worker handlers process them; responses flow back via per-jobId response
channels. See libs/isis/scene-from-image-composer/src/dispatch/ for the full
wiring.
5. Resume after interruption#
Composer runs can be 5–30 minutes. If the BFF restarts:
// 1. Look up unfinished pipelines from the repo:
const repo = new PostgresPipelineRepository({ prisma });
const orch = createPipelineOrchestrator(undefined, undefined, repo);
const awaiting = await repo.loadAwaitingCheckpoints(); // pending checkpoints
// (For RUNNING pipelines, the BFF should also track pipelineIds in its
// own state — the orchestrator doesn't surface a list-all method by design.)
// 2. Resume each:
const resumed = await composeSceneFromImage({
...,
repositoryConfig: { mode: 'postgres' },
resumeFromPipelineId: pipelineId,
});
Resume reads the snapshot, skips completed phases (hydrating their outputs from artifacts), and runs the remaining phases. End state matches a non-interrupted run; see composer.persistence.test.ts > SIGKILL-resume golden.
6. Observability hooks#
The composer's policyOutcomes surface is the primary hook. For per-phase
events, subscribe to the orchestrator:
import { createPipelineOrchestrator } from '@isis/operation-orchestrator';
const orch = createPipelineOrchestrator(undefined, undefined, repo);
orch.on('pipeline_started', (e) =>
metricsClient.gauge('pipeline.started', 1, { pipelineId: e.pipelineId })
);
orch.on('pipeline_completed', (e) =>
metricsClient.gauge('pipeline.completed', 1, {
pipelineId: e.pipelineId,
durationMs: e.durationMs,
})
);
orch.on('pipeline_error', (e) =>
metricsClient.gauge('pipeline.failed', 1, {
pipelineId: e.pipelineId,
error: e.error,
})
);
policyOutcomes.anomaly.anomalies carries any z-score threshold violations
across the run's tracked channels (object-count, grounding-recall,
grounding-novelty, coherence-score) — wire to PagerDuty / Slack as appropriate.
7. Cost reporting + budget enforcement#
Pre-flight budget caps:
policyContext: {
budget: {
caps: [
{ name: 'per-pipeline-cap', maxCostCents: 5000 },
{ name: 'env-phase-cap', maxCostCents: 1000, maxCostPerOpCents: 1000 },
],
perPhaseEstimateCents: { environment: 300, objects: 500, bake: 200 },
reportPhaseCostCents: (phase) => upstreamBillingClient.lastCost(phase),
},
},
If a pre-flight check fails (estimate + spent > cap), the composer throws
ComposerError({ code: 'invalid-input' }) BEFORE running the phase. Caller
catches + reports.
8. License compliance#
Per-artifact decisions:
policyContext: {
license: {
registry: orgLicenseRegistry,
deploymentTopology: 'self-hosted-saas',
commercialIntent: 'commercial-saas',
failOnDeny: true, // default; set false to audit without blocking
},
},
A deny aggregate decision throws ComposerError({ code: 'invalid-input' })
after the objects phase (before bake- phase compute burns). Per-artifact
assessments land in result.policyOutcomes.license.perArtifact for audit logs.
9. Gallery + lineage#
policyOutcomes.gallery is the OutputRecord the operator writes to the
gallery's storage tier (the composer doesn't own that write — the BFF or a
downstream consumer does).
policyOutcomes.lineage is the full LineageGraph — pipe into the gallery's
lineage indexer or store as a JSON blob alongside the manifest.
10. Troubleshooting#
ComposerError({ code: 'phase-failed', phase: 'environment' })— Marble API call failed. Check Marble API key + the upstream service's status page.ComposerError({ code: 'provider-missing', phase: 'objects' })— request hadobjects: [...]but noproviders.mesh. Add a mesh provider.PipelineAlreadyClosedError— tried to resume a pipeline already marked COMPLETED or CANCELLED. Checkrepo.loadPipeline(pipelineId).statusfirst.JobTimeoutError— dispatch mode hit response timeout. IncreasedispatchConfig.responseTimeoutMs(default 30 min) or check worker process health.DuplicateJobIdError— the same jobId was dispatched twice on the same JobDispatcher instance. Caller should use unique jobIds or wait for the first dispatch to settle before re-dispatching.
11. Health checks#
- Postgres:
SELECT 1against the isis database. - Redis (when dispatching):
PINGreturns PONG. - Composer wiring: run
composer.test.tsagainst fixtures — if 13 tests pass, the composer's public API contract is green.