Owner library: libs/isis/scene-from-image-composer (consumer);
libs/isis/job-envelope (provider; minor extensions). Estimate: ~1–2 weeks
Blocks: running composer at production scale. Dependencies: block B
(composer phases must exist before they can be dispatched); block H (state
persistence — composer must survive BFF restart while waiting on dispatched
jobs). Status: not started
Overview#
Heavy phases (Marble call, mesh generation, splat bake) currently call providers
in-process from the BFF. At scale this pins the BFF event loop on GPU work and
prevents horizontal scaling. Block G wires every heavy phase through
@isis/job-envelope to the existing GPU worker pool over Redis.
The composer continues to look the same to its callers — composeSceneFromImage
remains an awaitable Promise — but internally it submits typed envelopes, waits
on job-id futures, and reconstructs phase outputs from worker responses.
Why this is needed#
- A single Marble call is 60–300s; a single hybrid mesh bake at 200 steps is ~5min on GPU. With multiple objects per scene + multiple concurrent composer runs, the BFF cannot host the work.
- Job-envelope already exists with a Zod schema covering
gaussian-splatting,mesh-processing,text-to-3d,image-to-3d(libs/isis/job-envelope/src/job-envelope.ts:8-23). The plumbing is in place; block G wires the composer into it. - Production needs autoscaling GPU workers separate from the BFF. The job-envelope pattern is how every other heavy Isis pipeline is dispatched.
Files to create#
Composer-side dispatcher#
libs/isis/scene-from-image-composer/src/dispatch/job-dispatcher.ts— wraps a phase function so it dispatches an envelope instead of running in-process. Returns a Promise that resolves when the worker emits a matching completion event.libs/isis/scene-from-image-composer/src/dispatch/job-dispatcher.test.tslibs/isis/scene-from-image-composer/src/dispatch/redis-job-channel.ts— thin abstraction overioredispub/sub; abstracted for testability.libs/isis/scene-from-image-composer/src/dispatch/redis-job-channel.test.tslibs/isis/scene-from-image-composer/src/dispatch/types.ts—DispatchedPhase,JobChannel,JobChannelConfig.
Worker-side handlers#
The GPU worker pool is an existing service (not part of this folder); the composer ships handler functions that workers can register. These live in the composer lib so the contract is co-located with the dispatcher.
libs/isis/scene-from-image-composer/src/dispatch/handlers/environment-handler.ts— consumes an envelope, calls Marble in-process within the worker.libs/isis/scene-from-image-composer/src/dispatch/handlers/mesh-handler.tslibs/isis/scene-from-image-composer/src/dispatch/handlers/bake-handler.ts- One
*.test.tsper handler.
Files to modify#
libs/isis/job-envelope/src/job-envelope.ts:- Extend
IsisGenerationTypewith'scene-from-image-bake'(block E output is conceptually different fromgaussian-splatting). - Add typed payload shapes per generation type via discriminated union in a
new
payloads.ts.
- Extend
libs/isis/scene-from-image-composer/src/types.ts:- Add
dispatchMode: 'in-process' | 'job-envelope'toSceneFromImageRequest(default'in-process'for v1; production sets'job-envelope'). - Add
dispatchConfig?: { redisUrl: string; channelPrefix?: string }.
- Add
libs/isis/scene-from-image-composer/src/composer.ts:- When
dispatchMode === 'job-envelope', replace direct phase function calls withjobDispatcher.dispatch(envelope).
- When
libs/isis/scene-from-image-composer/package.json:- Add
ioredis(workspace catalog).
- Add
Public API surface#
// dispatch/types.ts
export interface JobChannelConfig {
readonly redisUrl: string;
readonly channelPrefix?: string; // default 'isis:scene-from-image'
readonly responseTimeoutMs?: number; // default 30 * 60_000 (30 min)
}
export interface DispatchedPhase<TInput, TOutput> {
dispatch(input: TInput): Promise<TOutput>;
}
// dispatch/job-dispatcher.ts
export class JobDispatcher {
constructor(channel: JobChannel, generationType: IsisGenerationType);
dispatch<TInput, TOutput>(
envelope: IsisJobEnvelope<TInput>
): Promise<TOutput>;
}
// dispatch/handlers/*-handler.ts
export interface PhaseHandler<TInput, TOutput> {
readonly generationType: IsisGenerationType;
handle(envelope: IsisJobEnvelope<TInput>): Promise<TOutput>;
}
export const environmentPhaseHandler: PhaseHandler<
EnvironmentPhaseInput,
EnvironmentPhaseOutput
>;
export const meshPhaseHandler: PhaseHandler<MeshPhaseInput, GeneratedMesh>;
export const bakePhaseHandler: PhaseHandler<BakePhaseInput, BakedGaussians>;
Algorithm#
JobDispatcher.dispatch(envelope):
jobId = envelope.id
// Subscribe to response channel BEFORE publishing — avoids race.
responsePromise = channel.subscribeOnce(`${prefix}:response:${jobId}`)
channel.publish(`${prefix}:request:${envelope.generationType}`, envelope)
response = await Promise.race([
responsePromise,
timeout(config.responseTimeoutMs, throwTimeoutError(jobId))
])
if response.status === 'failed': throw JobFailedError(response.error)
return response.output
Worker registration (the GPU worker process imports the handlers and listens on the request channel):
// inside the GPU worker process — pseudocode, not part of this lib
for handler in [environmentPhaseHandler, meshPhaseHandler, bakePhaseHandler]:
channel.subscribe(`${prefix}:request:${handler.generationType}`, async (envelope) => {
try {
output = await handler.handle(envelope)
channel.publish(`${prefix}:response:${envelope.id}`, { status: 'completed', output })
} catch (err) {
channel.publish(`${prefix}:response:${envelope.id}`, { status: 'failed', error: err.message })
}
})
Test plan#
Unit (job-dispatcher.test.ts):
- Dispatch an envelope, fixture channel echoes a completed response → resolves with output.
- Fixture channel echoes a failed response → rejects with
JobFailedError. - Fixture channel never responds → rejects with
JobTimeoutErrorafter configured timeout. - Two concurrent dispatches with different jobIds → both resolve in arbitrary order.
- Same jobId dispatched twice → second resolves once first's response is
received (or reject as
DuplicateJobIdError; recommend reject).
Unit (redis-job-channel.test.ts):
- Mock
ioredisclient; verify subscribe-then-publish ordering on dispatch. - Verify publish is called with the JSON-serialised envelope.
Unit (per handler):
environmentPhaseHandlerwith a stub Marble provider in scope → producesEnvironmentPhaseOutput.meshPhaseHandlerwith stub Hunyuan → producesGeneratedMesh.bakePhaseHandlerwith stubbakeMesh→ producesBakedGaussians.
Integration (composer.dispatch-mode.test.ts):
- Composer with
dispatchMode: 'job-envelope'and a fixture channel that routes envelopes through the in-process handlers → produces same output asdispatchMode: 'in-process'. The dispatch indirection is invisible to callers.
Adversarial:
- Dispatch with malformed envelope (Zod schema fail) → throws before publish.
- Channel publish throws → propagates as
JobChannelError. - Worker handler throws → propagated as
JobFailedErrorwith original message.
Acceptance criteria#
- All new tests pass.
npx tsc --noEmitclean.- Stub-indicator scan clean.
- Composer can be configured to dispatch via Redis without code changes beyond config.
- Manual QA: a real composer run against a real Redis + a single locally- run worker process completes end-to-end. Recorded in PR with logs.
Open questions#
- Redis channel prefix conventions. Recommend
isis:scene-from-image:<generation-type>:<request|response>:<jobId>. Aligns with existing Isis Redis conventions if any; if none documented, block G defines them. - Worker pool sizing. Out of scope for this lib; handled by ops/k8s. Documentation note in composer README.
- Backpressure. What happens when the worker pool is saturated? Default
behaviour: dispatcher waits up to
responseTimeoutMs(30 min); caller times out. Future: queue-depth check before dispatch; reject early if backlog > N. - Job priority.
IsisJobPriorityenum exists. Composer should set per-phase priority (env →normal, bake →highif a checkpoint is waiting, etc.). Defer to v1.5; v1 sets all tonormal.