Eve library

Block G · Job-Envelope and GPU Dispatch

Other

Revision 1

On this page

Owner library: libs/isis/scene-from-image-composer (consumer); libs/isis/job-envelope (provider; minor extensions). Estimate: ~1–2 weeks Blocks: running composer at production scale. Dependencies: block B (composer phases must exist before they can be dispatched); block H (state persistence — composer must survive BFF restart while waiting on dispatched jobs). Status: not started

Overview#

Heavy phases (Marble call, mesh generation, splat bake) currently call providers in-process from the BFF. At scale this pins the BFF event loop on GPU work and prevents horizontal scaling. Block G wires every heavy phase through @isis/job-envelope to the existing GPU worker pool over Redis.

The composer continues to look the same to its callers — composeSceneFromImage remains an awaitable Promise — but internally it submits typed envelopes, waits on job-id futures, and reconstructs phase outputs from worker responses.

Why this is needed#

  • A single Marble call is 60–300s; a single hybrid mesh bake at 200 steps is ~5min on GPU. With multiple objects per scene + multiple concurrent composer runs, the BFF cannot host the work.
  • Job-envelope already exists with a Zod schema covering gaussian-splatting, mesh-processing, text-to-3d, image-to-3d (libs/isis/job-envelope/src/job-envelope.ts:8-23). The plumbing is in place; block G wires the composer into it.
  • Production needs autoscaling GPU workers separate from the BFF. The job-envelope pattern is how every other heavy Isis pipeline is dispatched.

Files to create#

Composer-side dispatcher#

  • libs/isis/scene-from-image-composer/src/dispatch/job-dispatcher.ts — wraps a phase function so it dispatches an envelope instead of running in-process. Returns a Promise that resolves when the worker emits a matching completion event.
  • libs/isis/scene-from-image-composer/src/dispatch/job-dispatcher.test.ts
  • libs/isis/scene-from-image-composer/src/dispatch/redis-job-channel.ts — thin abstraction over ioredis pub/sub; abstracted for testability.
  • libs/isis/scene-from-image-composer/src/dispatch/redis-job-channel.test.ts
  • libs/isis/scene-from-image-composer/src/dispatch/types.ts — DispatchedPhase, JobChannel, JobChannelConfig.

Worker-side handlers#

The GPU worker pool is an existing service (not part of this folder); the composer ships handler functions that workers can register. These live in the composer lib so the contract is co-located with the dispatcher.

  • libs/isis/scene-from-image-composer/src/dispatch/handlers/environment-handler.ts — consumes an envelope, calls Marble in-process within the worker.
  • libs/isis/scene-from-image-composer/src/dispatch/handlers/mesh-handler.ts
  • libs/isis/scene-from-image-composer/src/dispatch/handlers/bake-handler.ts
  • One *.test.ts per handler.

Files to modify#

  • libs/isis/job-envelope/src/job-envelope.ts:
    • Extend IsisGenerationType with 'scene-from-image-bake' (block E output is conceptually different from gaussian-splatting).
    • Add typed payload shapes per generation type via discriminated union in a new payloads.ts.
  • libs/isis/scene-from-image-composer/src/types.ts:
    • Add dispatchMode: 'in-process' | 'job-envelope' to SceneFromImageRequest (default 'in-process' for v1; production sets 'job-envelope').
    • Add dispatchConfig?: { redisUrl: string; channelPrefix?: string }.
  • libs/isis/scene-from-image-composer/src/composer.ts:
    • When dispatchMode === 'job-envelope', replace direct phase function calls with jobDispatcher.dispatch(envelope).
  • libs/isis/scene-from-image-composer/package.json:
    • Add ioredis (workspace catalog).

Public API surface#

ts
// dispatch/types.ts

export interface JobChannelConfig {
  readonly redisUrl: string;
  readonly channelPrefix?: string; // default 'isis:scene-from-image'
  readonly responseTimeoutMs?: number; // default 30 * 60_000 (30 min)
}

export interface DispatchedPhase<TInput, TOutput> {
  dispatch(input: TInput): Promise<TOutput>;
}

// dispatch/job-dispatcher.ts

export class JobDispatcher {
  constructor(channel: JobChannel, generationType: IsisGenerationType);
  dispatch<TInput, TOutput>(
    envelope: IsisJobEnvelope<TInput>
  ): Promise<TOutput>;
}

// dispatch/handlers/*-handler.ts

export interface PhaseHandler<TInput, TOutput> {
  readonly generationType: IsisGenerationType;
  handle(envelope: IsisJobEnvelope<TInput>): Promise<TOutput>;
}

export const environmentPhaseHandler: PhaseHandler<
  EnvironmentPhaseInput,
  EnvironmentPhaseOutput
>;
export const meshPhaseHandler: PhaseHandler<MeshPhaseInput, GeneratedMesh>;
export const bakePhaseHandler: PhaseHandler<BakePhaseInput, BakedGaussians>;

Algorithm#

text
JobDispatcher.dispatch(envelope):
  jobId = envelope.id
  // Subscribe to response channel BEFORE publishing — avoids race.
  responsePromise = channel.subscribeOnce(`${prefix}:response:${jobId}`)
  channel.publish(`${prefix}:request:${envelope.generationType}`, envelope)
  response = await Promise.race([
    responsePromise,
    timeout(config.responseTimeoutMs, throwTimeoutError(jobId))
  ])
  if response.status === 'failed': throw JobFailedError(response.error)
  return response.output

Worker registration (the GPU worker process imports the handlers and listens on the request channel):

text
// inside the GPU worker process — pseudocode, not part of this lib
for handler in [environmentPhaseHandler, meshPhaseHandler, bakePhaseHandler]:
  channel.subscribe(`${prefix}:request:${handler.generationType}`, async (envelope) => {
    try {
      output = await handler.handle(envelope)
      channel.publish(`${prefix}:response:${envelope.id}`, { status: 'completed', output })
    } catch (err) {
      channel.publish(`${prefix}:response:${envelope.id}`, { status: 'failed', error: err.message })
    }
  })

Test plan#

Unit (job-dispatcher.test.ts):

  • Dispatch an envelope, fixture channel echoes a completed response → resolves with output.
  • Fixture channel echoes a failed response → rejects with JobFailedError.
  • Fixture channel never responds → rejects with JobTimeoutError after configured timeout.
  • Two concurrent dispatches with different jobIds → both resolve in arbitrary order.
  • Same jobId dispatched twice → second resolves once first's response is received (or reject as DuplicateJobIdError; recommend reject).

Unit (redis-job-channel.test.ts):

  • Mock ioredis client; verify subscribe-then-publish ordering on dispatch.
  • Verify publish is called with the JSON-serialised envelope.

Unit (per handler):

  • environmentPhaseHandler with a stub Marble provider in scope → produces EnvironmentPhaseOutput.
  • meshPhaseHandler with stub Hunyuan → produces GeneratedMesh.
  • bakePhaseHandler with stub bakeMesh → produces BakedGaussians.

Integration (composer.dispatch-mode.test.ts):

  • Composer with dispatchMode: 'job-envelope' and a fixture channel that routes envelopes through the in-process handlers → produces same output as dispatchMode: 'in-process'. The dispatch indirection is invisible to callers.

Adversarial:

  • Dispatch with malformed envelope (Zod schema fail) → throws before publish.
  • Channel publish throws → propagates as JobChannelError.
  • Worker handler throws → propagated as JobFailedError with original message.

Acceptance criteria#

  • All new tests pass.
  • npx tsc --noEmit clean.
  • Stub-indicator scan clean.
  • Composer can be configured to dispatch via Redis without code changes beyond config.
  • Manual QA: a real composer run against a real Redis + a single locally- run worker process completes end-to-end. Recorded in PR with logs.

Open questions#

  • Redis channel prefix conventions. Recommend isis:scene-from-image:<generation-type>:<request|response>:<jobId>. Aligns with existing Isis Redis conventions if any; if none documented, block G defines them.
  • Worker pool sizing. Out of scope for this lib; handled by ops/k8s. Documentation note in composer README.
  • Backpressure. What happens when the worker pool is saturated? Default behaviour: dispatcher waits up to responseTimeoutMs (30 min); caller times out. Future: queue-depth check before dispatch; reject early if backlog > N.
  • Job priority. IsisJobPriority enum exists. Composer should set per-phase priority (env → normal, bake → high if a checkpoint is waiting, etc.). Defer to v1.5; v1 sets all to normal.