# Block B · Scene-From-Image Composer Library

**Owner library:** `libs/isis/scene-from-image-composer` (new) **Estimate:** ~1
week on top of block A **Blocks:** none directly; downstream blocks (C, E, F, G,
H, I) extend the composer's surface. **Dependencies:** block A. **Status:** not
started

## Overview

The composer library itself. Defines the public API (`composeSceneFromImage`,
`createSceneFromImagePipeline`), the typed phase contracts, the artifact types
(`SceneFromImageRequest`, `SceneFromImageResult`), the splat-mesh frame-bridging
helper, the manifest-extension shape, and the in-process default orchestrator
configuration with auto-approving checkpoints.

This block targets the composer's **MVP slice**: explicit-prompt object mode
only, transform-only frame bridging (no bake), in-process providers, no GPU
dispatch. Later blocks layer the additional capabilities on top without changing
the public API contract.

## Files to create

### Library scaffolding

- `libs/isis/scene-from-image-composer/package.json`
- `libs/isis/scene-from-image-composer/tsconfig.json`
- `libs/isis/scene-from-image-composer/project.json`
- `libs/isis/scene-from-image-composer/README.md`
- `libs/isis/scene-from-image-composer/src/index.ts`

### Public API

- `libs/isis/scene-from-image-composer/src/types.ts` — `SceneFromImageRequest`,
  `SceneFromImageResult`, `ComposerError` discriminated union, `PhaseId` enum,
  `ObjectSpec` (placeholder; real one lands in block C).
- `libs/isis/scene-from-image-composer/src/composer.ts` — implements
  `composeSceneFromImage` (convenience) and `createSceneFromImagePipeline`
  (advanced).
- `libs/isis/scene-from-image-composer/src/composer.test.ts`

### Phase implementations

- `libs/isis/scene-from-image-composer/src/phases/environment-phase.ts` — calls
  `MarbleProvider.generateFromImage`, returns `EnvironmentPhaseOutput`.
- `libs/isis/scene-from-image-composer/src/phases/grounding-phase.ts` — calls
  `evaluate3DQualityInputGroundingGate`, returns `GroundingPhaseOutput`.
- `libs/isis/scene-from-image-composer/src/phases/objects-phase.ts` — iterates
  over `ObjectSpec[]`, calls the configured mesh provider for each, returns
  `ObjectsPhaseOutput`.
- `libs/isis/scene-from-image-composer/src/phases/align-phase.ts` — computes
  per-object 4×4 transforms; returns `AlignPhaseOutput`.
- `libs/isis/scene-from-image-composer/src/phases/audio-phase.ts` — generates
  ambient + per-object SFX via `SFXProvider`, returns `AudioPhaseOutput`.
- `libs/isis/scene-from-image-composer/src/phases/manifest-phase.ts` — builds
  the `OutputManifest` via `@isis/outputs`'s `manifest-builder.ts`, returns the
  final `SceneFromImageResult`.
- One `*.test.ts` per phase file.

### Frame-bridging helper

- `libs/isis/scene-from-image-composer/src/align/align-mesh-to-splat-frame.ts`
- `libs/isis/scene-from-image-composer/src/align/align-mesh-to-splat-frame.test.ts`
- `libs/isis/scene-from-image-composer/src/align/anchor-spec.ts` — `AnchorSpec`
  discriminated union and `resolveAnchor` helper (block B ships only the simple
  variants; block C extends with constraint-solver variants).

### Pipeline definition + orchestrator integration

- `libs/isis/scene-from-image-composer/src/pipeline/build-pipeline.ts` —
  produces a `PipelineDefinition` (consumable by block A's
  `PipelineOrchestrator`) from a `SceneFromImageRequest`.
- `libs/isis/scene-from-image-composer/src/pipeline/in-process-orchestrator.ts`
  — thin convenience wrapper for the `composeSceneFromImage` API; spins up an
  ephemeral `PipelineOrchestrator` with auto-approving checkpoints and awaits
  completion.
- `libs/isis/scene-from-image-composer/src/pipeline/build-pipeline.test.ts`

### Manifest extension types

- `libs/isis/scene-from-image-composer/src/manifest-extension.ts` —
  `ScenecomposerManifestMetadata` (typed shape under
  `OutputManifest.metadata.sceneFromImage`), validator, exported re-exports.

## Files to modify

- `tsconfig.base.json` — add path mapping
  `"@isis/scene-from-image-composer": ["libs/isis/scene-from-image-composer/src/index.ts"]`.
- `pnpm-workspace.yaml` — auto-picks up the new lib; verify.

## Public API surface

```ts
// types.ts (public)

export interface SceneFromImageRequest {
  readonly sourceImage: Buffer;
  readonly mimeType?: string; // 'image/png' default

  // Provider injection — composer doesn't construct providers itself.
  readonly providers: {
    readonly marble: MarbleProvider;
    readonly mesh?: I3DProvider; // Hunyuan/Tripo/Meshy/etc.
    readonly sfx?: ISFXProvider;
  };

  // QA gate adapter. If omitted, grounding phase is skipped.
  readonly groundingAdapter?: ThreeDQualityGroundingVlmAdapter;

  // What objects to add. Defaults to []. Block C extends the union.
  readonly objects?: readonly ObjectSpec[];

  // Ambient-SFX prompt. If omitted, derived from Marble's ambient_prompt.
  readonly ambient?: { readonly prompt: string; readonly durationSec?: number };

  // Per-phase opt-out.
  readonly phases?: {
    readonly skipGrounding?: boolean;
    readonly skipObjects?: boolean;
    readonly skipAudio?: boolean;
  };

  // Orchestrator hooks.
  readonly checkpoints?: Partial<Record<PhaseId, CheckpointPolicy>>;

  // Policy context (block I).
  readonly policyContext?: PolicyContext;

  // Identity/tenancy.
  readonly assetId: string;
  readonly tenantId?: string;
  readonly userId?: string;
}

export type PhaseId =
  | 'environment'
  | 'grounding'
  | 'objects'
  | 'align'
  | 'audio'
  | 'manifest';

export interface SceneFromImageResult {
  readonly manifest: OutputManifest;
  readonly phaseOutputs: {
    readonly environment: EnvironmentPhaseOutput;
    readonly grounding?: GroundingPhaseOutput;
    readonly objects?: ObjectsPhaseOutput;
    readonly align?: AlignPhaseOutput;
    readonly audio?: AudioPhaseOutput;
  };
  readonly checkpointHistory: readonly CheckpointDecision[];
  readonly metrics: {
    readonly totalDurationMs: number;
    readonly perPhaseMs: Record<PhaseId, number>;
  };
}

// composer.ts (public)

export async function composeSceneFromImage(
  input: SceneFromImageRequest
): Promise<SceneFromImageResult>;

export function createSceneFromImagePipeline(
  input: SceneFromImageRequest
): PipelineDefinition;
```

## Phase contracts

Each phase is an async function `(ctx: PhaseContext) => Promise<PhaseOutput>`
where `PhaseContext` carries the request + outputs of all preceding phases

- a logger + (block I) a policy hook.

```ts
// phases/environment-phase.ts
export interface EnvironmentPhaseOutput {
  readonly splatScene: Generated3DModel; // format: 'spz'
  readonly bounds: {
    readonly min: [number, number, number];
    readonly max: [number, number, number];
  };
  readonly previewUrls: readonly string[];
  readonly ambientPromptSuggestion?: string;
}

// phases/grounding-phase.ts
export interface GroundingPhaseOutput {
  readonly evaluation: ThreeDQualityInputGroundingEvaluation;
  readonly decision: ThreeDQualityInputGroundingDecision;
}

// phases/objects-phase.ts
export interface ObjectsPhaseOutput {
  readonly meshes: readonly GeneratedMesh[]; // one per ObjectSpec
}
export interface GeneratedMesh {
  readonly objectSpec: ObjectSpec;
  readonly model: Generated3DModel; // format: 'glb' typically
  readonly localBboxMeters: {
    readonly width: number;
    readonly height: number;
    readonly depth: number;
  };
}

// phases/align-phase.ts
export interface AlignPhaseOutput {
  readonly placements: readonly Placement[];
}
export interface Placement {
  readonly meshIndex: number; // index into ObjectsPhaseOutput.meshes
  readonly transform: Float32Array; // 16-element column-major 4×4
  readonly anchor: AnchorSpec;
}

// phases/audio-phase.ts
export interface AudioPhaseOutput {
  readonly ambient?: GeneratedSFX;
  readonly perObject: ReadonlyArray<{
    readonly meshIndex: number;
    readonly sfx: GeneratedSFX;
  }>;
}
```

## Frame-bridging algorithm (`align-mesh-to-splat-frame.ts`)

Inputs:

- `splatBoundsMeters: { min: [n,n,n]; max: [n,n,n] }` — Marble world bounds.
- `meshLocalBbox: { width, height, depth }` — from
  `getDocumentStats(parseGLB(modelData))`.
- `anchor: AnchorSpec` — where in the splat to put the mesh.
- `desiredHeightMeters?: number` — optional. If omitted, derived from anchor
  type (e.g. "centre-ground" defaults to 1m for visibility).

Output:

- `Float32Array(16)` — column-major 4×4 transform M such that for any local
  vertex `v`, `Mv` is the world-space placement.

Steps:

1. **Scale.** `s = desiredHeightMeters / meshLocalBbox.height`. (If
   `desiredHeightMeters` is omitted, default to the anchor's
   `defaultHeightMeters`.)
2. **Translation.** Compute `worldAnchor: [x, y, z]` from `anchor` and
   `splatBoundsMeters`:
   - `centre-ground`: `[(min.x+max.x)/2, min.y, (min.z+max.z)/2]`
   - `on-floor-front-centre`: `[(min.x+max.x)/2, min.y, max.z - 0.5]`
   - `{ x, y, z }`: as supplied.
3. **Mesh anchor offset.** Default mesh anchor is `bottom-centre` (y = 0, x = z
   = bbox centre in mesh-local frame). Compute scaled mesh-local bottom-centre:
   `[(scaled bbox centre x), 0, (scaled bbox centre z)]`.
4. **Translation matrix.** `t = worldAnchor - meshLocalBottomCentre * s`.
5. **Rotation.** v1 assumes Y-up for both Marble and GLB (Hunyuan, Tripo, Meshy,
   Rodin all default to Y-up GLB). For now: identity rotation. Future: detect
   axis convention from GLB `asset.extras` if present, fall back to identity.
6. **Compose.** `M = T * R * S` written in column-major.

Validation:

- Throws `AlignmentError` if `meshLocalBbox.height <= 0` (degenerate mesh).
- Throws if `splatBoundsMeters` is inverted (max < min) on any axis.

## In-process orchestrator

```ts
// pipeline/in-process-orchestrator.ts (sketch)

export async function runInProcess(
  input: SceneFromImageRequest
): Promise<SceneFromImageResult> {
  const orchestrator = createPipelineOrchestrator({
    enableCheckpoints: true,
    checkpointPolicy: input.checkpoints
      ? mergeWithDefaultPolicy(input.checkpoints)
      : { timeoutMs: 5 * 60_000, onTimeout: 'approve' },
  });
  const definition = createSceneFromImagePipeline(input);
  const pipeline = orchestrator.runPipeline(definition);
  return await collectResult(orchestrator, pipeline.id);
}
```

## Test plan

**Unit per phase:**

- Each phase test mocks the providers/adapter it depends on and asserts the
  output shape against fixture data. No network calls.
- `align-phase.test.ts`: deterministic transforms — synthetic 10×10×10 splat
  bounds + unit-cube mesh + `centre-ground` anchor produces
  `scale=1, translation=[0, -5, 0], rotation=I` to within FP epsilon. Multiple
  anchor variants tested (10+ cases).

**Integration (`composer.test.ts`):**

- Full happy path with all stub providers: produces a `SceneFromImageResult`
  with `manifest.files.length === 4` (splat + 1 mesh + ambient + 1 preview),
  `metadata.sceneFromImage.placements.length === 1`, and exactly the expected
  provenance records.
- Skip-objects mode: `manifest.files.length === 2` (splat + ambient).
- Skip-grounding mode: `phaseOutputs.grounding === undefined`.
- Skip-audio mode: `phaseOutputs.audio === undefined`.
- Failure: Marble provider throws → composer throws `ComposerError` with
  `phase: 'environment'`.
- Checkpoint rejection: handler returns `rejected` after `environment` phase →
  composer throws `ComposerError` with `phase: 'environment'`,
  `cause: 'checkpoint-rejected'`, no downstream phases run.
- Modification flow: handler returns `modified` with overlay
  `{ ambient: { prompt: 'override' } }` → audio phase receives the override.

**Adversarial:**

- Empty `objects: []` does not break align phase.
- Empty preview URLs from Marble: thumbnail unset, no crash.
- `groundingAdapter` omitted → grounding phase skipped silently.
- Manifest extension validator rejects malformed `placements` arrays.

## Acceptance criteria

- All new tests pass: `npx vitest run` in `libs/isis/scene-from-image-composer`.
- `npx tsc --noEmit` clean.
- Stub-indicator scan clean on all new files.
- The composer library has zero direct fetch/network calls outside provider
  injection (verified by adversarial grep).
- README documents the convenience and advanced APIs with one runnable snippet
  each.
- `tsconfig.base.json` path-mapping verified by importing
  `@isis/scene-from-image-composer` from a sibling lib.

## Open questions

- **Default mesh anchor.** Bottom-centre seems right but Hunyuan in practice
  produces some meshes that are not bbox-aligned (fur poking out, etc.). Should
  v1 use bbox-bottom-centre or geometric-bottom-centre? Start with
  bbox-bottom-centre; revisit if visual QA fails.
- **Checkpoint default per phase.** v1 ships `auto-approve-after: 5min` for
  every phase. Block I changes the default per environment (production vs CI vs
  local).
- **`MarbleProvider` lives in `3d-generation`** — composer needs to import the
  provider type. Adds a circular-import risk. Resolve by importing only
  `I3DProvider` from `3d-generation/types` in the composer, and `MarbleProvider`
  only in the test fixtures and the documentation. The caller injects a
  constructed provider.
