# 00 — Context and Rationale

## What started this

A user pointed at two external resources:

1. **neilsonnn/image-blaster** (https://github.com/neilsonnn/image-blaster) — a
   Claude skill that converts a single image into a complete 3D environment in
   ~5 minutes. It orchestrates three external services:
   - **World Labs Marble** (`marble-1.1`) — image → explorable Gaussian-splat
     environment, exported as `.spz`
   - **Hunyuan3D** via FAL — foreground objects → `.glb`/`.obj` meshes
   - **ElevenLabs SFX** — ambient sound + per-object audio cues
2. **HN discussion item 48150069** — discussion of image-blaster, with
   substantive comments from World Labs personnel and from `toisanji` reporting
   World Labs Marble's "hallucinates outside input scope" failure mode and that
   gpt-image-2 produces tighter image edits.

The question: what should we adopt into Isis? A first pass identified four
candidates:

| #   | Adoption                                                         | Status                                                             |
| --- | ---------------------------------------------------------------- | ------------------------------------------------------------------ |
| 1   | World Labs Marble provider in `libs/isis/3d-generation`          | ✅ shipped — `ce9a3e9a02`                                          |
| 2   | `.spz` exporter in `libs/isis/gaussian-splatting/exporters`      | deferred — picked up in [block D](./block-plans/D-spz-exporter.md) |
| 3   | Input-grounding QA gate in `libs/isis/3d-quality-gates/delivery` | ✅ shipped — `8524d7e13b`                                          |
| 4   | End-to-end "scene-from-image" composer                           | this folder                                                        |

## Why this folder exists

Adoption #4 — the composer that fuses environment + objects + ambient SFX behind
one call with operator checkpoints — is the headline win. Image-blaster is ~200
lines of TS gluing three external paid APIs. The value to Isis is not the
implementation; it's the **workflow shape** combined with our existing provider
portfolio (Marble, Hunyuan, Tripo, Meshy, Rodin, Trellis, ThreeDFY, ElevenLabs
SFX, Suno, Udio).

When we started planning the composer, two things became apparent:

1. The work depends on infrastructure that **doesn't yet exist** in Isis: an
   operator-checkpoint primitive in `operation-orchestrator`, a splat↔mesh
   frame-bridging helper, mesh-to-Gaussians rasterisation, lighting consistency
   between baked-radiance splats and PBR meshes, the `.spz` codec we previously
   deferred. Each of these is a real engineering block, not just glue.

2. The user explicitly asked for the **thorough, robust, long-term** version
   over a scope-reduced MVP. A lean composer that emits "splat + meshes side by
   side with no spatial relationship" is ~2 weeks but produces a
   forever-asterisk artefact that every consumer has to assemble at runtime and
   that has lighting mismatch baked in. The thorough version produces a single
   fully-baked `.spz` that loads and renders correctly anywhere a Niantic-spec
   splat viewer exists, ~18–23 weeks.

The thorough scope is too large for a single plan file. This folder breaks it
into nine work blocks, each with its own `block-plans/<letter>-*.md` doc, and a
single `ISIS_GAPS_TODOS.md` checklist that drives execution.

## Strategic intent

The composer is the headline use case but the underlying primitives serve a
broader purpose. Specifically:

- The **operator-checkpoint primitive** (block A) unlocks every multi-phase Isis
  operation — not just the composer. The lipsync pipeline, cosmetic pipeline,
  and venue capture pipeline all live in `apps/v2/` and would benefit from the
  same primitive.
- The **`.spz` codec** (block D) belongs to `gaussian-splatting`, not to the
  composer. It enables Isis to interoperate with any splat viewer, ingest
  third-party splats, and round-trip splat artefacts through external tools.
- The **mesh-to-Gaussians bake** (block E) plus **lighting consistency** (block
  F) form a reusable splat-composition kernel. Future use cases (procedural city
  generation, cinematic shot composition, AR scene placement) build on the same
  kernel.
- **Job-envelope dispatch** (block G), **state persistence** (block H), and
  **observability/policy/consensus** (block I) are platform-level integrations
  that any sufficiently large Isis operation should adopt; the composer is just
  the first one to actually need them all at once.

So the composer is the forcing function. The actual deliverables are the
primitives that the composer uses.

## What this folder is not

- A scope-reduced MVP plan. We considered and explicitly rejected the ~2-week
  lean version. See [`03-decisions.md`](./03-decisions.md) ADR-004 for the
  reasoning.
- A timeline commitment. Estimates in `06-rollout-plan.md` are sized work, not
  promised dates.
- A replacement for in-conversation iteration. Open questions are tracked in
  `07-risks-and-open-questions.md` and explicitly flagged where they need user
  input before a block can start.

## Memory and conventions in scope

The user's persistent memory captures repo-wide conventions that apply here.
Most relevant:

- `oshun_commit_policy` — commit + push to **branch and main** after every
  completed task.
- `oshun_task_verification` — checkbox is the sole truth; one task at a time;
  adversarial scan before signoff.
- `oshun_nx_broken` — bypass Nx with `npx vitest run` and `npx tsc --noEmit`
  from the library directory.
- `oshun_precommit_lint` — eslint blocks on unused imports + duplicate imports;
  commit subject must be lowercase.
- `oshun_stub_scanner_strictness` — pre-commit scan flags
  "would"/"placeholder"/"todo" tokens; mark legitimate hits with
  `// stub:legitimate <reason>` on the same line.
- `feedback_stub_legitimate_honesty` — never annotate a real tracked stub as
  legitimate to bypass pre-commit; solve it or split the commit.

These all apply throughout the work this folder plans.
