Eve library

00 — Context and Rationale

Other

Revision 1

On this page

What started this#

A user pointed at two external resources:

  1. neilsonnn/image-blaster (https://github.com/neilsonnn/image-blaster) — a Claude skill that converts a single image into a complete 3D environment in ~5 minutes. It orchestrates three external services:
    • World Labs Marble (marble-1.1) — image → explorable Gaussian-splat environment, exported as .spz
    • Hunyuan3D via FAL — foreground objects → .glb/.obj meshes
    • ElevenLabs SFX — ambient sound + per-object audio cues
  2. HN discussion item 48150069 — discussion of image-blaster, with substantive comments from World Labs personnel and from toisanji reporting World Labs Marble's "hallucinates outside input scope" failure mode and that gpt-image-2 produces tighter image edits.

The question: what should we adopt into Isis? A first pass identified four candidates:

# Adoption Status
1 World Labs Marble provider in libs/isis/3d-generation ✅ shipped — ce9a3e9a02
2 .spz exporter in libs/isis/gaussian-splatting/exporters deferred — picked up in block D
3 Input-grounding QA gate in libs/isis/3d-quality-gates/delivery ✅ shipped — 8524d7e13b
4 End-to-end "scene-from-image" composer this folder

Why this folder exists#

Adoption #4 — the composer that fuses environment + objects + ambient SFX behind one call with operator checkpoints — is the headline win. Image-blaster is ~200 lines of TS gluing three external paid APIs. The value to Isis is not the implementation; it's the workflow shape combined with our existing provider portfolio (Marble, Hunyuan, Tripo, Meshy, Rodin, Trellis, ThreeDFY, ElevenLabs SFX, Suno, Udio).

When we started planning the composer, two things became apparent:

  1. The work depends on infrastructure that doesn't yet exist in Isis: an operator-checkpoint primitive in operation-orchestrator, a splat↔mesh frame-bridging helper, mesh-to-Gaussians rasterisation, lighting consistency between baked-radiance splats and PBR meshes, the .spz codec we previously deferred. Each of these is a real engineering block, not just glue.

  2. The user explicitly asked for the thorough, robust, long-term version over a scope-reduced MVP. A lean composer that emits "splat + meshes side by side with no spatial relationship" is ~2 weeks but produces a forever-asterisk artefact that every consumer has to assemble at runtime and that has lighting mismatch baked in. The thorough version produces a single fully-baked .spz that loads and renders correctly anywhere a Niantic-spec splat viewer exists, ~18–23 weeks.

The thorough scope is too large for a single plan file. This folder breaks it into nine work blocks, each with its own block-plans/<letter>-*.md doc, and a single ISIS_GAPS_TODOS.md checklist that drives execution.

Strategic intent#

The composer is the headline use case but the underlying primitives serve a broader purpose. Specifically:

  • The operator-checkpoint primitive (block A) unlocks every multi-phase Isis operation — not just the composer. The lipsync pipeline, cosmetic pipeline, and venue capture pipeline all live in apps/v2/ and would benefit from the same primitive.
  • The .spz codec (block D) belongs to gaussian-splatting, not to the composer. It enables Isis to interoperate with any splat viewer, ingest third-party splats, and round-trip splat artefacts through external tools.
  • The mesh-to-Gaussians bake (block E) plus lighting consistency (block F) form a reusable splat-composition kernel. Future use cases (procedural city generation, cinematic shot composition, AR scene placement) build on the same kernel.
  • Job-envelope dispatch (block G), state persistence (block H), and observability/policy/consensus (block I) are platform-level integrations that any sufficiently large Isis operation should adopt; the composer is just the first one to actually need them all at once.

So the composer is the forcing function. The actual deliverables are the primitives that the composer uses.

What this folder is not#

  • A scope-reduced MVP plan. We considered and explicitly rejected the ~2-week lean version. See 03-decisions.md ADR-004 for the reasoning.
  • A timeline commitment. Estimates in 06-rollout-plan.md are sized work, not promised dates.
  • A replacement for in-conversation iteration. Open questions are tracked in 07-risks-and-open-questions.md and explicitly flagged where they need user input before a block can start.

Memory and conventions in scope#

The user's persistent memory captures repo-wide conventions that apply here. Most relevant:

  • oshun_commit_policy — commit + push to branch and main after every completed task.
  • oshun_task_verification — checkbox is the sole truth; one task at a time; adversarial scan before signoff.
  • oshun_nx_broken — bypass Nx with npx vitest run and npx tsc --noEmit from the library directory.
  • oshun_precommit_lint — eslint blocks on unused imports + duplicate imports; commit subject must be lowercase.
  • oshun_stub_scanner_strictness — pre-commit scan flags "would"/"placeholder"/"todo" tokens; mark legitimate hits with // stub:legitimate <reason> on the same line.
  • feedback_stub_legitimate_honesty — never annotate a real tracked stub as legitimate to bypass pre-commit; solve it or split the commit.

These all apply throughout the work this folder plans.