Eve library

Block D · `.spz` Exporter

Other

Revision 1

On this page

Owner library: libs/isis/gaussian-splatting Estimate: ~1–3 weeks (depending on Niantic spec ambiguities surfaced) Blocks: E (output format), I (gallery — splat assets in gallery need a portable format). Dependencies: none directly; can run in parallel with block A. Status: not started — picks up the deferred task #3 from the original adoption work.

Overview#

Implement a bit-exact Niantic .spz codec (encoder + decoder) in libs/isis/gaussian-splatting/src/exporters/. Validate against the Niantic reference loader by round-tripping a corpus of reference splats.

The current exporter (isis-splat-exporter.ts) writes a proprietary .isplat format for Unreal/Unity engines and stays as a separate surface. Block D adds .spz alongside, not in place of.

Why this is needed#

  • Niantic's .spz is the de facto interop format for Gaussian splats. Web viewers (gsplat.js, mkkellogg/GaussianSplats3D), engine plugins (Unreal LumaAI plugin, Unity gsplat-unity), and SDK loaders all consume it.
  • The Marble provider already returns .spz as opaque bytes. To produce composed splats (block E) we need to encode .spz from in-memory Gaussians, not just decode.
  • Block I (gallery) needs splats in a format that web viewers can render inline; .spz is the choice.

Files to create#

  • libs/isis/gaussian-splatting/src/exporters/spz-exporter.ts — encoder.
  • libs/isis/gaussian-splatting/src/exporters/spz-exporter.test.ts
  • libs/isis/gaussian-splatting/src/exporters/spz-decoder.ts — decoder (needed for round-trip tests and for ingesting Marble's .spz output as in-memory Gaussians during the bake step).
  • libs/isis/gaussian-splatting/src/exporters/spz-decoder.test.ts
  • libs/isis/gaussian-splatting/src/exporters/spz-binary-layout.ts — the binary layout constants and quantisation helpers, shared by encoder and decoder.
  • libs/isis/gaussian-splatting/src/exporters/spz-binary-layout.test.ts — exhaustive quantisation round-trip tests for each field.
  • libs/isis/gaussian-splatting/test-fixtures/spz-reference/ — directory containing 5–10 reference .spz files downloaded from public sources (Niantic samples, viewer demos), plus per-file expected metadata (gaussian count, sh degree, bounds) for assertion targets.

Files to modify#

  • libs/isis/gaussian-splatting/src/exporters/index.ts — export SpzExporter, SpzDecoder, types.
  • libs/isis/gaussian-splatting/package.json — add a pako dependency for gzip (already available in the workspace pnpm catalog as a transitive, but needs an explicit declaration here).

Niantic .spz spec summary#

Public spec: https://github.com/nianticlabs/spz. The format is gzipped. Once decompressed, the layout is:

Offset (uncompressed) Field Size Notes
0 magic u32 LE 0x5053474e ("NGSP" reversed; Niantic Gaussian Splat)
4 version u32 LE currently 2
8 numPoints u32 LE gaussian count
12 shDegree u8 0..3
13 fractionalBits u8 for fixed-point positions; default 12
14 flags u8 bit 0: antialiased
15 reserved u8 always 0

After the 16-byte header, sections follow in this order:

  1. Positions: numPoints * 3 * 24-bit signed fixed-point. Decoded as value = signed24 / (1 << fractionalBits) per axis.
  2. Alphas: numPoints * u8. Decoded as alpha = sigmoid((u8 / 255 - 0.5) * SCALE) where SCALE = 8.0 per the Niantic reference.
  3. Colors: numPoints * 3 * u8. SH0-derived 8-bit RGB.
  4. Scales: numPoints * 3 * u8. Decoded as scale = exp((u8 - 64) / 16.0) per axis (log-scale quantisation).
  5. Rotations: numPoints * 3 * u8. Three components of a unit quaternion (w reconstructed). Decoded as xyz = (u8 - 127.5) / 127.5; w = sqrt(max(0, 1 - x*x - y*y - z*z)).
  6. SH coefficients: numPoints * shCoefficientsPerGaussian * u8 where shCoefficientsPerGaussian = (degree+1)^2 * 3 - 3 (skipping SH0 which was already written as colors). Decoded as coeff = (u8 - 128) * (1.0 / 128.0).

The whole concatenated buffer is then gzipped.

Note. Confirm every quantisation formula and offset against the Niantic reference loader before declaring done. The exact constants (SCALE = 8.0 for alpha, (u8 - 64) / 16.0 for scale, (u8 - 127.5) / 127.5 for rotation) are sourced from public references but must be verified on a known-good .spz round trip.

Public API surface#

ts
// spz-exporter.ts

export interface SpzExportOptions {
  readonly fractionalBits?: number; // default 12
  readonly antialiased?: boolean; // default false
  readonly shDegree?: 0 | 1 | 2 | 3; // inferred from input if omitted
}

export class SpzExporter {
  encode(scene: LoadedScene, options?: SpzExportOptions): Promise<Buffer>;
}

export function createSpzExporter(): SpzExporter;

// spz-decoder.ts

export interface SpzDecodeResult {
  readonly scene: LoadedScene;
  readonly metadata: {
    readonly version: number;
    readonly fractionalBits: number;
    readonly antialiased: boolean;
    readonly shDegree: number;
  };
}

export class SpzDecoder {
  decode(buffer: Buffer): Promise<SpzDecodeResult>;
}

export function createSpzDecoder(): SpzDecoder;

LoadedScene is the existing in-memory representation from splat-renderer.ts:114-117 (positions, scales, rotations, opacities, colors, optional SH).

Algorithms#

Encode#

text
encode(scene, options):
  1. validate scene shape (positions length === 3*N, scales === 3*N, etc.)
  2. write 16-byte header:
     magic          = 0x5053474e
     version        = 2
     numPoints      = scene.gaussianCount
     shDegree       = options.shDegree ?? inferShDegree(scene)
     fractionalBits = options.fractionalBits ?? 12
     flags          = options.antialiased ? 0x01 : 0x00
     reserved       = 0
  3. write positions section (signed 24-bit fixed-point per axis)
     for each i in 0..N:
       x_q = round(scene.positions[3i + 0] * (1 << fractionalBits))
       check fits in [-2^23, 2^23 - 1]; throw on overflow
       writeInt24LE(buffer, offset, x_q); offset += 3
       (repeat for y, z)
  4. write alphas (u8 per point) — invert sigmoid + scale to u8
  5. write colors (u8 RGB per point)
  6. write scales (u8 per axis) — log-scale quantise
  7. write rotations (u8 xyz per point) — drop w, sign-bit from rotation[3]
  8. write SH coefficients (u8 per coefficient)
  9. gzip the entire buffer (level 6, default)
  10. return gzipped buffer

Decode#

text
decode(gzippedBuffer):
  1. ungzip
  2. read header (16 bytes); validate magic
  3. allocate Float32Arrays for positions/scales/rotations/colors/opacities/sh
  4. read positions section, dequantise signed-24 → float
  5. read alphas, dequantise u8 → sigmoid space
  6. read colors, dequantise u8 → [0, 1]
  7. read scales, dequantise u8 → exp(.)
  8. read rotations, dequantise u8 → unit quaternion (reconstruct w)
  9. read SH if shDegree > 0
  10. return { scene: LoadedScene, metadata }

inferShDegree#

text
inferShDegree(scene):
  if !scene.sphericalHarmonics: return 0
  shPerGaussian = scene.sphericalHarmonics.length / scene.gaussianCount
  // shPerGaussian = (deg+1)^2 * 3 - 3 → solve for deg
  // deg = sqrt((shPerGaussian + 3) / 3) - 1
  return ROUND_AND_VALIDATE_INT(...) ?? throw new Error('SH length invalid')

Test plan#

Unit (spz-binary-layout.test.ts):

  • Quantisation/dequantisation round-trips for each field type (position fixed-point at fractionalBits ∈ {8, 10, 12, 14}, scale log-quantisation, rotation unit-quaternion sign-bit reconstruction). Each assert known expected u8/u24 outputs against synthetic input values.
  • Position overflow at fractionalBits=12 throws (input value 2^11 + ε).
  • Rotation w-reconstruction handles edge case xyz exactly on unit sphere (w = 0).

Unit (spz-exporter.test.ts):

  • Encode 100-Gaussian synthetic scene → round-trip via SpzDecoder → every field within quantisation tolerance.
  • Encode the same scene at fractionalBits ∈ {10, 12, 14} produces different file sizes (lower = smaller, expected ratios asserted).
  • Encode with shDegree: 0 produces a file with no SH section.
  • Encode with antialiased: true sets bit 0 of flags byte (verified by ungzipping and reading byte 14).

Reference corpus (spz-decoder.test.ts):

  • For each file in test-fixtures/spz-reference/:
    • Decode → assert metadata.version === 2, metadata.fractionalBits === <expected>, scene.gaussianCount === <expected> (per-file sidecar JSON of expected values).
    • Re-encode → bit-exact match against the original (gzipped) bytes. Asserts our encoder produces output indistinguishable from Niantic's.
  • This is the definition of done for the codec: byte-for-byte match.

Adversarial:

  • Decoder rejects bad magic (mid-byte corruption).
  • Decoder rejects inconsistent header (numPoints * fields > buffer length).
  • Encoder rejects scene with mismatched array lengths.
  • Encoder rejects scene with SH length not matching shDegree formula.

Acceptance criteria#

  • npx vitest run src/exporters/ clean.
  • npx tsc --noEmit clean from libs/isis/gaussian-splatting.
  • Stub-indicator scan clean.
  • Bit-exact encode round-trip for every reference fixture in test-fixtures/spz-reference/.
  • A composer-produced .spz (synthetic scene) loads in the mkkellogg/GaussianSplats3D web viewer without errors. Manual verification step; recorded in the PR description with a screenshot.
  • The decoder correctly ingests a Marble-produced .spz (real one downloaded from a Marble run) and reproduces the same Gaussian count and bounds Marble reported via metadata.

Open questions#

  • Reference corpus sourcing. Need 5–10 known-good .spz files. Sources: Niantic's published demos, World Labs Marble outputs, public splat galleries. License audit needed for each — the test fixtures may not redistribute. Alternative: synthesise reference splats with the encoder, then assert decoder handles them correctly. Less rigorous but copyright-safe.
  • Quantisation constants verification. The exact alpha/scale/rotation decode formulas are sourced from public references but not from the Niantic source directly (license precludes lifting code). v1 assumes the constants are correct; if visual QA shows artifacts, re-check against the Niantic header/source comments before changing math.
  • pako vs Node zlib. Both work; pako is browser-friendly if we ever ship a worker version. Recommended: pako (already in workspace catalog).
  • Streaming encode/decode. For large splats (>50M Gaussians) the buffer-based API allocates a lot. v1 ships buffer-based; streaming is a v2 nice-to-have.