# @isis/3d-inference-local Self-hosted 3D inference primitives for the Isis domain. This library is the local runtime kernel for Phase 71. It provides: - Nx project metadata and build/test/lint wiring - strongly typed contracts for local 3D model descriptors, GPU server config, queueing, warm-up, and lifecycle management - provider interoperability helpers so self-hosted models can plug into the existing `@isis/3d-generation` provider surface - disk-aware model weight caching with access tracking, cache inspection, and LRU eviction for large local model installs - per-GPU VRAM budget management with pressure detection and automatic model offloading hooks for local multi-model serving - capability-aware multi-model scheduling that selects, loads, reuses, and unloads text-to-3D, image-to-3D, and texturing models based on incoming request mix - a starvation-aware request queue with interactive, batch, and background priority classes to prepare later batching and multi-GPU dispatch - a batch aggregation layer that groups queue-compatible 3D requests into throughput-oriented activation batches using a configurable aggregation window - a multi-GPU load balancer that routes requests across VRAM-aware schedulers while honoring soft affinity and hard per-model GPU pinning - explicit VRAM-tier placement that keeps ~6GB models on consumer GPUs, routes 24GB-class workloads onto A100/H100-style accelerators when available, and skips GPUs that can never satisfy a model's VRAM floor - a model prewarm manager that keeps hot models resident in VRAM, demotes cold models into disk-backed offloaded state for fast reloads, and plugs directly into the existing warmup-handler surface on the local GPU model server - a queue-depth autoscaler that watches 3D generation backlog, chooses the highest-priority RunPod inference/shared pool with spare headroom, and emits threshold/cooldown-aware scale-up requests through a provisioner callback - spot/preemptible-capacity-aware pool routing so batch/background 3D jobs prefer cheaper interruptible GPU pools with automatic on-demand fallback while interactive traffic stays on stable on-demand capacity by default - a GPU utilization monitor that correlates time-windowed utilization samples with scheduler residency, raises lifecycle-aware alerts for underutilized GPUs and idle resident model instances, and exposes poll/reconcile APIs for external alert delivery - a GPU node-pool manager that keeps 3D inference capacity isolated from training workloads, tracks Kubernetes/RunPod/manual pool membership, and exposes pool-aware node selection plus heartbeat-backed capacity snapshots - Meshy-compatible SSE progress streams with resumable `Last-Event-ID` replay, task-scoped stream aliases, and typed progress publishing APIs - a unified provider-oriented REST API for capabilities, text/image job submission, status polling, cancellation, per-job streams, and result asset downloads - package-local CUDA container assets for NVIDIA Container Toolkit deployment, including a Docker build, compose stack, and env-driven runtime bootstrap - a package-local Helm chart for Kubernetes GPU deployments with NVIDIA node selectors, GPU resource requests/limits, persistent model/cache volumes, and optional runtime-module mounts - RunPod serverless adapters that proxy the local Isis-compatible REST/SSE API onto RunPod `/run`, `/runsync`, `/status`, `/cancel`, and `/health` endpoints for scale-to-zero operation - a built-in cost dashboard that tracks self-hosted cost per asset and compares it against configurable cloud API pricing profiles such as Meshy plans - a dependency-aware TRELLIS.2 loader that registers the official Microsoft checkpoints, resolves the cross-repo sparse-structure decoder dependency, and produces bf16/fp16 runtime load plans with low-VRAM guidance - a runnable TRELLIS text-to-3D mesh pipeline that patches the official Microsoft text-conditioned checkpoint bundle into a fully local runtime bundle, executes the upstream Python pipeline, and emits staged progress for text conditioning, sparse-structure sampling, SLat sampling, and mesh extraction - a runnable TRELLIS.2 single-image reconstruction pipeline that patches the official Microsoft image-conditioned checkpoint bundle into a fully local runtime bundle and emits staged progress for preprocessing, image conditioning, sparse-structure sampling, shape SLat sampling, and mesh extraction - a runnable TRELLIS.2 multi-image reconstruction pipeline that accepts 2-6 reference views, fuses the strongest views into one conditioning sequence, and returns raw mesh output with explicit selected-view metadata - a runnable TRELLIS.2 PBR extraction pipeline that bakes `albedo`, `metallic`, `roughness`, `alpha`, and combined metallic-roughness textures from the official TRELLIS.2 texturing pipeline - a runnable TRELLIS.2 Gaussian splatting pipeline that projects TRELLIS.2 mesh+appearance outputs into GraphDECO-compatible `.ply` splats for real-time viewing workflows - a runnable TRELLIS.2 GLB/GLTF export pipeline that packages textured meshes as single-file GLB or embedded-resource GLTF assets - preset-driven TRELLIS.2 quality controls that map `draft`, `standard`, `high`, and `ultra` onto released pipeline resolutions, extraction grids, and token/step budgets - a TRELLIS.2 prompt adapter that converts Isis prompt templates into object-centric reference-image prompts and structured guidance metadata for the released image-conditioned runtime - deterministic TRELLIS.2 seed control with normalized replayable seeds across mesh, texturing, GLTF, and Gaussian export paths - TRELLIS.2 negative prompt support that converts Isis exclusions into upstream reference-image avoidance guidance for the released image-conditioned runtime - TRELLIS.2 style conditioning presets for `photorealistic`, `stylized`, `low-poly`, and `cartoon` prompt-to-image guidance - a deterministic TRELLIS.2 benchmark suite that scores local quality against Meshy 6, Rodin Gen-1.5, and Tripo v2.5 commercial baselines with weighted case metrics and threshold gates - TRELLIS.2 update automation that polls Microsoft GitHub, falls back from releases to tags to branch-head commits when needed, downloads fresh runtime source archives, and syncs the paired Hugging Face checkpoints - a Hunyuan3D 2.1 shape loader that registers Tencent’s official `tencent/Hunyuan3D-2.1` checkpoint bundle, restricts downloads to the shape subfolder, and resolves standard, low-VRAM, and 6GB mmgp load plans with offline runtime hints - a Hunyuan3D 2.1 MMGP coordinator that stages text, shape, paint, super-resolution, and bake phases through the existing GPU memory manager so 6GB-class consumer GPUs can execute the published Hunyuan flows sequentially - a Hunyuan3D 2.1 Blender addon integration that installs as a single `.py` addon, submits text/image jobs through the local provider API, polls job status, and imports completed models back into the active Blender scene - a Hunyuan3D 2.1 text-to-shape pipeline that follows Tencent’s published flow of HunyuanDiT prompt-to-image generation, optional background removal, and local Hunyuan3D 2.1 shape reconstruction with preview-image capture - a Hunyuan3D 2.1 image-to-shape pipeline for direct image-conditioned shape reconstruction, including background-normalized preview capture, quality presets, and explicit anchor-image selection when callers provide multiple references to a single-image-conditioned runtime - Hunyuan3D 2.1 batch generation for text-to-shape and image-to-shape runs, with compatibility-aware micro-batching on standard-memory GPUs and honest warm-sequential fallback when low-VRAM or mmgp staging is required - Hunyuan3D 2.1 mesh topology control for geometry-only OBJ output, including a documented PyMeshLab quad-dominant post-process on top of Tencent’s released triangle mesh generation path - a Hunyuan3D 2.1 style fine-tuning pipeline that prepares Tencent-compatible HunyuanDiT LoRA datasets, drives the official native DeepSpeed trainer, and emits reusable adapter manifests for local text-to-shape runs - an SF3D loader that registers Stability AI’s gated `stable-fast-3d` weights, resolves consumer-GPU `fp16` and native `bf16` load plans, and documents the compiled UV unwrap / texture baking runtime surface needed for later sub-second image-to-3D execution - a SPAR3D loader that registers Stability AI’s gated `stable-point-aware-3d` weights, resolves standard vs low-VRAM execution plans, and exposes the editable 512-point `xyzrgb` intermediate representation used between point diffusion and final mesh extraction - a Sparc3D loader for operator-supplied private pretrained bundles, with local-filesystem or private Hugging Face mirror sources, bundle-manifest validation, and explicit 1024³ readiness checks instead of pretending public checkpoints exist - a Sparc3D image-to-3D pipeline that executes an operator-supplied Python entrypoint from the imported private bundle, scales between 256³, 512³, and 1024³ resolution plans for quality-vs-speed tradeoffs, and returns provider-compatible GLB/OBJ/PLY assets - strict Sparc3D watertight-manifold validation that independently inspects OBJ and ASCII PLY topology, rejects meshes with holes or non-manifold edges in `surfaceMode: 'watertight'`, and records print-ready diagnostics for 3D printing workflows - explicit Sparc3D open-surface analysis that preserves thin sheets and disconnected shells as first-class outputs, requires topology sidecars for GLB exports, and records boundary/component diagnostics instead of forcing everything into closed solids - a deterministic Sparc3D geometric-fidelity benchmark that compares the local Sparc3D stack against local TRELLIS.2 and Hunyuan3D 2.1 baselines on watertight, open-surface, thin-structure, and printable reconstruction cases - a MeshAnything V2 topology-cleanup handoff for Sparc3D that retopologizes the operator mesh through the released `main.py` CLI, preserves OBJ analysis sidecars for validation, and rechecks watertight/open-surface guarantees after cleanup instead of trusting the post-process blindly - a Kaolin FlexiCubes mesh-extraction path that can remesh finalized geometry from TRELLIS text/image/multi-image, Hunyuan3D 2.1 text/image/batch shape runs, TripoSG image/scribble/multi-view, and Sparc3D, while explicitly rejecting texture-first SF3D and SPAR3D requests where remeshing would discard baked materials - automatic MeshAnything V2 post-processing across TRELLIS text/image/multi-image, Hunyuan3D 2.1 text/image/batch shape runs, Sparc3D cleanup handoffs, and the TripoSG family whenever callers ask for topology controls those native runtimes do not honestly expose on their own - a TripoSG 1.5B loader that registers the official `VAST-AI/TripoSG` rectified-flow weights plus the required `briaai/RMBG-1.4` background-removal dependency, preserves the upstream 8GB CUDA floor, and models a more realistic ~12GB operating envelope for end-to-end local runs - a runnable TripoSG image-to-3D pipeline that follows the released RMBG foreground-isolation and `TripoSGPipeline` sampling path, supports step / guidance / seed / face-count control, and returns geometry-only GLB, OBJ, or PLY assets - a runnable TripoSG-scribble sketch-to-3D pipeline that follows the released CFG-distilled prompt+scribble conditioning path, exposes scribble/prompt confidence controls, and returns rapid-prototyping geometry assets without enabling the flash decoder that upstream flags for boundary artifacts - a TripoSG multi-view consistency mode that honestly layers on top of the published single-image pipeline by batching per-view reconstructions and reranking them with geometric medoid scoring instead of claiming a non-public fused multi-view checkpoint - a TripoSG vs TripoSR comparison harness that runs the local TripoSG surface against the repo’s legacy Tripo baseline and returns an explicit validation report with verdict and confidence instead of hand-wavy quality claims - a runnable SPAR3D single-image pipeline that follows the released image-to-point-cloud-to-mesh path, exports the intermediate `points.ply` artifact alongside the textured GLB, and supports seed plus guidance control for the point diffusion stage - a runnable SPAR3D point-cloud editing pipeline that accepts generated or uploaded `ply` point clouds, applies translation / scale / recolor edits to the released 512-point `xyzrgb` representation, and remeshes the edited cloud back into a textured GLB - a persistent SPAR3D preview loop that keeps the released runtime loaded, reuses the prepared image and current point cloud across iterations, and regenerates interactive preview meshes from edited point clouds without paying cold-start model load cost on every edit - a runnable SF3D single-image pipeline that follows Stability AI’s released background-removal, foreground-cropping, `run_image(...)`, and textured-GLB export path while surfacing precision downgrades and textured completion assets through the local provider contract - a speed-optimized SF3D batch image-to-3d pipeline that uses the released list-image `run_image(...)` path, compatibility-aware micro-batching, and VRAM-capped throughput heuristics to target 100+ images/minute on fast-path settings - a quality-vs-speed router that automatically keeps latency-sensitive image requests on SF3D, promotes fidelity-sensitive requests onto TRELLIS.2, and composes TRELLIS geometry with the released GLB/GLTF export stage so the quality route still returns delivery-ready assets - an SF3D UV unwrap export path that derives a UV layout artifact directly from the official Stability AI unwrapped mesh and returns it as a provider texture asset alongside the textured GLB - an SF3D illumination-disentangled texture export that surfaces the released albedo texture directly from the model’s material decoder, alongside normal, roughness, and metallic outputs needed for downstream PBR workflows Subsequent Phase 71 tasks build on this package for containerization, GPU orchestration, model-specific runtime integrations, and cost tracking. ## TRELLIS.2 Loader `Local3DTrellis2ModelLoader` is the first model-specific runtime integration in the package. It wraps the existing Hugging Face-backed `Local3DModelRegistry` and bakes in the official Microsoft upstream layout: - `microsoft/TRELLIS.2-4B` for the main image-to-3D and texturing checkpoints - `microsoft/TRELLIS-image-large` for the sparse-structure decoder still referenced by `pipeline.json` - runtime-side dependencies on `facebook/dinov3-vitl16-pretrain-lvd1689m` and `briaai/RMBG-2.0` Example: ```ts import { Local3DModelRegistry, createLocal3DTrellis2ModelLoader, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const trellis = createLocal3DTrellis2ModelLoader({ registry }); const plan = await trellis.resolveLoadPlan({ precision: 'fp16', gpuDevice: { id: 0, computeCapability: '8.9', vramBudgetMb: 24_576, }, }); ``` The resolved plan includes: - the installed main and sparse-structure checkpoint locations - pipeline metadata from `pipeline.json` and `texturing_pipeline.json` - per-component native precision and runtime precision - automatic bf16 validation against GPU compute capability - `flash-attn` vs `xformers` attention backend selection - low-VRAM operating mode defaults and estimated runtime VRAM requirements The default runtime profile is `bf16` with an estimated 16GB VRAM requirement. Requesting `fp16` switches the loader into low-VRAM mode and targets an 8GB operating profile by downcasting the bf16 flow checkpoints at load time while keeping the native fp16 decoder components unchanged. ## Hunyuan3D 2.1 Shape Loader The package now also exposes `Local3DHunyuan3D21ModelLoader` for Tencent’s official Hunyuan3D 2.1 shape-generation runtime: ```ts import { Local3DModelRegistry, createLocal3DHunyuan3D21ModelLoader, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const hunyuan = createLocal3DHunyuan3D21ModelLoader({ registry }); const plan = await hunyuan.resolveLoadPlan({ allowMemoryDowngrade: true, enableFlashVdm: true, gpuDevice: { id: 0, computeCapability: '8.9', vramBudgetMb: 12_288, }, }); ``` This loader is grounded in Tencent’s published Hunyuan3D 2.1 sources and model card: - the Hugging Face repo `tencent/Hunyuan3D-2.1` publishes the shape DiT subfolder, optional ShapeVAE files, and the separate PBR paint bundle - the official runtime entrypoint is `Hunyuan3DDiTFlowMatchingPipeline.from_pretrained(...)` - Tencent’s README states a 10GB VRAM target for shape generation, while the upstream runtime also exposes `low_vram_mode` and CPU offload hooks The resolved load plan includes: - selective download of only `hunyuan3d-dit-v2-1/config.yaml` and `hunyuan3d-dit-v2-1/model.fp16.ckpt` so the loader does not pull the 21GB paint bundle just to run shape inference - offline runtime hints for `from_pretrained`, including the local model path, subfolder, config path, checkpoint path, and the upstream `conditioner->model->vae` CPU offload order - three memory profiles: `standard` (10GB), `low-vram` (8GB with `enable_model_cpu_offload`), and `mmgp` (6GB staged profile coordinated by `Local3DHunyuan3D21MmgpCoordinator`) - auxiliary artifact metadata for the published ShapeVAE files so later fine-tuning and advanced VAE-specific workflows can reuse the same registry install metadata ## Hunyuan3D 2.1 Text-to-Shape Pipeline The package now also exposes a runnable local text-to-shape pipeline on top of the Hunyuan loader: ```ts import { Local3DModelRegistry, createLocal3DHunyuan3D21TextToShapePipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const pipeline = createLocal3DHunyuan3D21TextToShapePipeline({ registry, runtimeRoot: '/opt/Hunyuan3D-2.1', allowMemoryDowngrade: true, }); const result = await pipeline.run({ prompt: 'weathered bronze astrolabe', negativePrompt: 'busy background', meshTopology: 'quad-dominant', styleAdapterPath: '/models/hunyuan-style-loras/porcelain-style/adapter-manifest.json', styleAdapterScale: 0.8, quality: 'high', seed: 7, }); ``` This matches Tencent’s published text workflow: - the Hunyuan3D 2.1 gradio app enables text-to-3D by first generating a reference image from `Tencent-Hunyuan/HunyuanDiT-v1.1-Diffusers-Distilled` - the generated image then goes through optional background removal before `Hunyuan3DDiTFlowMatchingPipeline.from_pretrained(...)` reconstructs the shape - the Diffusers HunyuanDiT pipeline supports direct `negative_prompt`, `num_inference_steps`, and `guidance_scale` controls, which Oshun maps from the text request surface At runtime the Node pipeline: - downloads the distilled HunyuanDiT Diffusers bundle plus the previously defined Hunyuan3D 2.1 shape checkpoint into `Local3DModelRegistry` - resolves the shape loader first so low-VRAM and 6GB mmgp fallbacks carry into the text flow automatically - invokes a package-local Python runner that emits staged progress for text-to-image loading, image generation, background removal, shape sampling, mesh extraction, and final asset writes - can fuse Tencent-native HunyuanDiT style LoRA adapters into the Diffusers text-to-image stage by pointing `styleAdapterPath` at either a checkpoint directory containing `adapter_model.safetensors` or the manifest emitted by the fine-tuning pipeline below - defaults quad-dominant requests to `obj` and runs PyMeshLab `meshing_tri_to_quad_dominant` as an explicit post-process, because Tencent’s released runtime still emits triangle meshes - routes `targetPolyCount` requests through an automatic MeshAnything V2 post-process after shape extraction so face-budget control is enforced instead of being treated as metadata-only intent - returns both the generated mesh and the reference PNG used for reconstruction, so callers can persist the exact image that conditioned the final shape ## Hunyuan3D 2.1 Style LoRA Fine-Tuning Oshun now also packages a native HunyuanDiT LoRA fine-tuning pipeline for project-specific asset styles. This task is grounded in Tencent’s official `HunyuanDiT` LoRA workflow rather than an ad-hoc Diffusers training loop: ```ts import { Local3DModelRegistry, createLocal3DHunyuan3D21FineTuningPipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const pipeline = createLocal3DHunyuan3D21FineTuningPipeline({ registry, hyditTrainingRoot: '/opt/HunyuanDiT', }); const result = await pipeline.run({ adapterName: 'Porcelain Style', triggerPhrase: 'porcelain style', examples: trainingExamples, validationPrompts: ['garden pavilion', 'lotus flower'], rank: 64, maxTrainingSteps: 2_000, }); ``` The fine-tuning pipeline: - downloads Tencent’s native `Tencent-Hunyuan/HunyuanDiT-v1.1` `t2i/model` checkpoint layout through `Local3DModelRegistry` so the official trainer sees the same `pytorch_model_ema.pt` and `pytorch_model_module.pt` structure it expects - writes a HunyuanDiT-compatible CSV caption file, converts it into Arrow shards via Tencent’s `hydit/data_loader/csv2arrow.py`, and emits the matching YAML index consumed by `hydit/train_deepspeed.py` - launches Tencent’s official DeepSpeed LoRA trainer with the published v1.1 style-conditioning recipe (`--use-style-cond`, `--size-cond 1024 1024`, `--beta-end 0.03`) instead of pretending the local Diffusers-distilled checkpoint can be trained natively - packages the latest checkpoint into an `adapter-manifest.json` file whose `checkpointPath` can be fed straight back into `Local3DHunyuan3D21TextToShapePipeline` through `styleAdapterPath` ## Hunyuan3D 2.1 Image-to-Shape Pipeline The package now also exposes a direct image-conditioned shape pipeline: ```ts import { Local3DModelRegistry, createLocal3DHunyuan3D21ImageToShapePipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const pipeline = createLocal3DHunyuan3D21ImageToShapePipeline({ registry, runtimeRoot: '/opt/Hunyuan3D-2.1', allowMemoryDowngrade: true, }); const result = await pipeline.run({ images: [inputImageBuffer], meshTopology: 'quad-dominant', quality: 'high', seed: 7, }); ``` This follows the released Hunyuan3D 2.1 image-conditioned runtime directly: - it resolves the same Hunyuan3D 2.1 shape loader introduced in `71.1.3.1` - it uses Tencent’s `Hunyuan3DDiTFlowMatchingPipeline.from_pretrained(...)` image path, with the same optional CPU-offload and `flashvdm` hooks - it can post-process the generated triangle mesh into a quad-dominant `obj` using PyMeshLab when callers request preserved face topology for downstream DCC workflows - it now also routes `targetPolyCount` requests through an automatic MeshAnything V2 remesh pass so face-count control works for image-conditioned runs and mixed-format outputs instead of remaining advisory - it emits the background-normalized preview image used for reconstruction so the exact conditioning input is preserved alongside the mesh As of March 26, 2026, the published `hunyuan3d-dit-v2-1/config.yaml` still targets `SingleImageEncoder` plus `ImageProcessorV2`, not Tencent’s separate multi-view preprocessor. Oshun therefore handles multi-image requests by selecting one anchor image deterministically (`first-image` or `largest-image`) and records that decision in result metadata instead of pretending the released 2.1 checkpoint performs native multi-view fusion. ## Hunyuan3D 2.1 Batch Generation The text-to-shape and image-to-shape pipelines now also expose `runBatch(...)` for throughput-oriented asset generation: ```ts const textBatch = await textPipeline.runBatch({ targetBatchSize: 4, requests: [ { prompt: 'bronze raven', outputFormat: 'glb', seed: 11 }, { prompt: 'marble lion', meshTopology: 'quad-dominant', seed: 12 }, ], }); const imageBatch = await imagePipeline.runBatch({ targetBatchSize: 4, requests: [ { images: [frontImage], outputFormat: 'glb' }, { images: [sideImage], quality: 'high' }, ], }); ``` This batching path is grounded in Tencent’s released runtime rather than a fake queue-only abstraction: - the text batch runner feeds prompt lists through the Diffusers `HunyuanDiTPipeline` and then passes the resulting image list into `Hunyuan3DDiTFlowMatchingPipeline` - the image batch runner feeds a list of preprocessed conditioning images directly into `Hunyuan3DDiTFlowMatchingPipeline`, which Tencent’s published implementation already supports - Oshun groups requests by the inference settings that truly affect tensor compatibility, keeps one Python process and one loaded runtime per compatible cohort, and only splits batches when settings like guidance, octree resolution, style LoRA, or memory mode differ - standard-memory execution uses a conservative honest cap of `2` assets per micro-batch on 24GB-class GPUs and `4` on 48GB-class GPUs; low-VRAM and mmgp execution intentionally falls back to warm-sequential processing because CPU offload and staged residency erase the benefit of resident micro-batches - per-item output formats, preview PNGs, and optional quad-dominant PyMeshLab post-processing are still preserved after the shared batched inference pass - per-item `targetPolyCount` requests are preserved too, with automatic MeshAnything V2 post-processing applied only to the specific batch items that asked for face-budget control The Hunyuan3D-Paint texture stack remains a single-mesh runtime upstream, so these batch APIs currently optimize geometry throughput first and leave texturing as an explicitly separate stage. ## Hunyuan3D-Paint Loader The package now also exposes a dedicated loader for the published `Hunyuan3D-Paint` texture-generation stack: ```ts import { Local3DModelRegistry, createLocal3DHunyuan3DPaintModelLoader, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const loader = createLocal3DHunyuan3DPaintModelLoader({ registry, runtimeRoot: '/opt/Hunyuan3D-2.1', }); const plan = await loader.resolveLoadPlan({ maxSelectedViews: 6, resolution: 2048, allowMemoryDowngrade: true, }); ``` This loader is grounded in Tencent’s published paint runtime and separates the different moving parts explicitly: - it models Tencent’s documented baseline of roughly 21GB VRAM for 6 views at a 2048-map output profile, while exposing explicit `512`, `1024`, `2048`, and `4096` texture-map targets through a runtime resolution plan - it selectively downloads only the `hunyuan3d-paintpbr-v2-1` checkpoint files plus the required local `facebook/dinov2-giant` dependency through `Local3DModelRegistry` - it surfaces the additional runtime prerequisites Tencent documents but does not host in the model repo, including the `RealESRGAN_x4plus.pth` checkpoint, the custom rasterizer build, and the mesh-painter compile script - because Tencent does not publish a dedicated low-VRAM paint switch, Oshun’s `low-vram` and `mmgp` modes are exposed as explicit staged-runtime plans with warnings instead of being misrepresented as native upstream toggles - runtime hints include the paint config path, custom diffusers pipeline path, DINO path, RealESRGAN path, renderer entrypoints, component residency strategy, the selected view/bake/downsample profile for the requested texture size, and the albedo/MR material outputs expected by the released stack ## Hunyuan3D-Paint PBR Texture Synthesis The package now also exposes a full offline texture-synthesis pipeline for the published `Hunyuan3D-Paint` runtime: ```ts import { Local3DModelRegistry, createLocal3DHunyuan3DPaintTextureSynthesisPipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const pipeline = createLocal3DHunyuan3DPaintTextureSynthesisPipeline({ registry, runtimeRoot: '/opt/Hunyuan3D-2.1', allowMemoryDowngrade: true, }); const result = await pipeline.run({ mesh: inputMeshBuffer, meshFormat: 'glb', prompt: 'weathered brass with dark patina in recessed areas', outputFormat: 'glb', textureResolution: 1024, }); ``` This pipeline follows Tencent’s released texture stack closely while fixing the gaps required for self-hosted production use: - it keeps the released `hunyuan3d-paintpbr-v2-1` diffusion model fully offline by loading the already-registered local checkpoint directory instead of calling upstream `snapshot_download(...)` at inference time - it supports both image-guided and text-guided texture synthesis; when only a text prompt is supplied, Oshun uses a neutral white style canvas as the required image-conditioning anchor and records that decision in warnings and metadata - it stages multiview diffusion, RealESRGAN enhancement, and UV baking as separate phases so the `low-vram` loader profile can offload components between steps instead of keeping the whole Tencent stack resident at once - it exposes true texture-map size control for `512`, `1024`, `2048`, and `4096` outputs; lower sizes use an oversampled bake plus deterministic downsampling, while `4096` switches into a native 4K bake path - it auto-downloads Tencent’s required `RealESRGAN_x4plus.pth` checkpoint into `hy3dpaint/ckpt/` if the runtime checkout is missing it - it emits the textured mesh plus standalone `albedo`, `metallic`, `roughness`, `metallicRoughness`, and geometry-derived `normal` maps through the standard provider completion contract - GLB output uses Tencent’s own published PBR conversion utility so the embedded material matches the exported standalone texture maps ## Hunyuan3D 2.1 Combined Shape+Texture Pipeline The package now also exposes an end-to-end orchestration pipeline that chains the released Hunyuan 2.1 shape stage into the local `Hunyuan3D-Paint` texture stage: ```ts import { Local3DModelRegistry, createLocal3DHunyuan3D21ShapeAndTexturePipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const pipeline = createLocal3DHunyuan3D21ShapeAndTexturePipeline({ registry, runtimeRoot: '/opt/Hunyuan3D-2.1', allowMemoryDowngrade: true, }); const result = await pipeline.run({ prompt: 'weathered bronze astrolabe', outputFormat: 'glb', styleAdapterPath: '/models/hunyuan-style-loras/porcelain-style/adapter-manifest.json', }); ``` This combined pipeline is grounded in Tencent’s published end-to-end demo flow while staying explicit about the runtime model: - it supports both text-driven and image-driven entrypoints by delegating to the already-isolated Oshun shape pipelines and then feeding the exact emitted preview image into the paint stage as the texture-conditioning anchor - it reports both Tencent’s published co-resident combined budget (`LOCAL_3D_HUNYUAN3D21_COMBINED_STANDARD_VRAM_MB`, 29GB class) and Oshun’s actual sequential peak budget so schedulers can reason about the official full stack versus the implemented staged execution path - `mmgp` mode resolves to the documented 6GB-class sequential path by combining the existing shape and paint MMGP profiles, while `standard` mode preserves the published full-stack envelope in metadata - final provider completion metadata includes both the shape-stage and texture-stage metadata snapshots so downstream systems can audit the entire generation chain instead of only the last stage - text-mode combined runs pass `styleAdapterPath` and `styleAdapterScale` through to the HunyuanDiT reference-image stage before texturing, so a custom style LoRA still influences the geometry bootstrap even though the texture bake remains triangle-based - the textured pipeline stays triangle-based by design: quad-dominant requests are rejected here because the released Hunyuan3D-Paint UV baking path re-triangulates meshes during texturing, so Oshun only exposes preserved quad output on the standalone geometry-only shape pipelines ## Hunyuan3D 2.1 MMGP Coordinator `Local3DHunyuan3D21MmgpCoordinator` is the consumer-GPU orchestration layer for the Hunyuan 6GB path. It sits above the existing shape and paint pipelines and uses the shared `Local3DGpuMemoryManager` to keep only one heavy stage resident at a time while tracing the exact stage sequence that ran: ```ts import { createLocal3DGpuDeviceConfig, createLocal3DHunyuan3D21MmgpCoordinator, createLocal3DHunyuan3D21ShapeAndTexturePipeline, Local3DModelRegistry, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ modelsRoot: '/models', cachePolicy: { maxDiskUsageGb: 400, minFreeDiskGb: 50 }, }); const pipeline = createLocal3DHunyuan3D21ShapeAndTexturePipeline({ registry, runtimeRoot: '/opt/hunyuan3d', }); const coordinator = createLocal3DHunyuan3D21MmgpCoordinator({ gpuDevices: [ createLocal3DGpuDeviceConfig(0, { name: 'RTX 3060', vramBudgetMb: 6_144, }), ], }); const execution = await coordinator.runShapeAndTexture(pipeline, { prompt: 'weathered bronze astrolabe', memoryMode: 'mmgp', }); console.log(execution.plan.stages.map((stage) => stage.stageId)); console.log(execution.trace.peakReservedMb); ``` The coordinator exposes planning helpers for standalone text-to-shape, image-to-shape, texture synthesis, and combined shape+texture runs. For 6GB cards it automatically caps the default GPU reserve so the published MMGP envelope can use the whole device instead of inheriting the server-wide 2GB reserve intended for multi-tenant deployments. ## Hunyuan3D 2.1 Blender Addon Integration The package also ships a Blender-side integration surface for direct viewport generation. Oshun’s implementation follows Tencent’s official addon direction of embedding generation into the sidebar, but targets the existing local provider API instead of a one-off `/generate` route and uses job polling plus auto-import instead of a single blocking request. ```ts import { createLocal3DHunyuan3D21BlenderAddonIntegration } from '@isis/3d-inference-local'; const integration = createLocal3DHunyuan3D21BlenderAddonIntegration({ apiBaseUrl: 'http://127.0.0.1:4413', }); const descriptor = await integration.describeInstall(); await integration.copyAddonScript('/tmp/hunyuan3d-blender-addon'); console.log(descriptor.scriptPath); console.log(descriptor.installSteps); ``` The bundled addon: - installs as a single `hunyuan3d_2_1_blender_addon.py` file - targets `/v1/3d/capabilities`, `/v1/3d/text-to-3d/jobs`, and `/v1/3d/image-to-3d/jobs` - submits `preferredModelKey: 'hunyuan3d-2.1'` so the local server can route the request into the Hunyuan stack - exposes prompt, negative prompt, image reference, quality, output format, texturing, texture resolution, and background-removal controls directly in the sidebar - polls job state in the background, supports cancellation, downloads the final model asset, and imports `glb`, `gltf`, `obj`, or `ply` output back into a dedicated Blender collection - can auto-select and frame the imported result so the finished mesh appears in the viewport with minimal operator work ## Hunyuan3D 2.1 Multiview Reference Generation The package now also exposes a bridge pipeline that turns a single reference image into canonical front/left/back/right view images before later reconstruction stages: ```ts import { Local3DModelRegistry, createLocal3DHunyuan3D21MultiviewImageGenerationPipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const pipeline = createLocal3DHunyuan3D21MultiviewImageGenerationPipeline({ registry, runtimeRoot: '/opt/Hunyuan3D-2.1', allowMemoryDowngrade: true, }); const result = await pipeline.run({ images: [inputImageBuffer], quality: 'high', viewOrder: ['front', 'left', 'back', 'right'], }); ``` This pipeline is intentionally grounded in Tencent’s published 2.1 sources rather than inventing a fake native novel-view checkpoint: - as of March 26, 2026, Tencent’s released 2.1 stack still ships single-image shape reconstruction plus the separate `Hunyuan3D-Paint` multiview diffusion module, not a standalone single-image `front/left/back/right` generator - Oshun therefore bootstraps a coarse shape mesh with the released `Hunyuan3DDiTFlowMatchingPipeline`, renders normal and position control maps for the requested camera views, and then runs the published `hunyuan3d-paintpbr-v2-1` diffusion bundle to synthesize canonical reference images - the pipeline downloads the exact paint checkpoint files plus the local `facebook/dinov2-giant` dependency through `Local3DModelRegistry`, keeps `HF_HUB_OFFLINE=1` for the runner, returns the synthesized views as `completion.textures[]` with `reference-view-*` types, and preserves the background-normalized input preview plus the bootstrap mesh path for later debugging Because this flow uses Tencent’s published paint runtime, the runtime root still needs a built Hunyuan3D 2.1 checkout with the renderer dependencies available under `hy3dpaint/`. ## TRELLIS Text Pipeline The package now also ships a text-to-3D mesh pipeline for the official Microsoft TRELLIS text-conditioned checkpoints: ```ts import { Local3DModelRegistry, createLocal3DTrellisTextTo3DPipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const pipeline = createLocal3DTrellisTextTo3DPipeline({ registry, runtimeRoot: '/opt/TRELLIS', }); const result = await pipeline.run({ prompt: 'weathered bronze astrolabe', outputFormat: 'obj', seed: 7, }); ``` This pipeline is built from the current official upstream Microsoft sources: - `microsoft/TRELLIS-text-large` for the text-conditioned flow checkpoints - `microsoft/TRELLIS-image-large` for the shared decoders required by the text pipeline - the original `microsoft/TRELLIS` Python runtime, which currently contains the official text-conditioned pipeline implementation This is an inference from the current upstream repos and model cards: the released `TRELLIS.2` repo is image-conditioned today, while Microsoft’s text-conditioned weights remain on the original `TRELLIS` runtime. At runtime the Node pipeline: - downloads the official checkpoint bundles into `Local3DModelRegistry` - rewrites the upstream `pipeline.json` dependency references into a fully local bundle so inference can run offline from the downloaded weights - executes the official Python pipeline via a package-local runner script - emits progress stages for runtime load, text encoding, sparse-structure sampling, SLat sampling, mesh extraction, and final mesh write-out - automatically hands the extracted mesh into MeshAnything V2 when callers ask for `meshTopology` or `targetPolyCount` controls beyond the native TRELLIS extractor surface For this task the pipeline intentionally focuses on raw mesh extraction, so the supported output formats are `obj` and `ply`. Textured/PBR exports and GLB/GLTF packaging are implemented in later TRELLIS tasks. ## TRELLIS.2 Image Pipeline For single-image reconstruction, the package now exposes a TRELLIS.2 image pipeline on top of the official `trellis2` runtime: ```ts import { Local3DModelRegistry, createLocal3DTrellis2ImageTo3DPipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const pipeline = createLocal3DTrellis2ImageTo3DPipeline({ registry, runtimeRoot: '/opt/TRELLIS.2', }); const result = await pipeline.run({ images: [imageBuffer], outputFormat: 'obj', removeBackground: true, pipelineType: '1024_cascade', }); ``` At runtime the Node wrapper: - downloads `microsoft/TRELLIS.2-4B` plus the shared sparse-structure decoder from `microsoft/TRELLIS-image-large` - rewrites the upstream `pipeline.json` dependency reference into a local bundle so inference runs from registry-managed weights instead of remote downloads - executes only the geometry branch of the official TRELLIS.2 image pipeline for this task, producing raw mesh output in `obj` or `ply` - emits progress stages for image preprocessing, image feature extraction, sparse-structure sampling, shape SLat sampling, mesh extraction, and final write-out - optionally accepts `meshExtractionResolution` in 64-step increments between `64` and `512` so callers can trade extraction fidelity against decode cost This task intentionally stops at raw mesh reconstruction. Multi-image conditioning, PBR extraction, Gaussian splatting output, and GLB/GLTF export are implemented in later TRELLIS subtasks. ## TRELLIS.2 Multi-Image Pipeline For higher-fidelity reconstruction from multiple references, the package now also exposes a dedicated TRELLIS.2 multi-image pipeline: ```ts import { Local3DModelRegistry, createLocal3DTrellis2MultiImageTo3DPipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const pipeline = createLocal3DTrellis2MultiImageTo3DPipeline({ registry, runtimeRoot: '/opt/TRELLIS.2', }); const result = await pipeline.run({ images: [frontView, leftView, rightView, backView], outputFormat: 'obj', pipelineType: '1024_cascade', conditioningMode: 'concat', }); ``` At runtime the Node wrapper: - reuses the same patched local `microsoft/TRELLIS.2-4B` bundle as the single-image pipeline, so all checkpoint downloads stay registry-managed - accepts 2-6 input views, preprocesses each one independently, and passes them into a dedicated multi-image runner - fuses multiple image-conditioning feature sets into one TRELLIS conditioning sequence, with `concat` as the default strategy and `mean` as a lower-memory alternative - selects up to 4 conditioning views when more than 4 inputs are supplied, matching the current upstream training regime while still accepting 5-6 references from callers - supports the same optional `meshExtractionResolution` override, allowing lower-cost 64³-512³ extraction grids without changing the underlying TRELLIS pipeline type - automatically runs MeshAnything V2 after reconstruction when callers request `meshTopology` or `targetPolyCount`, so multi-image runs get the same honest topology-control surface as the single-image path - emits selected-view metadata so downstream systems know which input views were actually used for conditioning This is based on the current upstream TRELLIS.2 training sources: the official dataset code samples up to 4 conditioning views, while community multi-image forks demonstrate multi-view inference on top of the released TRELLIS.2 image weights. Oshun therefore accepts 2-6 inputs, then selects and fuses the strongest subset for the official checkpoint bundle. ## TRELLIS.2 PBR Material Extraction The package now also exposes a dedicated material extraction pipeline built on top of the official `texturing_pipeline.json` runtime: ```ts import { Local3DModelRegistry, createLocal3DTrellis2PbrMaterialExtractionPipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const pipeline = createLocal3DTrellis2PbrMaterialExtractionPipeline({ registry, runtimeRoot: '/opt/TRELLIS.2', }); const result = await pipeline.run({ mesh, meshFormat: 'obj', image: referenceImage, texturingResolution: 1024, textureSize: 2048, }); ``` This pipeline: - loads the official TRELLIS.2 texturing runtime via `texturing_pipeline.json` - encodes the input mesh back into shape SLat, samples texture SLat from the reference image, decodes the PBR voxel field, and rasterizes UV textures - returns typed texture assets for `albedo`, `metallic`, `roughness`, `alpha`, and a combined `metallicRoughness` map ready for later GLTF/GLB packaging - reuses the official UV-unwrapping and rasterization path from Microsoft’s texturing pipeline so the extracted maps stay aligned with upstream behavior ## TRELLIS.2 Gaussian Splatting The package now also exposes a TRELLIS.2 Gaussian splatting export pipeline for real-time viewing stacks that prefer splats over triangle meshes: ```ts import { Local3DModelRegistry, createLocal3DTrellis2GaussianSplattingPipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const pipeline = createLocal3DTrellis2GaussianSplattingPipeline({ registry, runtimeRoot: '/opt/TRELLIS.2', }); const result = await pipeline.run({ mesh, meshFormat: 'obj', image: referenceImage, texturingResolution: 1024, gaussianCount: 32768, }); ``` This pipeline: - reuses the official TRELLIS.2 texturing runtime to infer appearance from the reference image instead of inventing a separate local material model - samples the normalized TRELLIS.2 mesh surface, queries the decoded PBR voxel field at each sampled point, and fits anisotropic surface-aligned Gaussian splats - writes a GraphDECO-compatible binary `.ply` asset with SH DC color, opacity, scale, and quaternion rotation fields so existing Gaussian viewers can ingest it directly - returns the asset through the standard model-completion contract using the logical output format `gaussian-splatting` This is an inference from the current upstream sources, not a direct Microsoft feature: official TRELLIS.2 ships mesh and texturing decoders today, while the older `microsoft/TRELLIS` runtime defines the canonical Gaussian PLY schema and viewer/export path. Oshun combines those two upstream pieces so TRELLIS.2 outputs can participate in real-time Gaussian workflows immediately. ## TRELLIS.2 GLB/GLTF Export The package now also exposes a dedicated GLB/GLTF export pipeline for turning a TRELLIS.2 mesh plus reference image into a textured delivery asset: ```ts import { Local3DModelRegistry, createLocal3DTrellis2GltfExportPipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const pipeline = createLocal3DTrellis2GltfExportPipeline({ registry, runtimeRoot: '/opt/TRELLIS.2', }); const result = await pipeline.run({ mesh, meshFormat: 'obj', image: referenceImage, outputFormat: 'glb', texturingResolution: 1024, textureSize: 2048, }); ``` This pipeline: - reuses the official TRELLIS.2 texturing runtime end-to-end, including mesh preprocessing, texture latent sampling, PBR voxel decode, and textured mesh postprocessing - exports textured GLB directly through the same `trimesh`-based path shown in the official `example_texturing.py` - postprocesses GLTF export bundles into a single `.gltf` file by inlining emitted image and buffer resources as data URIs, so callers do not need sidecar texture files - returns the final model asset through the standard completion contract with `model/gltf-binary` or `model/gltf+json` content typing This is partly an inference from upstream exporter behavior: the official TRELLIS.2 example demonstrates single-file GLB export directly, while single-file embedded GLTF is assembled locally by inlining the `trimesh` exported resources into one JSON payload. ## TRELLIS.2 Quality Presets `resolveLocal3DTrellis2QualityPreset` exposes the preset table used by the single-image and multi-image TRELLIS.2 pipelines. Explicit request fields still win, but when callers provide only `quality`, Oshun now maps that preset onto a concrete TRELLIS.2 runtime configuration: - `draft`: `512` pipeline, `128³` mesh extraction, `8` sparse-structure steps, `8` shape-SLat steps - `standard`: `1024_cascade` pipeline, `256³` extraction, official released `12`-step samplers - `high`: direct `1024` pipeline, `384³` extraction, `14`-step samplers - `ultra`: `1536_cascade` pipeline, `512³` extraction, `18`-step samplers, and a larger token budget This mapping is grounded in the released TRELLIS.2 resolution tiers and model card speed table: Microsoft documents `512³`, `1024³`, and `1536³` inference targets with roughly `~3s`, `~17s`, and `~60s` runtimes on H100 hardware. ## TRELLIS.2 Prompt Adapter The released TRELLIS.2 checkpoints are image-conditioned, so prompt engineering in Oshun works by translating Isis prompt templates into guidance for the upstream image-generation step rather than pretending the runtime consumes free text directly: ```ts import { adaptLocal3DTrellis2TextRequestPrompt } from '@isis/3d-inference-local'; const adaptation = adaptLocal3DTrellis2TextRequestPrompt({ prompt: 'weathered bronze astrolabe with engraved markings', category: 'prop', style: 'photorealistic, cinematic', quality: 'high', generateTextures: true, pbrTextures: true, }); ``` The adapter emits: - a `referenceImagePrompt` tuned for clean single-object renders - a sectioned `structuredPrompt` split into subject, style, camera, lighting, background, and detail guidance - extracted material/style/detail tags so callers can inspect or postprocess the adaptation before handing it to an upstream image model - negative-prompt adaptation helpers that emit `negativeReferenceImagePrompt` guidance for excluding unwanted features before TRELLIS.2 reconstruction When callers pass `prompt` into the TRELLIS.2 single-image or multi-image mesh pipelines, Oshun now surfaces the derived `promptAdaptation` metadata in the result payload instead of silently discarding the text. When callers pass `negativePrompt`, Oshun now surfaces a `negativePromptAdaptation` payload that translates exclusions such as `busy background`, `extra limbs`, or `text watermark` into upstream reference-image avoidance guidance. This is an inference from the current upstream runtime: the released TRELLIS.2 image checkpoints still use zero-image `neg_cond`, so the negative prompt needs to act before reconstruction rather than inside the model. Style guidance is now normalized through `resolveLocal3DTrellis2StyleConditioning`. The built-in presets map Isis `style` values such as `photorealistic`, `stylized`, `low-poly`, and `cartoon` onto consistent reference-image tags, lighting cues, and detail guidance so the upstream image generation step stays aligned with the intended 3D art direction. ## TRELLIS.2 Seed Control TRELLIS.2 requests now normalize seed handling through one shared resolver: - omitting `seed` uses deterministic default seed `1` - passing an explicit integer seed replays that exact value - passing `seed: -1` requests a randomized seed, but Oshun records the resolved seed in result metadata so the run can still be replayed later The normalized `seed` and `seedSource` fields are now attached to the single-image, multi-image, PBR extraction, GLTF export, and Gaussian splatting completion metadata. ## TRELLIS.2 Benchmark Suite The package now also ships a deterministic benchmark harness for tracking whether the self-hosted TRELLIS.2 stack is actually surpassing commercial baselines on the kinds of assets where Oshun cares about quality: ```ts import { runDefaultLocal3DTrellis2BenchmarkSuite } from '@isis/3d-inference-local'; const report = runDefaultLocal3DTrellis2BenchmarkSuite({ generatedAt: '2026-03-26T00:00:00.000Z', }); ``` The benchmark report includes: - weighted per-case scoring across geometry fidelity, topology cleanliness, texture fidelity, prompt alignment, multi-view consistency, thin-structure preservation, and material realism - provider profiles for `trellis2-local`, `meshy-6`, `rodin-gen-1.5`, and `tripo-v2.5`, including first-party reference URLs that document the current commercial API surface used for calibration - case-level winner detection plus pairwise TRELLIS-vs-baseline win rates and average score margins - threshold gates so CI can fail when TRELLIS.2 regresses against any baseline The default dataset covers five representative asset classes: - hard-surface product detail - thin-structure preservation - stylized character prompt fidelity - multi-view product consistency - organic photoreal asset quality Each case is intentionally benchmarked with modality-specific weights instead of one flat score. Multi-image cases emphasize view consistency, thin-structure cases emphasize open-space preservation, and stylized cases emphasize prompt and surface fidelity. ## TRELLIS.2 Update Automation `Local3DTrellis2ModelUpdateAutomation` adds an operations-oriented update loop on top of the existing loader and registry: ```ts import { Local3DModelRegistry, createLocal3DTrellis2ModelUpdateAutomation, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const automation = createLocal3DTrellis2ModelUpdateAutomation({ registry, }); const check = await automation.checkForUpdates(); const result = await automation.pullLatest(); ``` This automation now: - checks Microsoft GitHub in priority order: latest release, latest tag, then branch-head commit when the repo has no formal releases or tags - uses conditional `If-None-Match` polling on the active detection endpoint so repeated checks can reuse the cached upstream snapshot without wasting GitHub API quota - persists update state under the registry manifests directory, including the last seen upstream snapshot, last applied snapshot, last pulled runtime archive, and last synced Hugging Face weight revisions - downloads the upstream GitHub source tarball into a local runtime-archive cache whenever a new upstream snapshot is applied - keeps Hugging Face weight sync explicit: the automation still downloads the current TRELLIS.2 and sparse-structure checkpoint revisions through `Local3DModelRegistry`, while warning that Microsoft does not currently expose a direct GitHub-release-to-Hugging-Face revision mapping As of March 26, 2026, the live Microsoft TRELLIS.2 repository does not expose a GitHub `releases/latest` payload or published tags through the public GitHub API, so this automation currently tracks `main` branch head commits in practice and upgrades automatically if formal releases or tags appear later. ## Progress Streaming `Local3DGpuModelServer` now exposes progress publication and SSE fan-out: ```ts const server = createLocal3DGpuModelServer(); server.publishProgress({ jobId: 'job-123', status: 'running', progressPct: 42, stage: 'mesh-generation', modelKey: 'trellis-2', gpuId: 0, }); ``` Clients can subscribe on the configured `binding.ssePath` and optionally filter by `jobId`: ```text GET /v1/3d/events?jobId=job-123 Last-Event-ID: 5 ``` For Meshy-style task streams, the server also accepts: ```text GET /v1/3d/events/job-123/stream ``` The stream emits `event: message` payloads shaped like Meshy's task progress updates, and terminal states include the richer task payload fields such as `type`, timestamps, `result`, and `task_error`. ## REST API `Local3DProviderRestApi` plugs into `Local3DGpuModelServer` so the same local runtime can expose job-oriented HTTP endpoints: ```ts const providerApi = createLocal3DProviderRestApi({ capabilities: createLocal3DProviderCapabilities([ 'text-to-3d', 'image-to-3d', ]), onSubmit(submission, api) { api.updateTask(submission.id, { status: 'processing', progress: 10, stage: 'bootstrapping-runtime', }); }, }); const server = createLocal3DGpuModelServer({ providerApi, }); ``` Supported routes: ```text GET /v1/3d/capabilities GET /v1/3d/costs/dashboard GET /v1/3d/quality/hunyuan-vs-meshy/dashboard GET /v1/3d/jobs GET /v1/3d/jobs/:jobId DELETE /v1/3d/jobs/:jobId GET /v1/3d/jobs/:jobId/stream GET /v1/3d/jobs/:jobId/assets/:assetId POST /v1/3d/text-to-3d/jobs POST /v1/3d/image-to-3d/jobs ``` Text submissions reuse the shared Isis-compatible request shape. Image submissions accept base64 strings or `data:` URLs, which the controller decodes back into `Buffer[]` payloads for the runtime hooks. Completed tasks can expose in-memory model, thumbnail, and texture assets as downloadable URLs while still publishing the same lifecycle over SSE. ## Container Deployment The package now includes a package-scoped GPU container build under `docker/`. The image compiles only the local inference runtime surface with TypeScript and runs it on top of an NVIDIA CUDA runtime image with Node 22: ```text docker build -f docker/Dockerfile -t oshun/isis-3d-inference-local:latest . docker run --rm --gpus all -p 4413:4413 oshun/isis-3d-inference-local:latest ``` For compose-based GPU startup, use: ```text docker compose -f docker/docker-compose.gpu.yml up --build ``` Runtime behavior is driven by environment variables such as `LOCAL_3D_CAPABILITIES`, `LOCAL_3D_GPU_DEVICES_JSON`, and `LOCAL_3D_RUNTIME_MODULE`. The container bootstrap can dynamically load a runtime module from disk, so later model-specific integrations can be mounted into the same base image without replacing the HTTP, SSE, and health-check layers. ## Kubernetes Helm Chart The package now ships with a GPU-oriented Helm chart under `helm/` for cluster deployments: ```text helm lint helm/ helm upgrade --install isis-3d-local helm/ \ --namespace isis-system \ --create-namespace ``` The chart defaults to: - scheduling only onto GPU nodes with `node-type=gpu` and `nvidia.com/gpu.present=true` - requesting and limiting one `nvidia.com/gpu` device per pod - provisioning persistent volumes for model weights and Hugging Face / Torch caches - mounting `/dev/shm` as memory-backed storage for inference workloads - exposing the runtime over a `ClusterIP` service on port `4413` For shared storage or externally supplied runtime modules, override values such as `persistence.models.existingClaim`, `persistence.cache.existingClaim`, and `runtimeModules.configMapName`/`runtimeModules.existingClaim`. ## Windows Portable Deployment `createLocal3DHunyuan3D21WindowsPortableDeployment(...)` assembles a self-contained Windows Hunyuan3D 2.1 bundle around: - a prepared `@isis/3d-inference-local` app dist - a portable Node runtime - a portable Python runtime with the Hunyuan dependencies already installed - a prepared Hunyuan3D 2.1 checkout - optional preseeded model and Hugging Face caches ```ts const deployment = createLocal3DHunyuan3D21WindowsPortableDeployment({ identity: { bundleId: 'hunyuan3d-2-1-win-portable', label: 'Hunyuan3D 2.1 Portable', version: '0.1.0', }, sources: { appDistDir: 'dist/libs/isis/3d-inference-local', nodeRuntimeDir: 'vendor/node-win-x64', pythonRuntimeDir: 'vendor/python-win-x64', hunyuanRuntimeDir: 'vendor/Hunyuan3D-2.1', modelCacheDir: 'vendor/model-cache', }, }); await deployment.assembleBundle('dist/windows/hunyuan3d-2-1-portable'); ``` The generated bundle includes: - `scripts/RUN-API.cmd` and `scripts/RUN-API.ps1` to start the Isis-compatible `/v1/3d` server with a generated Hunyuan runtime module - `scripts/RUN-GRADIO.cmd` and `scripts/RUN-GRADIO.ps1` to launch the upstream Hunyuan Gradio app when `gradio_app.py` is present in the bundled runtime - `scripts/DOCTOR.cmd` and `scripts/DOCTOR.ps1` to verify the portable runtime layout before launch - `portable-manifest.json` and `PORTABLE-README.md` documenting the assembled artifact, runtime settings, launchers, and upstream references The bundle runtime is backed by `createLocal3DHunyuan3D21RuntimeHooks(...)`, which routes local REST submissions into the text-to-shape, image-to-shape, or combined shape-plus-PBR pipelines depending on whether the request asks for textures. The launchers set the Windows-specific environment for model caches, the prepared Hunyuan runtime root, portable Python, and the generated runtime-module path so the package can boot without Docker or a system-wide Python/Node installation. This implementation is informed by the official `Tencent-Hunyuan/Hunyuan3D-2.1` Windows guidance published with the June 13, 2025 open-source release and by the existing `YanWenKun/Hunyuan3D-2-WinPortable` community pack, both reviewed on `2026-03-26`. ## SF3D Loader `Local3DSf3DModelLoader` covers Stability AI’s official Stable Fast 3D runtime layout while keeping the package honest about the consumer-GPU constraints and gated model access: ```ts import { Local3DModelRegistry, createLocal3DSf3DModelLoader, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const sf3d = createLocal3DSf3DModelLoader({ registry }); const plan = await sf3d.resolveLoadPlan({ precision: 'fp16', allowPrecisionDowngrade: true, gpuDevice: { id: 0, computeCapability: '8.6', vramBudgetMb: 8_192, }, }); ``` The loader is grounded in Stability AI’s published Stable Fast 3D sources as reviewed on `2026-03-26`: - the gated Hugging Face repository `stabilityai/stable-fast-3d` exposes `config.yaml` and `model.safetensors` - the official runtime entrypoint is `SF3D.from_pretrained(...)` - the released `run.py` example uses CUDA `torch.autocast(..., dtype=torch.bfloat16)` - the official requirements include native `texture_baker` and `uv_unwrapper` packages, plus `pynanoinstantmeshes` for remeshing - Stability AI documents Windows support as experimental and recommends Visual Studio 2022 for extension builds The resolved load plan includes: - gated-model install metadata and an explicit warning that Hugging Face access must be approved before automated downloads can succeed in a real environment - two half-precision operating profiles: `fp16` for the ~4GB minimum consumer profile and `bf16` for the ~6GB recommended profile on Ampere-or-newer GPUs - a documented downgrade path from `bf16` to `fp16` when the GPU lacks native bf16 support or enough VRAM, using `allowPrecisionDowngrade` - runtime hints for `SF3D.from_pretrained(...)`, CUDA autocast, CPU fallback via `SF3D_USE_CPU`, supported remesh modes, and the upstream `--texture-resolution` CLI flag - explicit dependency manifests for the official repo, pinned Python packages, and the compiled `texture_baker` / `uv_unwrapper` extensions needed by later UV and texture-output tasks The `fp16` plan is intentionally labeled as an inference from the official sources rather than a direct upstream default: Stability AI’s published example uses `bf16` autocast on CUDA, but the Phase 71 requirement is a half-precision consumer-GPU loader that can operate in the 4GB-to-6GB range when later pipelines are layered on top. ## SPAR3D Loader `Local3DSpar3DModelLoader` extends the Stability AI integration with the newer point-aware runtime and makes the editable point-cloud intermediate explicit in the local model contract: ```ts import { Local3DModelRegistry, createLocal3DSpar3DModelLoader, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const spar3d = createLocal3DSpar3DModelLoader({ registry }); const plan = await spar3d.resolveLoadPlan({ allowMemoryDowngrade: true, gpuDevice: { id: 0, computeCapability: '8.9', vramBudgetMb: 8_192, }, }); ``` The loader is grounded in Stability AI’s published SPAR3D sources and model tree as reviewed on `2026-03-26`: - the gated Hugging Face repository `stabilityai/stable-point-aware-3d` exposes `config.yaml` and `model.safetensors` - the official runtime entrypoint is `SPAR3D.from_pretrained(...)` - the released `run.py` example uses CUDA `torch.autocast(..., dtype=torch.bfloat16)`, `SPAR3D_LOW_VRAM`, and `return_points=True` - the official `spar3d.system` path first generates a denoised `pc_cond` point-cloud condition and then feeds that editable point cloud into the final mesh stage - the official Gradio demo allows users to export, edit, re-import, subsample, and pad the point cloud before final meshing The resolved load plan includes: - a standard runtime profile at roughly `10.5GB` VRAM and a published low-VRAM fallback at roughly `7GB`, with automatic downgrade via `allowMemoryDowngrade` - explicit runtime hints for `SPAR3D.from_pretrained(...)`, `SPAR3D_LOW_VRAM`, CPU fallback via `SPAR3D_USE_CPU`, and experimental Apple Silicon fallback via `PYTORCH_ENABLE_MPS_FALLBACK` - a typed `pointCloudIntermediate` contract describing the released `xyzrgb` representation: 512 points, 6 channels, `pc_cond` as the batch key, `pointcloud` as the `run_image(...)` argument, and `normalize_pc_bbox` as the normalization step before meshing - low-VRAM dynamic-module paging metadata that mirrors the released point-diffusion, main-mesh, and estimator groups unloaded and reloaded by the upstream runtime - dependency manifests for the official repo, CLIP / AlphaCLIP, optional remesh packages, optional Gradio point-cloud editor packages, and the compiled `texture_baker` / `uv_unwrapper` extensions The plan also preserves one upstream ambiguity instead of hiding it: Stability AI’s current README mentions both `~10.5GB` default VRAM and `~6GB` single-image manual inference. Oshun treats the explicit low-VRAM section as authoritative for fallback planning, keeps the discrepancy in `warnings`, and leaves later SPAR3D inference tasks to validate the true end-to-end footprint under the selected output settings. ## Sparc3D Loader `Local3DSparc3DModelLoader` adds an honest loader contract for Sparc3D even though the public project still does not ship downloadable runtime weights: ```ts import { Local3DModelRegistry, createLocal3DSparc3DModelLoader, createLocal3DSparc3DOperatorBundleSource, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const sparc3d = createLocal3DSparc3DModelLoader({ registry, weightsSource: createLocal3DSparc3DOperatorBundleSource({ rootDir: '/models/private/sparc3d', }), }); const plan = await sparc3d.resolveLoadPlan({ gpuDevice: { id: 0, computeCapability: '8.9', vramBudgetMb: 24_576, }, }); ``` This loader is grounded in the live public Sparc3D materials reviewed on `2026-03-26`: - the project page still says code may be released only upon approval of Math Magic, so Oshun does not pretend an official inference runtime or weight tree is publicly downloadable yet - the public Hugging Face space is demo-facing and does not expose a checkpoint bundle that `Local3DModelRegistry` can download directly - the paper and project page still establish the product direction: sparse high-resolution `1024^3` geometry, Sparconv-VAE, and Sparcubes decoding The resolved load plan therefore treats Sparc3D as a validated private bundle format instead of a fake public repo: - `Local3DModelRegistry` can import the pretrained bundle from a local filesystem path or from a private mirrored Hugging Face repository - the bundle root must contain `bundle.manifest.json`, and that manifest must declare the `image-encoder`, `sparconv-vae`, `latent-diffusion`, and `sparcubes-decoder` checkpoint entries - the manifest must also advertise `1024` in `runtime.supportedVoxelResolutions`, so later high-resolution image-to-3D stages are blocked early if the wrong checkpoint family is staged - runtime precision, VRAM planning, latency, batch size, supported surface modes, and Python package requirements are all read from the bundle manifest; unknown values stay at `0` with an explicit warning instead of being guessed This keeps Sparc3D usable inside the local model-registry flow today while remaining accurate about the current upstream publication gap. ## Sparc3D Image-to-3D Pipeline `Local3DSparc3DImageTo3DPipeline` turns that private-bundle contract into an actual runnable `image-to-3d` stage with explicit `256^3`, `512^3`, and `1024^3` resolution plans: ```ts import { Local3DModelRegistry, createLocal3DSparc3DImageTo3DPipeline, createLocal3DSparc3DOperatorBundleSource, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const sparc3d = createLocal3DSparc3DImageTo3DPipeline({ registry, weightsSource: createLocal3DSparc3DOperatorBundleSource({ rootDir: '/models/private/sparc3d', }), }); const result = await sparc3d.run({ images: [imageBuffer], outputFormat: 'glb', quality: 'standard', surfaceMode: 'watertight', seed: 42, }); ``` This pipeline is grounded in the same public Sparc3D sources rechecked on `2026-03-27`: - the public GitHub README still exposes only the paper/project links and the `1024^3` positioning, not a published inference runtime - the project page still says code may be released only upon approval of Math Magic, so the executable path remains operator-supplied rather than upstream downloadable The runnable stage therefore uses a generic but explicit operator contract: - `Local3DModelRegistry` imports the full private bundle, including checkpoint files and the Python module named by `runtime.packageRoot` - the bundled runner loads `bundle.manifest.json`, resolves `runtime.entrypoint`, imports that module from the staged bundle, and invokes the callable with the image path, output path, requested voxel resolution, surface mode, seed, bundle manifest path, and resolved component paths - single-image conditioning is enforced up front; multi-image requests fail before runtime startup because the current Sparc3D task is specifically the single-image path - quality presets now map onto actual voxel plans: `draft -> 256^3`, `standard -> 512^3`, and `high/ultra -> 1024^3` - explicit `voxelResolution` overrides are supported when the operator bundle advertises that resolution in `runtime.supportedVoxelResolutions` The result surface matches the local provider conventions used elsewhere in this package: one model asset (`glb`, `obj`, or `ply`), an optional prepared-image thumbnail, and structured metadata carrying the bundle revision, selected voxel resolution, scaled VRAM estimate, selected surface mode, seed, output format, runtime warnings, and any operator-reported mesh statistics. ### Watertight Manifold Enforcement `surfaceMode: 'watertight'` now does more than forward a hint into the private bundle runtime: - Oshun independently parses the returned mesh topology for `obj` and ASCII `ply` outputs and computes boundary-edge, non-manifold-edge, degenerate-face, and connected-component counts before accepting the result as print-ready - `glb` exports remain supported, but the operator bundle must also emit an OBJ or ASCII-PLY analysis sidecar through `analysis_mesh_path`, because the final GLB container is not treated as a trustworthy topology-analysis source inside this library - if the mesh is not closed and manifold, the request fails instead of quietly returning a supposedly watertight asset - completion metadata now records `watertightMesh`, `repairActions`, and `printableSolid`, which makes the 3D-printing contract explicit to downstream orchestration This is intentionally strict. Sparc3D's public paper and project page position the method around high-fidelity `1024^3` watertight surfaces, so Oshun treats a failed watertight verification as a hard correctness error rather than a soft warning. ### Open-Surface Support `surfaceMode: 'open-surface'` now has a distinct validation path aimed at thin surfaces and disconnected parts: - Oshun uses the same OBJ / ASCII-PLY topology analysis flow to inspect boundary edges, connected components, degenerate faces, and non-manifold edges for open-surface requests - boundary edges and multiple connected components are treated as valid signals that thin sheets or floating sub-meshes survived reconstruction, instead of being misclassified as failures - `glb` exports must still include an analysis sidecar so the library can inspect whether open boundaries or loose parts were preserved - completion metadata now records `openSurfaceMesh`, including `preservesOpenBoundaries`, `hasDisconnectedComponents`, and `supportsThinStructures` Open-surface mode still rejects clearly broken geometry such as non-manifold edges or degenerate faces. The difference from watertight mode is that boundary loops and disconnected components are allowed and explicitly surfaced instead of being treated as defects. ### Geometric Fidelity Benchmark `runDefaultLocal3DSparc3DGeometricFidelityBenchmark()` adds a deterministic geometry-only comparison surface for the local Sparc3D, TRELLIS.2, and Hunyuan3D 2.1 stacks: ```ts import { runDefaultLocal3DSparc3DGeometricFidelityBenchmark } from '@isis/3d-inference-local'; const report = runDefaultLocal3DSparc3DGeometricFidelityBenchmark(); console.log(report.summary); ``` The benchmark intentionally avoids texture-centric scoring and instead focuses on the geometric claims that make Sparc3D interesting inside Oshun: - `geometryFidelity` and `surfaceDetailRetention` weight silhouette accuracy and dense relief retention - `watertightReliability` measures printable-solid performance on closed-shape cases - `openSurfacePreservation` and `thinStructurePreservation` keep the open-surface and delicate-geometry paths first-class instead of forcing every provider into a watertight-only rubric - pairwise report rows expose Sparc3D's win rate and worst-case margin against the local TRELLIS.2 and Hunyuan3D 2.1 baselines Like the existing TRELLIS and Hunyuan dashboards, this is a deterministic evaluation dataset checked into the package, which keeps CI coverage reliable and makes geometry-regression thresholds explicit without requiring live model runs during tests. ### MeshAnything V2 Cleanup The package now also exposes a dedicated MeshAnything V2 model loader so the official Hugging Face weights and runtime caveats can be planned explicitly before any cleanup or remeshing workflow runs: ```ts import { Local3DModelRegistry, createLocal3DMeshAnythingV2ModelLoader, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d' }); const loader = createLocal3DMeshAnythingV2ModelLoader({ registry }); const plan = await loader.resolveLoadPlan(); console.log(plan.install.installDir); console.log(plan.runtime.maxOutputFaces); console.log(plan.warnings); ``` This loader is grounded in the public MeshAnything V2 release reviewed on `2026-03-27`: - the canonical official Hugging Face repository is `Yiwen-ntu/MeshAnythingV2` and the published weight set currently includes `config.json`, `model.safetensors`, and `350m.pth` - Oshun models MeshAnything V2 as a `remeshing` family member even though the Hugging Face card uses `image-to-3d`, because the released runtime actually consumes dense meshes or point clouds through `main.py` - the published operational envelope is about `8GB` VRAM and `45s` per mesh on an A6000, while Oshun recommends extra headroom for Marching Cubes preprocessing and flash-attn allocation spikes - the upstream model class still bootstraps config from `facebook/opt-350m`, so fully offline deployments must pre-cache that config or patch the runtime to stay air-gapped - the Git checkout remains part of the runtime contract because the released entrypoint imports `mesh_to_pc.py` plus `MeshAnything/miche/shapevae-256.yaml` from the repository tree - outputs remain within the current upstream `<=1600` face artist-mesh regime and should be treated as topology cleanup / controlled retopology rather than unlimited mesh refinement Sparc3D can now hand its topology-analysis OBJ into the released MeshAnything V2 CLI as an explicit post-process step: ```ts import { Local3DModelRegistry, createLocal3DSparc3DImageTo3DPipeline, createLocal3DSparc3DOperatorBundleSource, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d' }); const sparc3d = createLocal3DSparc3DImageTo3DPipeline({ registry, weightsSource: createLocal3DSparc3DOperatorBundleSource({ rootDir: '/models/private/sparc3d', }), meshAnythingRuntimeRoot: '/models/runtime/MeshAnythingV2', }); const result = await sparc3d.run({ images: [imageBuffer], outputFormat: 'glb', surfaceMode: 'watertight', topologyCleanup: 'meshanything-v2', quality: 'high', }); ``` This integration is deliberately grounded in the current upstream MeshAnything V2 surface reviewed on `2026-03-27`: - the official repository exposes `main.py` for mesh-inference rather than a stable Python library API, so Oshun drives that CLI through a dedicated runner - upstream recommends `--mc` preprocessing for non-Marching-Cubes meshes; Oshun enables it by default for watertight Sparc3D cleanup and disables it for `surfaceMode: 'open-surface'` so intended boundary loops are not sealed away - the current upstream model targets an artist-mesh envelope capped by the released `<=1600` face regime, so cleanup is treated as topology improvement with an expected density reduction rather than a fidelity-preserving no-op The returned metadata now includes `topologyCleanup` with before/after face and vertex counts, boundary / manifold deltas, Marching Cubes settings, and cleanup warnings. Oshun then reruns the same watertight or open-surface verification on the cleaned mesh before returning it, so a post-process cannot silently break the geometry contract established by the original Sparc3D request. The same MeshAnything V2 runtime can now also be reused as an automatic post-process helper across the other generation families in this package: - TRELLIS text, TRELLIS.2 single-image, and TRELLIS.2 multi-image now automatically remesh when callers request `meshTopology` or `targetPolyCount` - Hunyuan3D 2.1 text/image single runs, both shape batch APIs, and the combined shape+texture orchestrator now enforce `targetPolyCount` through the same post-process helper - TripoSG image, scribble, and multi-view consistency runs automatically hand quad/topology-sensitive requests to MeshAnything V2 instead of leaving them as warning-only intent - Sparc3D automatically activates its MeshAnything cleanup handoff when callers ask for face-budget or topology controls, while surfacing that the dedicated animation-ready edge-flow pass currently lives on the MeshAnything remeshing API rather than Sparc3D's own request layer ### FlexiCubes Mesh Extraction `meshExtractionMethod: 'flexicubes'` now requests a Kaolin-backed FlexiCubes remesh across the geometry-first generation families in this package: ```ts import { Local3DModelRegistry, createLocal3DTripoSGImageTo3DPipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d' }); const pipeline = createLocal3DTripoSGImageTo3DPipeline({ registry, runtimeRoot: '/opt/TripoSG', }); const result = await pipeline.run({ images: [inputImageBuffer], outputFormat: 'obj', quality: 'high', meshExtractionMethod: 'flexicubes', }); ``` This feature is intentionally honest about the current upstream boundaries: - the official NVIDIA Kaolin surface reviewed on `2026-03-27` is `kaolin.non_commercial.flexicubes.FlexiCubes`, so Oshun now drives the real Kaolin extractor instead of pretending native model outputs were already FlexiCubes meshes - most released 3D runtimes used here return finalized meshes rather than caller-controlled signed-distance fields, so Oshun reconstructs a temporary scalar field from nearest-surface point-and-normal projection before invoking FlexiCubes - TRELLIS text/image/multi-image, Hunyuan3D 2.1 text/image/batch shape runs, TripoSG image/scribble/multi-view, and Sparc3D all expose this as a real post-extraction path with per-run `flexiCubesPostProcess` metadata - Sparc3D keeps an OBJ or ASCII-PLY analysis sidecar when FlexiCubes is used so open-surface / watertight validation and optional MeshAnything cleanup still inspect the remeshed geometry instead of stale native output - SF3D and SPAR3D now reject `meshExtractionMethod: 'flexicubes'` explicitly because those released Stability AI pipelines bake textured GLB assets from their native mesh path and Oshun cannot re-extract geometry there without discarding baked materials FlexiCubes now also defaults to adaptive resolution on top of Kaolin's public regular voxel-grid API. Oshun keeps the real `construct_voxel_grid(...)` topology exposed by Kaolin, then applies a bounded axis warp derived from face-dihedral and open-boundary complexity histograms before reconstructing the scalar field. That gives flatter regions coarser cell spacing while redistributing density toward thin, high-curvature, and boundary-heavy areas without inventing an upstream octree API that Kaolin does not currently expose. Grid selection is still preset-driven at the coarse level: `draft` maps to `128`, mid-tier requests map to `192`, and higher-detail plans map to `256` unless a family already exposes a stronger native resolution signal such as Sparc3D voxel resolution. Low-level callers can still set `adaptivityMode: 'uniform'` on the FlexiCubes postprocessor when they need a strictly uniform reconstruction grid for controlled experiments or comparisons. The low-level FlexiCubes postprocessor also now exposes an optional `differentiableRefinement` stage for iterative mesh cleanup: ```ts import { createLocal3DFlexiCubesAutomaticPostProcessor } from '@isis/3d-inference-local'; const processor = createLocal3DFlexiCubesAutomaticPostProcessor({ workDir: '/tmp/oshun-flexicubes', }); const refined = await processor.run({ sourceLabel: 'Sparc3D image-to-3d generation', mesh: inputMeshBuffer, meshFormat: 'obj', outputFormat: 'obj', gridResolution: 256, differentiableRefinement: { iterations: 48, cameraBatchSize: 4, trainResolution: 256, }, }); ``` This path follows the official FlexiCubes optimization workflow rather than a made-up post-smoothing pass: Oshun initializes the FlexiCubes SDF from the input mesh, renders mask/depth supervision from that source mesh with `nvdiffrast`, and then optimizes FlexiCubes SDF values, voxel deformations, and per-cube weights before re-extracting the final surface. Because this is a real differentiable optimization loop, it is intentionally opt-in on the low-level API instead of silently running on every FlexiCubes request. For regression tracking, the package now also exposes a deterministic FlexiCubes-vs-Marching-Cubes benchmark harness through `runDefaultLocal3DFlexiCubesVsMarchingCubesBenchmark(...)`. The default dataset captures five equal-budget extraction cases across hard-surface, thin-structure, photogrammetry, organic, and optimization-oriented workloads, and scores both extractors on geometry fidelity, sharp feature retention, triangle quality, topology cleanliness, thin-structure preservation, and optimization readiness. That keeps the quality claim tied to an explicit, versioned benchmark surface instead of ad hoc eyeballing. When CUDA is available, the FlexiCubes postprocessor now also uses Kaolin's mesh-ops stack directly for the heavy mesh-prep stages instead of relying only on `trimesh` and a CPU KD-tree. Surface sampling runs through `kaolin.ops.mesh.sample_points`, nearest-face normals come from `kaolin.ops.mesh.face_normals`, the scalar field is built with `kaolin.metrics.trianglemesh.point_to_mesh_distance`, and watertight sources use `kaolin.ops.mesh.check_sign` for sign classification. Open meshes still get the accelerated Kaolin distance path, but switch sign inference to a nearest-face-normal heuristic, and full CPU fallback is recorded explicitly in the returned `kaolinMeshOps` report metadata and warning stream. ### MeshAnything V2 Remeshing Pipeline The package now also exposes a dedicated remeshing pipeline for direct dense mesh cleanup: ```ts import { Local3DModelRegistry, createLocal3DMeshAnythingV2RemeshingPipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d' }); const pipeline = createLocal3DMeshAnythingV2RemeshingPipeline({ registry, runtimeRoot: '/models/runtime/MeshAnythingV2', }); const result = await pipeline.run({ mesh: inputDenseMeshBuffer, meshFormat: 'obj', meshTopology: 'quad-dominant', animationReadyTopology: true, animationRigProfile: 'humanoid', edgeFlowStrength: 'balanced', }); ``` This pipeline is grounded in the current upstream MeshAnything V2 runtime, but it closes the practical gaps needed for local production use: - it resolves the MeshAnything V2 Hugging Face weights through the local model registry and passes the downloaded weight directory into the runtime instead of pretending a hard-coded remote model id is sufficient - the Python runner now drives the upstream `mesh_to_pc.py` plus `MeshAnythingV2.from_pretrained(...)` path directly, which keeps the released geometry logic while avoiding the earlier remote-only CLI handoff - Oshun patches the upstream `facebook/opt-350m` config bootstrap to use the local `config.json` when a local model directory is supplied, so the remesh path can run without a hidden extra network fetch - `meshTopology: 'quad-dominant'` is implemented as a real two-stage cleanup: MeshAnything V2 first produces the artist-mesh triangle result, then Oshun runs a quad-dominant conversion pass and preserves the result as OBJ - `animationReadyTopology: true` adds an explicit edge-flow optimization stage between the artist mesh and the quad conversion pass; Oshun uses articulation band heuristics plus documented PyMeshLab remeshing/edge-flip filters so likely neck, shoulder, elbow, hip, knee, and ankle regions can be biased toward cleaner rigging loops for `generic-biped`, `humanoid`, and `quadruped` assets - face-count control now snaps requests onto the supported `200`, `400`, `800`, and `1600` artist-mesh budgets and applies that simplification before any optional quad conversion, so later topology stages work from the requested density envelope instead of a fixed upstream output - the pipeline exports both the normalized preprocessed input mesh and the MeshAnything artist-mesh OBJ sidecar so topology analysis and later quality scoring can compare before/after structure instead of only the final file - returned metadata includes normalized input, artist-mesh, and final-output topology summaries, plus the Marching Cubes setting, seed, and warnings that affected the cleanup path - animation-ready runs also return an `edgeFlowOptimization` report with the optimized articulation bands, quad coverage, valence-4 ratio, edge-length consistency, and a heuristic closed-loop score for the final OBJ topology - every remesh run now also returns a `topologyQualityComparison` report that scores the normalized pre-remesh mesh against the post-remesh topology on boundary-edge ratio, non-manifold/degenerate penalties, connected-component count, face-regularity, valence variance, and target-topology preference so Oshun can quantify whether the remesh materially improved topology quality ### MeshAnything V2 LOD Chain Generation The package now also exposes a dedicated MeshAnything V2 LOD-chain generator for game-oriented asset packaging: ```ts import { Local3DModelRegistry, createLocal3DMeshAnythingV2LodChainGenerator, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d' }); const generator = createLocal3DMeshAnythingV2LodChainGenerator({ registry, runtimeRoot: '/models/runtime/MeshAnythingV2', }); const result = await generator.run({ mesh: inputDenseMeshBuffer, meshFormat: 'obj', outputFormat: 'glb', }); ``` This chain generator is grounded in the same upstream MeshAnything V2 limits and the published `MSFT_lod` ordering model: - Oshun emits `LOD0` as the highest-detail MeshAnything cleanup pass, then derives `LOD1` through `LOD4` from the supported `1600`, `800`, `400`, and `200` face budgets - the returned manifest includes descending screen-coverage thresholds aligned with the high-to-low ordering used by the Khronos `MSFT_lod` extension, so downstream engines can wire the exported files into their own LOD switching policy without reverse-engineering the chain order - each LOD preserves the same topology preferences as the source remesh request, including quad-dominant conversion and the optional animation-ready edge-flow pass, instead of silently dropping those constraints on lower-detail levels - Oshun currently exports the chain as separate OBJ/PLY/GLB assets plus a JSON manifest rather than merging them into a single glTF container, which keeps the output honest and compatible with non-glTF game build pipelines ### MeshAnything V2 Batch Remeshing The package now also exposes a dedicated MeshAnything V2 bulk-cleanup pipeline for asset-library remediation: ```ts import { Local3DModelRegistry, createLocal3DMeshAnythingV2BatchRemeshingPipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d' }); const pipeline = createLocal3DMeshAnythingV2BatchRemeshingPipeline({ registry, runtimeRoot: '/models/runtime/MeshAnythingV2', }); const result = await pipeline.runBatch( [ { mesh: firstDenseMeshBuffer, meshFormat: 'obj', outputFormat: 'glb', meshTopology: 'triangle', }, { mesh: secondDenseMeshBuffer, meshFormat: 'obj', animationReadyTopology: true, animationRigProfile: 'humanoid', targetFaceCount: 800, }, ], { requestedBatchSize: 2 } ); ``` This batch pipeline is grounded in the released MeshAnything V2 runtime and the upstream folder-input execution model, but it fills in the production gaps for real library cleanup: - the public `main.py --input_dir ...` path is treated as the honest upstream batching surface, and Oshun converts that into manifest-driven CUDA micro-batches instead of pretending the original single-file CLI was already a full provider-ready batch API - compatible jobs share one MeshAnything V2 model load and one batched forward pass while still preserving per-item output formats, face-count simplification, normalized-input sidecars, and artist-mesh analysis OBJs - compatibility grouping is explicit: Marching Cubes enablement/resolution, the sampling flag, and stochastic sampling seeds must match before assets are placed into the same micro-batch - Oshun keeps the VRAM envelope honest by defaulting to micro-batch size `2` when no GPU budget is supplied, allowing `2` on typical `24GB` cards, and only exposing `4` on `48GB+` plans; `256`-resolution Marching Cubes and stochastic sampling stay capped at `2` - post-remesh topology work still happens per asset, so quad-dominant conversion, animation-ready edge-flow optimization, topology-quality scoring, and provider-compatible completion payloads survive the batch path instead of being silently dropped for throughput - returned batch metadata now reports compatibility-group count, micro-batch count, effective execution mode, and measured throughput in items per minute so operators can reason about whether the cleanup job actually delivered library-scale gains ## SPAR3D Image-to-3D Pipeline `Local3DSpar3DImageTo3DPipeline` turns the released SPAR3D point-aware runtime into a provider-compatible single-image workflow and keeps the intermediate point cloud visible instead of hiding it behind the final mesh: ```ts import { Local3DModelRegistry, createLocal3DSpar3DImageTo3DPipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const pipeline = createLocal3DSpar3DImageTo3DPipeline({ registry, runtimeRoot: '/opt/stable-point-aware-3d', }); const result = await pipeline.run({ images: [inputImageBuffer], quality: 'standard', guidanceScale: 3, seed: 1234, }); ``` This pipeline is grounded in the same official SPAR3D sources reviewed on `2026-03-26`, but it intentionally follows the lower-level Gradio flow instead of the one-line CLI helper so the editable point cloud remains available: - image preprocessing still uses the released `transparent-background` foreground-removal path plus `foreground_crop(...)` - point diffusion is run explicitly through `forward_pdiff_cond(...)` and `sampler.sample_batch_progressive(...)` so `guidanceScale` and `seed` are honored for the intermediate `pc_cond` - the intermediate 512-point `xyzrgb` cloud is exported as `points.ply` alongside the final textured `mesh.glb` - final meshing still uses the released `generate_mesh(...)` path with optional `none` / `triangle` / `quad` remeshing and texture baking The returned provider completion includes: - a primary textured `glb` asset - a secondary `ply` model asset containing the intermediate point cloud - a thumbnail of the processed RGBA input used for reconstruction - metadata for point-cloud generation time, mesh generation time, point count, selected memory mode, guidance scale, seed, and the emitted runtime keys `standard` quality maps to the released `1024` texture-resolution default and stays closest to the published fast path. `high` and `ultra` remain available, but the pipeline warns when those settings move the run away from the fastest advertised SPAR3D profile. ## SPAR3D Point-Cloud Editing `Local3DSpar3DPointCloudEditingPipeline` exposes the same editable intermediate that Stability AI’s published Gradio demo uses and turns it into a provider-safe local workflow: ```ts import { Local3DModelRegistry, createLocal3DSpar3DPointCloudEditingPipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const pipeline = createLocal3DSpar3DPointCloudEditingPipeline({ registry, runtimeRoot: '/opt/stable-point-aware-3d', }); const result = await pipeline.run({ images: [inputImageBuffer], quality: 'standard', editOperations: [ { kind: 'translate', pointIndices: [0, 1, 2], delta: [0.05, 0.0, -0.03] }, { kind: 'recolor', pointIndices: [0, 1, 2], color: [255, 180, 32] }, ], }); ``` This pipeline is grounded in the same official SPAR3D sources reviewed on `2026-03-26`, including the published Gradio demo and runtime internals: - when `pointCloudData` is omitted, Oshun follows the released image-to-point diffusion path to generate the editable `pc_cond` - when `pointCloudData` is supplied, the pipeline accepts an uploaded `ply` point cloud, normalizes it to SPAR3D’s 512-point conditioning size through padding or subsampling, and bypasses point-diffusion-only controls such as `guidanceScale` and `seed` - edit operations are applied sequentially to the `xyzrgb` cloud before `generate_mesh(...)` reconstructs the final textured mesh - the supported edit primitives match the concrete editing affordances the official demo advertises directly: point translation, point scaling, and point recoloring - the edited point cloud is exported as `ply` alongside the final textured `glb`, so callers can round-trip into external tools such as MeshLab or Blender for more advanced edits and then re-run meshing locally The returned provider completion includes: - a primary textured `glb` asset generated from the edited point cloud - a secondary `ply` asset containing the final edited conditioning cloud - a thumbnail of the processed RGBA input image used for conditioning - metadata for point-cloud source, edit operation count, applied operation kinds, edit time, mesh time, point count, and the selected memory mode `standard` quality stays aligned with the released 1024-texture fast path. Higher quality tiers remain available for better texture fidelity, but the pipeline preserves the same warnings when those settings move the run away from the advertised low-latency SPAR3D profile. ## SPAR3D Preview Loop `Local3DSpar3DPreviewLoop` turns SPAR3D’s published “edit point cloud, then re-run meshing” demo pattern into a real warmed local session instead of a cold-start wrapper: ```ts import { Local3DModelRegistry, createLocal3DSpar3DPreviewLoop, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const loop = createLocal3DSpar3DPreviewLoop({ registry, runtimeRoot: '/opt/stable-point-aware-3d', }); const { session } = await loop.start({ images: [inputImageBuffer], quality: 'high', meshTopology: 'quad-dominant', targetVertexCount: 3200, }); const fastPreview = await session.preview({ editOperations: [ { kind: 'translate', pointIndices: [0, 1, 2], delta: [0.04, 0, -0.02] }, ], }); const finalMesh = await session.finalize(); await session.close(); ``` This preview loop is grounded in the same official SPAR3D sources reviewed on `2026-03-26`, especially the published Gradio demo’s “Re-run with point cloud” flow and the released `generate_mesh(...)` runtime internals: - the Python worker loads `SPAR3D.from_pretrained(...)` once, preprocesses the conditioning image once, and keeps the current 512-point `pc_cond` resident between renders - `session.preview(...)` defaults to the real fast-path settings for interactive feedback: `512` texture resolution and `remesh: 'none'` - `session.finalize(...)` uses the higher-quality defaults captured at session start, so callers can iterate quickly and then produce the final mesh without reloading the model - `session.reset()` restores the original generated or uploaded point cloud so a preview loop can recover from bad edits without re-running image conditioning - uploaded replacement point clouds still go through the same 512-point normalization that the official demo applies before meshing Each preview/final render returns: - a textured `glb` mesh for the current edited point cloud - a `ply` export of the current conditioning cloud after any new edits - the prepared conditioning thumbnail - per-render metadata for edit counts, render mode, mesh timings, point-cloud source, and texture/remesh settings The preview loop is intentionally conservative about what counts as “interactive”: raising texture resolution above `512` or enabling remeshing during `session.preview(...)` is allowed, but surfaced as a warning because it moves the session away from the released fast edit path. ## SF3D Image-to-3D Pipeline `Local3DSf3DImageTo3DPipeline` wraps the official Stable Fast 3D single-image runtime into the Oshun provider surface and keeps the released preprocessing and export path intact: ```ts import { Local3DModelRegistry, createLocal3DSf3DImageTo3DPipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const pipeline = createLocal3DSf3DImageTo3DPipeline({ registry, runtimeRoot: '/opt/stable-fast-3d', defaultPrecision: 'fp16', allowPrecisionDowngrade: true, }); const result = await pipeline.run({ images: [inputImageBuffer], quality: 'standard', meshTopology: 'quad-dominant', targetVertexCount: 12000, }); ``` This implementation is based on Stability AI’s released code paths reviewed on `2026-03-26`: - `run.py` removes the background, resizes the foreground crop, runs `SF3D.from_pretrained(...).run_image(...)`, and exports `mesh.glb` - `gradio_app.py` uses the same `foreground_ratio` preprocessing and drives the direct `generate_mesh(...)` path with a textured GLB preview - `sf3d/system.py` handles UV unwrap, texture baking, and PBR material assembly during mesh generation rather than as a post-export step The pipeline surface intentionally reflects those upstream behaviors: - only a single image is accepted for this task because the released SF3D runtime is single-image conditioned - output is restricted to textured `glb` so the provider completion preserves the official textured-mesh artifact instead of discarding materials - `quality` maps onto texture-atlas size: `draft=512`, `standard=1024`, `high=2048`, `ultra=4096` - `meshTopology` maps onto the official remesh options: `triangle -> triangle`, `quad-dominant -> quad`, unset -> `none` - the completion payload includes both the textured GLB and the processed input preview used for reconstruction At runtime the Node wrapper: - resolves the previously introduced SF3D load plan so gated-model warnings, precision downgrades, and VRAM estimates stay aligned with the loader - writes the incoming image buffer to disk, launches the bundled Python runner, and streams JSON progress for runtime load, preprocessing, mesh generation, texture bake, and final export - preserves the upstream `remove_background` and `resize_foreground` flow, surfaces peak memory and generation duration metadata from the runner, and returns a provider completion asset with `model/gltf-binary` content ## SF3D Batch Image-to-3D Pipeline `Local3DSf3DBatchImageTo3DPipeline` exposes the real SF3D batched inference path from Stability AI’s published CLI instead of simulating batching at the queue layer: ```ts import { Local3DModelRegistry, createLocal3DSf3DBatchImageTo3DPipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const pipeline = createLocal3DSf3DBatchImageTo3DPipeline({ registry, runtimeRoot: '/opt/stable-fast-3d', gpuDevice: { id: 0, computeCapability: '8.9', vramBudgetMb: 24_576, }, }); const result = await pipeline.runBatch( [{ images: [chairImage] }, { images: [lampImage] }, { images: [shoeImage] }], { requestedBatchSize: 8 } ); ``` This batch pipeline is grounded in the same official SF3D sources reviewed on `2026-03-26`: - Stability AI’s published `run.py` already accepts `--batch_size` and passes a list of prepared images into `model.run_image([...])` - the bundled batch runner preserves the official preprocessing sequence for every item: RGBA load, optional `rembg`, and `resize_foreground(...)` - compatible requests are grouped by precision, background-removal settings, foreground ratio, texture resolution, remesh mode, and target vertex count so each micro-batch maps cleanly onto one native SF3D invocation - the micro-batch size is capped by a conservative VRAM heuristic derived from the already-documented 4GB `fp16` and 6GB `bf16` runtime envelopes - remeshing requests intentionally fall back to warm-sequential execution because the per-mesh CPU remesh stage is not the released high-throughput path The batch result includes: - one provider-compatible `Local3DSf3DImageTo3DResult` per input request - batch metadata for compatibility groups, micro-batch count, effective batch size, total duration, and measured items-per-minute throughput - a `targetThroughputAchieved` flag checked against the Phase 71 goal of `100` images per minute `draft` and `standard` quality with `remesh: 'none'` are the intended throughput settings. Higher texture resolutions or remeshing remain available, but the pipeline warns when those controls move a run away from the fastest official SF3D path. ## Quality-vs-Speed Routing `Local3DQualityVsSpeedRouter` promotes the first automatic model-selection layer inside the local 3D stack. It is intentionally narrow and honest: SF3D remains the default speed path for single-image preview-style requests, while TRELLIS.2 takes over when the request is clearly quality-driven or needs a delivery format that the current SF3D surface does not expose. ```ts import { Local3DModelRegistry, createLocal3DQualityVsSpeedRouter, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const router = createLocal3DQualityVsSpeedRouter({ registry, sf3dRuntimeRoot: '/opt/stable-fast-3d', trellisRuntimeRoot: '/opt/TRELLIS.2', }); const speedResult = await router.runImage({ images: [thumbnailImage], quality: 'draft', }); const qualityResult = await router.runImage({ images: [heroReference], quality: 'ultra', outputFormat: 'glb', }); ``` The router is grounded in the same upstream implementations reviewed on `2026-03-26`: - Stability AI’s released SF3D stack is the fast single-image textured GLB path with published sub-second latency and native batched `run_image([...])` - Microsoft’s released TRELLIS.2 runtime is the higher-fidelity path with documented 512³ / 1024³ / 1536³ quality tiers and a first-party GLB export flow via `o_voxel.postprocess.to_glb(...)` - Oshun’s TRELLIS route composes those existing local TRELLIS geometry and GLB export stages so a quality-routed request does not regress from “delivery asset” back to “raw mesh intermediate” Current routing rules: - single-image `draft` and latency-sensitive `standard` requests default to SF3D - `high` / `ultra` quality, prompt-guided image requests, and non-GLB delivery requests route to TRELLIS.2 - multi-image requests stay on TRELLIS.2 because SF3D remains single-image - explicit quad-dominant topology requests can stay on SF3D when that topology control matters more than TRELLIS fidelity heuristics The router exposes `selectImageRoute(...)`, `runImage(...)`, `runBatch(...)`, and `runText(...)`. Mixed batches are supported: speed-route items are collapsed into the SF3D batch pipeline when available, while quality-route outliers run through TRELLIS sequentially with per-item routing metadata preserved. ## Intelligent Model Routing `Local3DIntelligentModelRouter` is the broader model-selection layer for Phase 71. It keeps the existing quality-vs-speed router intact, but expands the decision space across the local 3D families already implemented in this package: - organic and character-leaning prompts favor Hunyuan3D 2.1 - hard-surface, architectural, and multi-image requests favor TRELLIS.2 - speed-critical single-image preview jobs favor SF3D - high-resolution, printable, or thin-structure geometry jobs favor Sparc3D - sketch and doodle-flavored image prompts favor TripoSG ```ts import { Local3DModelRegistry, createLocal3DIntelligentModelRouter, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const router = createLocal3DIntelligentModelRouter({ registry, sf3dRuntimeRoot: '/opt/stable-fast-3d', trellisRuntimeRoot: '/opt/TRELLIS.2', hunyuanRuntimeRoot: '/opt/hunyuan3d', triposgRuntimeRoot: '/opt/TripoSG', }); const decision = router.selectImageRoute({ images: [referenceImage], prompt: 'high-resolution printable ornate statue with fine filigree', quality: 'ultra', outputFormat: 'obj', }); const result = await router.runImage({ images: [referenceImage], prompt: 'hard-surface mech cockpit with industrial panels', outputFormat: 'glb', }); ``` The routing heuristics were aligned against the released upstream capability surfaces reviewed on `2026-03-27` and then mapped onto Oshun’s existing local pipelines: - Hunyuan3D 2.1 is the strongest local path for organic, production-style asset generation and direct GLB delivery - TRELLIS.2 remains the broadest geometry-plus-texturing path for hard-surface and multi-image reconstruction - SF3D is still the fastest released single-image textured preview route - Sparc3D is the geometry-first route when watertight, printable, or high-resolution reconstruction matters most - TripoSG is the best fit for sketch-like inputs and stylized reference images Every routed result preserves a structured `routing` block in provider completion metadata with the selected family, selected model key, execution mode, candidate scores, prompt-analysis evidence, and human-readable reasons. TRELLIS image routes also preserve the existing geometry-plus-GLB/GLTF export composition so a routed request can still come back as a delivery-ready asset instead of an intermediate mesh only. ## Quality-Aware Model Routing `Local3DQualityAwareModelRouter` sits on top of the intelligent router and executes multiple compatible candidates before selecting a winner. The current implementation is benchmark-backed rather than hand-wavy: - candidate families are shortlisted from the existing intelligent-router prompt analysis and compatibility scores - compatible candidates are executed in parallel or in bounded parallel cohorts depending on `maxConcurrentCandidates` - each successful candidate is scored from a weighted mix of benchmark prior, prompt-fit prior, artifact-quality signals, delivery fitness, latency efficiency, and warning penalties - benchmark priors come from the existing TRELLIS.2 benchmark suite, the Hunyuan3D 2.1 quality dashboard, and the Sparc3D geometric-fidelity benchmark; SF3D and TripoSG use explicit local heuristics because Phase 71 does not yet include equivalent comparative dashboards for them - failed candidates are preserved in the quality-routing report and excluded from ranking instead of aborting the whole request ```ts import { Local3DModelRegistry, createLocal3DQualityAwareModelRouter, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const router = createLocal3DQualityAwareModelRouter({ registry, trellisRuntimeRoot: '/opt/TRELLIS.2', hunyuanRuntimeRoot: '/opt/hunyuan3d', triposgRuntimeRoot: '/opt/TripoSG', maxConcurrentCandidates: 2, }); const result = await router.runImage({ images: [referenceImage], prompt: 'printable cathedral ornament with fine filigree detail', outputFormat: 'obj', quality: 'ultra', }); console.log(result.decision.selectedFamily); console.log( result.candidates.map((candidate) => ({ family: candidate.family, rank: candidate.rank, score: candidate.score?.totalScore, })) ); ``` ## Cost-Aware Model Routing `Local3DCostAwareModelRouter` keeps self-hosted generation as the default path and only falls back to configured cloud providers once the selected local route is genuinely blocked by GPU saturation or hard self-hosted capacity limits. - the router reuses the existing intelligent local-routing decision instead of inventing a second local selection stack - scheduler snapshots are inspected to determine whether the selected local model can run immediately, whether every eligible GPU is saturated, and how much queue delay the request is likely to inherit - queue-delay budgets are priority-aware by default (`interactive`, `batch`, `background`) and can be overridden per request - cloud providers are filtered by request compatibility before selection; exact output-format support, texture-resolution limits, multi-image support, PBR support, variation limits, and advanced mesh-control safety are all checked - when cloud fallback is needed, the router ranks compatible providers by known cloud price profile first and latency second - advanced local-only controls such as `meshExtractionMethod`, `meshTopology`, and `quadDominantOptimization` keep the request on the self-hosted path even under saturation rather than silently degrading them The cloud-provider interface is intentionally structural, so existing provider instances that expose `generateFromText(...)`, `generateFromImage(...)`, and `getCapabilities()` can be passed in directly. That keeps `@isis/3d-inference-local` buildable without taking a hard compile-time dependency on the broader hosted-provider package surface. ```ts import { createLocal3DCostAwareModelRouter } from '@isis/3d-inference-local'; const router = createLocal3DCostAwareModelRouter({ intelligentRouter, scheduler, queue, costDashboard, cloudProviders: [ { id: 'meshy', provider: meshyProvider }, { id: 'rodin', provider: rodinProvider }, ], }); const result = await router.runImage({ images: [referenceImage], prompt: 'high-detail production prop', quality: 'high', outputFormat: 'glb', priority: 'interactive', }); console.log(result.decision.selectedTarget); console.log(result.decision.loadAssessment.estimatedQueueDelayMs); console.log(result.completion.metadata?.costRouting); ``` Cloud-fallback executions are normalized back into the same provider-completion shape as local executions, persisted into the configured `workDir`, and tagged with a `costRouting` metadata block that preserves the selected target, selected local model key, selected cloud provider, queue/load evidence, and the estimated self-hosted versus cloud cost comparison. The result surface keeps the winning routed output at the top level, preserves the full winning intelligent-route result under `winner`, and attaches a structured `qualityRouting` metadata block to provider completion metadata with candidate ranks, score margins, benchmark sources, and the scoring weights used for the final decision. ## Generation A/B Testing `Local3DGenerationABTestingFramework` provides a deterministic experiment layer for Phase 71 model-routing work. It is purpose-built for comparing the three surfaces Oshun actually changes in this package: - model-version upgrades - prompt-strategy rewrites - post-processing pipeline swaps The framework keeps the statistical surface explicit instead of hiding it behind "experiment" marketing language: - subjects are assigned to variants deterministically with sticky hashing - every experiment has an explicit control variant plus a declared primary metric - `success-rate` uses a two-sample z-test for proportions - `quality-score`, `latency-ms`, `cost-usd`, and `gpu-utilization-pct` use a Welch-style two-sample t-test - the snapshot reports confidence intervals, p-values, effect sizes, sample-size sufficiency, and a recommendation that stays conservative until the minimum sample target is reached ```ts import { createLocal3DGenerationABTestingFramework, createLocal3DModelVersionExperimentDefinition, } from '@isis/3d-inference-local'; const experiments = createLocal3DGenerationABTestingFramework(); experiments.createExperiment( createLocal3DModelVersionExperimentDefinition({ id: 'hunyuan-2-1-patch', name: 'Hunyuan 2.1 patch validation', primaryMetric: { id: 'quality-score', }, secondaryMetrics: [ { id: 'latency-ms', }, { id: 'cost-usd', }, ], minimumSampleSizePerVariant: 24, variants: [ { id: 'control', label: 'Hunyuan 2.1.0', modelKey: 'hunyuan3d-2.1', modelVersion: '2.1.0', isControl: true, }, { id: 'candidate', label: 'Hunyuan 2.1.1', modelKey: 'hunyuan3d-2.1', modelVersion: '2.1.1', }, ], }) ); experiments.startExperiment('hunyuan-2-1-patch'); const assignment = experiments.assignVariant('hunyuan-2-1-patch', 'subject-42'); experiments.recordOutcome('hunyuan-2-1-patch', { subjectId: 'subject-42', variantId: assignment.variantId, success: true, qualityScore: 0.87, latencyMs: 18400, costUsd: 0.21, }); const snapshot = experiments.getSnapshot('hunyuan-2-1-patch'); console.log(snapshot.summary.winningVariantId); console.log(snapshot.primaryMetricComparisons[0]?.pValue); ``` ## Generation Analytics Dashboard `Local3DGenerationAnalyticsDashboard` is the operational dashboard surface for the same routing stack. It tracks the signals that matter when model routing is making production decisions instead of benchmark-only ones: - per-model success and failure rates - quality scores extracted from routed result metadata - average, p50, and p95 latency - GPU utilization from live sample capture or embedded task telemetry - self-hosted cost plus cloud-price comparison deltas - optional experiment summaries when the A/B framework is attached ```ts import { createLocal3DGenerationABTestingFramework, createLocal3DGenerationAnalyticsDashboard, createLocal3DProviderRestApi, } from '@isis/3d-inference-local'; const experiments = createLocal3DGenerationABTestingFramework(); const analytics = createLocal3DGenerationAnalyticsDashboard({ experimentFramework: experiments, }); analytics.recordGpuUtilizationSample({ gpuId: 0, utilizationPct: 72, memoryUtilizationPct: 58, activeModelKeys: ['hunyuan3d-2.1'], }); const providerApi = createLocal3DProviderRestApi({ capabilities, generationAnalyticsDashboard: analytics, }); const snapshot = analytics.getSnapshot({ cloudProfileIds: ['meshy-pro'], }); console.log(snapshot.summary.successRate); console.log(snapshot.modelBreakdown[0]?.averageLatencyMs); console.log(snapshot.gpuBreakdown[0]?.latestUtilizationPct); console.log( analytics.getGpuUtilizationSamples({ gpuId: 0, since: new Date(Date.now() - 15 * 60_000), }).length ); ``` When the dashboard is attached to `Local3DProviderRestApi`, it is also exposed as a REST snapshot: - `GET /v1/3d/analytics/generations/dashboard` - filters: `kind`, `status`, `modelKey`, `gpuId`, `limit`, `selfHostedProfileId`, `cloudProfileIds`, and `experimentStatus` ## GPU Utilization Monitor `Local3DGpuUtilizationMonitor` consumes those sampled GPU traces plus the multi-model scheduler snapshot and turns them into actionable cluster-hygiene alerts: - underutilized GPUs are flagged when a recent utilization window stays below a configurable threshold - idle resident model instances are flagged when they keep holding VRAM without active assignments for too long - active and resolved alerts keep timestamps and stable identifiers so external automation can forward only state changes ```ts import { Local3DGpuUtilizationMonitor, createLocal3DGenerationAnalyticsDashboard, createLocal3DMultiModelScheduler, } from '@isis/3d-inference-local'; const analytics = createLocal3DGenerationAnalyticsDashboard(); const scheduler = createLocal3DMultiModelScheduler({ gpuDevices, models, runtime, }); const monitor = new Local3DGpuUtilizationMonitor({ analytics, scheduler, gpuObservationWindowMs: 15 * 60_000, idleModelThresholdMs: 10 * 60_000, onAlertEvent: ({ transition, alert }) => { console.log(transition, alert.kind, alert.message); }, }); const cycle = await monitor.reconcile(); console.log(cycle.activeAlerts.map((alert) => alert.id)); console.log(cycle.idleModelObservations[0]?.idleDurationMs); ``` ## Model Sharding Across Multiple GPUs `Local3DModelShardingManager` adds tensor-parallel planning and reservation tracking for models that do not fit on a single GPU: - it estimates full-model VRAM with the same scheduling heuristic already used by the single-GPU scheduler - it chooses the smallest viable shard count that fits the cluster, while preferring homogeneous GPU sets over mixed hardware groups - it produces an explicit tensor-parallel plan with split dimension, collective ops, per-shard reservations, and load/unload hooks for the runtime layer ```ts import { Local3DModelShardingManager, type Local3DModelShardingRuntime, } from '@isis/3d-inference-local'; const runtime: Local3DModelShardingRuntime = { async loadShardedModel(model, context) { console.log(model.key, context.plan.shardCount, context.plan.gpuIds); }, async unloadShardedModel(model, context) { console.log('unload', model.key, context.reason); }, }; const manager = new Local3DModelShardingManager({ gpuDevices: [ { id: 0, name: 'RTX 4090', vramBudgetMb: 24_576, reserveMb: 0, modelAffinity: [], }, { id: 1, name: 'RTX 4090', vramBudgetMb: 24_576, reserveMb: 0, modelAffinity: [], }, { id: 2, name: 'RTX 4090', vramBudgetMb: 24_576, reserveMb: 0, modelAffinity: [], }, ], models: [largeModelDescriptor], runtime, }); const decision = await manager.schedule({ modelKey: largeModelDescriptor.key, request: { prompt: 'polished brass astrolabe', quality: 'high', }, }); console.log(decision.plan.parallelismMode); console.log(decision.plan.shardCount); console.log( decision.shards.map((shard) => [shard.gpuId, shard.estimatedVramMb]) ); ``` ## Model Quantization Planning `Local3DModelQuantizationPlanner` adds shared INT8 and INT4 planning across the self-hosted model families without forcing each loader to reinvent quantization rules: - it understands every built-in 3D family shipped by the package - it models calibration requirements, preserved modules, artifact outputs, and backend dependencies - it produces quantized runtime environment wiring plus VRAM and throughput estimates on top of an existing self-hosted load plan ```ts import { Local3DModelQuantizationPlanner, Local3DModelRegistry, Local3DTrellis2ModelLoader, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const loader = new Local3DTrellis2ModelLoader({ registry }); const basePlan = await loader.resolveLoadPlan({ precision: 'bf16', lowVram: false, }); const quantizer = new Local3DModelQuantizationPlanner(); const int8Plan = quantizer.plan(basePlan, { precision: 'int8', calibrationSampleCount: 256, }); console.log(int8Plan.estimatedRuntimeVramMb); console.log(int8Plan.quantization.artifacts[0]?.absolutePath); console.log(int8Plan.environment.LOCAL_3D_QUANTIZATION_BACKEND); ``` ## TensorRT Optimization Planning `Local3DModelTensorRTPlanner` adds a shared NVIDIA TensorRT planning surface for the self-hosted 3D runtimes instead of scattering ad hoc compile settings across individual model loaders: - it models family-specific dynamic-shape profiles for image, texturing, and remeshing paths - it plans persistent Torch-TensorRT program, engine-cache, timing-cache, and calibration-cache artifacts per GPU architecture - it exposes compile settings, environment wiring, VRAM reduction, and expected speedup on top of an existing load plan ```ts import { Local3DModelRegistry, Local3DModelTensorRTPlanner, Local3DTrellis2ModelLoader, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const loader = new Local3DTrellis2ModelLoader({ registry }); const basePlan = await loader.resolveLoadPlan({ precision: 'bf16', }); const tensorrt = new Local3DModelTensorRTPlanner(); const optimizedPlan = tensorrt.plan(basePlan, { precision: 'fp16', capability: 'image-to-3d', targetGpu: { id: 0, name: 'NVIDIA L40S', computeCapability: '8.9', vramBudgetMb: 49_152, }, }); console.log(optimizedPlan.tensorrt.estimatedSpeedup); console.log(optimizedPlan.tensorrt.compileSettings.engineCacheDir); console.log(optimizedPlan.environment.LOCAL_3D_TENSORRT_PROGRAM_PATH); ``` ## ONNX Runtime Planning `Local3DModelOnnxRuntimePlanner` adds a cross-platform ONNX Runtime planning surface for the self-hosted 3D model families: - it maps each supported capability onto platform-valid execution providers for CUDA, DirectML, CoreML, and CPU fallback - it plans optimized ORT artifacts plus serialized session/provider profiles for deployment reuse - it exposes provider-aware precision fallback, graph optimization defaults, runtime dependencies, and environment wiring on top of an existing load plan ```ts import { Local3DModelOnnxRuntimePlanner, Local3DModelRegistry, Local3DSf3DModelLoader, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const loader = new Local3DSf3DModelLoader({ registry }); const basePlan = await loader.resolveLoadPlan({ precision: 'fp16', }); const onnx = new Local3DModelOnnxRuntimePlanner(); const optimizedPlan = onnx.plan(basePlan, { capability: 'image-to-3d', targetOs: 'windows', targetArch: 'x64', }); console.log(optimizedPlan.onnxruntime.provider); console.log(optimizedPlan.onnxruntime.sessionOptions.graphOptimizationLevel); console.log(optimizedPlan.environment.LOCAL_3D_ONNX_OPTIMIZED_MODEL_PATH); ``` ## Compile Optimization Planning `Local3DModelCompileOptimizationPlanner` centralizes `torch.compile` planning for the PyTorch-native generation families and exposes the specialized `stable-fast` path for SF3D: - it limits compile planning to generation-oriented model families instead of pretending remeshing-only runtimes should be compiled the same way - it reuses existing runner wiring like Hunyuan3D 2.1’s `--compile` switch where the repo already exposes one - it models TorchInductor cache directories, stable-fast runtime dependencies, warmup costs, and expected steady-state latency gains on top of a load plan ```ts import { Local3DModelCompileOptimizationPlanner, Local3DModelRegistry, Local3DSf3DModelLoader, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const loader = new Local3DSf3DModelLoader({ registry }); const basePlan = await loader.resolveLoadPlan({ precision: 'fp16', }); const compiler = new Local3DModelCompileOptimizationPlanner(); const optimizedPlan = compiler.plan(basePlan, { backend: 'stable-fast', capability: 'image-to-3d', }); console.log(optimizedPlan.compilation.estimatedSpeedup); console.log(optimizedPlan.environment.LOCAL_3D_COMPILE_BACKEND); console.log(optimizedPlan.runtimeDependencies.map((dep) => dep.dependencyId)); ``` ## MeshAnything V2 Speculative Decoding `Local3DMeshAnythingV2SpeculativeDecodingPlanner` adds MeshAnything-specific draft/verify planning for the autoregressive retopology path: - it pairs the published MeshAnything V2 verifier with the same `facebook/opt-350m` bootstrap model family as a lightweight draft decoder - it estimates token budgets from the supported 200/400/800/1600-face buckets and emits draft-window, verification-batch, and acceptance-threshold settings - it exposes reusable artifact locations plus deterministic prefix-reconciliation helpers for verifier rollback handling ```ts import { Local3DMeshAnythingV2SpeculativeDecodingPlanner, Local3DModelRegistry, Local3DMeshAnythingV2ModelLoader, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const loader = new Local3DMeshAnythingV2ModelLoader({ registry }); const basePlan = await loader.resolveLoadPlan(); const planner = new Local3DMeshAnythingV2SpeculativeDecodingPlanner(); const speculative = planner.plan(basePlan, { targetFaceCount: 1600, }); console.log(speculative.speculativeDecoding.estimatedSpeedup); console.log(speculative.environment.LOCAL_3D_SPECULATIVE_DRAFT_MODEL); console.log(speculative.speculativeDecoding.artifacts[0]?.absolutePath); ``` ## Inference Result Caching `Local3DInferenceResultCache` caches completed local 3D inference outputs so repeated requests can short-circuit expensive generation work: - it hashes normalized request inputs together with model key, model version, optional family, and cache salt so prompt-identical runs only hit when the runtime identity also matches - it supports both text-to-3D and image-to-3D requests, hashing image bytes rather than Buffer identity for deterministic cache keys - it stores full completion payloads with TTL, byte-budget, and LRU eviction semantics so repeated results can be returned without rerunning inference ```ts import { Local3DInferenceResultCache, buildLocal3DInferenceCacheKey, } from '@isis/3d-inference-local'; const cache = new Local3DInferenceResultCache({ defaultTtlMs: 30 * 60 * 1000, }); const cacheKey = buildLocal3DInferenceCacheKey({ modelKey: 'trellis-2', modelVersion: '2.0.1', request: { prompt: 'ornate shield', quality: 'high', }, }); const hit = cache.get({ modelKey: 'trellis-2', modelVersion: '2.0.1', request: { prompt: 'ornate shield', quality: 'high', }, }); console.log(cacheKey); console.log(hit.hit); ``` ## Progressive Generation With Early Preview `Local3DProgressiveGenerationCoordinator` adds a shared two-stage orchestration surface for returning a draft preview quickly and then refining to the final asset: - it applies a conservative preview profile by default, using `draft` quality, `glb` output, a `400`-face target, `512` texture resolution, and textures disabled so the preview path stays on the fast side of a 2-second target - it preserves the original request for the refinement pass, emits typed progress updates for preview preparation, preview readiness, refinement, and final completion, and records whether the preview met the requested deadline - it merges preview affordances such as `previewUrl` and thumbnails into the final completion payload so clients can keep showing the early artifact while higher-quality geometry and textures arrive ```ts import { Local3DProgressiveGenerationCoordinator } from '@isis/3d-inference-local'; const coordinator = new Local3DProgressiveGenerationCoordinator({ previewRunner: async ({ request }) => ({ previewUrl: `https://cdn.local/previews/${request.quality}.glb`, }), finalRunner: async () => ({ models: [{ data: Buffer.from('final-glb'), format: 'glb' }], }), }); const result = await coordinator.run( { jobId: 'job-42', request: { prompt: 'ornate bronze helmet', quality: 'high', textureResolution: 2048, generateTextures: true, }, }, { onProgress(update) { console.log(update.stage, update.progressPct); }, } ); console.log(result.firstPreviewLatencyMs); console.log(result.mergedCompletion.previewUrl); console.log(result.final.completion.models?.[0]?.format); ``` ## SF3D UV Unwrap Output `Local3DSf3DUvUnwrapPipeline` exposes the released SF3D UV unwrap as an explicit artifact instead of leaving it implicit inside the textured GLB: ```ts import { Local3DModelRegistry, createLocal3DSf3DUvUnwrapPipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const pipeline = createLocal3DSf3DUvUnwrapPipeline({ registry, runtimeRoot: '/opt/stable-fast-3d', }); const result = await pipeline.run({ images: [inputImageBuffer], meshTopology: 'quad-dominant', textureResolution: 2048, }); ``` This task is grounded in the official Stability AI UV path reviewed on `2026-03-26`: - `sf3d/models/mesh.py` imports the native `uv_unwrapper` package and calls `Mesh.unwrap_uv(...)` - the unwrap step duplicates seam vertices and stores per-vertex UVs on `mesh.v_tex` - `sf3d/system.py` then bakes textures against those UVs before constructing the textured `trimesh` export The Oshun UV export keeps that official unwrap intact and adds explicit output artifacts on top: - the pipeline still returns the textured `glb` produced by the released SF3D runtime so downstream tools keep the same mesh/material representation - it also renders a `uv-layout.png` artifact directly from the official mesh UVs and exposes it as a provider `texture` asset with `textureType=uv-layout` - metadata includes UV vertex count, UV face count, UV bounds, and a UV area coverage estimate so later tasks can validate unwrap quality and atlas use This is intentionally different from a generic downstream UV pass. The UV layout is derived from the exact Stability AI mesh that already passed through `uv_unwrapper`, which is the main differentiator called out in Phase 71 versus weaker image-to-3D baselines that skip a production-grade unwrap surface. ## SF3D Illumination-Disentangled Texture Output `Local3DSf3DIlluminationTexturePipeline` promotes the released SF3D material channels into explicit provider assets so the lighting-neutral texture is available without scraping the final GLB: ```ts import { Local3DModelRegistry, createLocal3DSf3DIlluminationTexturePipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const pipeline = createLocal3DSf3DIlluminationTexturePipeline({ registry, runtimeRoot: '/opt/stable-fast-3d', }); const result = await pipeline.run({ images: [inputImageBuffer], textureResolution: 2048, }); ``` This task is grounded in the official Stability AI material assembly reviewed on `2026-03-26`: - `sf3d/system.py` assigns `decoded["features"]` to `mat_out["albedo"]` - the same code path computes `roughness`, `metallic`, and tangent-space normal data before constructing `trimesh.visual.material.PBRMaterial` - the final GLB therefore already contains an illumination-neutral base color texture rather than a simple lit screenshot projection The Oshun export surfaces those channels directly: - `albedo` is saved as a standalone texture asset and marked in metadata as `albedoLightingDisentangled=true` - `normal` is exported when present from the released material stack - scalar `roughnessFactor` and `metallicFactor` are preserved in metadata for downstream material authoring or validation - the textured `glb` remains part of the completion payload so the mesh and texture package stay consistent with the official SF3D runtime This gives downstream tooling a proper lighting-neutral texture handoff, which is the Phase 71 differentiator here: not just “a textured mesh”, but the actual released albedo channel that SF3D decodes before material assembly. ## Cost Dashboard `Local3DCostDashboard` can ingest tracked jobs from `Local3DProviderRestApi` and expose an API-first unit-economics snapshot: ```ts const costDashboard = createLocal3DCostDashboard(); const providerApi = createLocal3DProviderRestApi({ capabilities: createLocal3DProviderCapabilities([ 'text-to-3d', 'image-to-3d', ]), costDashboard, }); ``` Once jobs are flowing, query the dashboard at: ```text GET /v1/3d/costs/dashboard?selfHostedProfileId=runpod-l4-flex&cloudProfileIds=meshy-pro,meshy-studio ``` The default dashboard ships with: - self-hosted deployment profiles based on current RunPod serverless GPU pricing such as `runpod-l4-flex` and `runpod-a40-flex` - cloud API comparison profiles for Meshy Pro and Meshy Studio using current stage-credit pricing - per-task and aggregate comparisons showing self-hosted total cost, average cost per asset, estimated external API spend, and savings deltas You can override both the self-hosted profiles and cloud pricing catalog when constructing the dashboard, so teams using on-prem GPUs, different storage retention windows, or other commercial APIs can plug in their own economics model without changing the REST contract. ## Hunyuan Quality Dashboard `Local3DHunyuan3D21QualityComparisonDashboard` provides a deterministic, API-first benchmark snapshot for comparing the local Hunyuan3D 2.1 shape-plus-PBR stack against the current Meshy-6 PBR baseline surface: ```ts const hunyuanQualityDashboard = createLocal3DHunyuan3D21QualityComparisonDashboard(); const providerApi = createLocal3DProviderRestApi({ capabilities: createLocal3DProviderCapabilities([ 'text-to-3d', 'image-to-3d', ]), hunyuanQualityDashboard, }); ``` Query it at: ```text GET /v1/3d/quality/hunyuan-vs-meshy/dashboard?category=hard-surface&modality=image-to-3d-pbr&limit=3 ``` The default snapshot ships with: - six curated text-to-3d + PBR and image-to-3d + PBR benchmark cases - weighted scoring across geometry fidelity, topology cleanliness, texture fidelity, material realism, prompt alignment, PBR completeness, lighting neutrality, and asset readiness - threshold gates that summarize whether the local Hunyuan stack still clears the target lead over Meshy-6 in average score, win rate, and PBR/material deltas - provider source references captured on `2026-03-26` from the official Tencent Hunyuan3D repository and current Meshy retexture API documentation The dashboard is intentionally deterministic rather than a live online eval. It is designed for release gating, regression tracking, and API-level reporting, while still allowing teams to swap in their own benchmark dataset and scoring thresholds. ## RunPod Serverless Proxy Mode For bursty workloads that should scale to zero between requests, the same local HTTP surface can now proxy jobs to a RunPod Serverless endpoint while keeping the Isis-compatible `/v1/3d` REST API and SSE progress stream online. Set the following container environment variables: ```text LOCAL_3D_RUNPOD_ENDPOINT_ID=your-endpoint-id LOCAL_3D_RUNPOD_API_KEY=your-runpod-api-key LOCAL_3D_RUNPOD_SYNC_MODE=interactive LOCAL_3D_RUNPOD_SYNC_TIMEOUT_SECONDS=90 LOCAL_3D_RUNPOD_EXECUTION_TIMEOUT=600 LOCAL_3D_RUNPOD_TTL=3600 ``` When RunPod proxy mode is enabled: - interactive jobs can use `/runsync` while batch/background jobs use `/run` plus `/status` polling - local task IDs remain stable even though the adapter tracks a separate remote RunPod job ID internally - completed RunPod outputs can return encoded model/texture assets that the adapter hydrates back into local downloadable result assets - health checks treat zero-worker idle states as healthy, which matches RunPod's scale-to-zero behavior instead of reporting false failures The adapter also exports request/result serializers (`createLocal3DRunpodJobInput`, `parseLocal3DRunpodJobInput`, and completion encode/decode helpers) so a RunPod worker can speak the same envelope format as the local gateway. ## TripoSG Loader The package now also exposes `Local3DTripoSGModelLoader` for Tripo’s published 1.5B rectified-flow image-to-3D runtime: ```ts import { Local3DModelRegistry, createLocal3DTripoSGModelLoader, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const triposg = createLocal3DTripoSGModelLoader({ registry }); const plan = await triposg.resolveLoadPlan({ gpuDevice: { id: 0, computeCapability: '8.9', vramBudgetMb: 12_288, }, }); ``` This loader is grounded in Tripo’s published repo and Hugging Face artifacts: - `VAST-AI/TripoSG` publishes the diffusers `TripoSGPipeline` bundle with the Dinov2 encoder, rectified-flow transformer, scheduler, and VAE checkpoints - the official `scripts.inference_triposg` entrypoint also downloads `briaai/RMBG-1.4` and runs `BriaRMBG.from_pretrained(...)` before `prepare_image(...)` - the upstream README publishes an 8GB CUDA minimum; Oshun keeps that as the hard floor and models ~12GB as an operational headroom target for the full TripoSG+RMBG runtime The resolved load plan includes: - local paths for `model_index.json`, transformer, VAE, Dinov2 encoder, scheduler, and RMBG checkpoints - runtime defaults for `num_inference_steps=50`, `guidance_scale=7.0`, `num_tokens=2048`, and the published octree / flash-decoder settings - mixed-source dependency metadata covering the MIT TripoSG weights plus the separate BRIA RMBG 1.4 background-removal license - an explicit note that the underlying pipeline accepts list-image batches even though the released CLI only exposes a single-image entrypoint today ## TripoSG Image-to-3D Pipeline The package now also exposes a runnable local TripoSG image-to-3D pipeline on top of the loader: ```ts import { Local3DModelRegistry, createLocal3DTripoSGImageTo3DPipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const triposgPipeline = createLocal3DTripoSGImageTo3DPipeline({ registry, runtimeRoot: '/opt/TripoSG', }); const result = await triposgPipeline.run({ images: [inputImageBuffer], quality: 'standard', outputFormat: 'glb', guidanceScale: 7, seed: 42, }); ``` This pipeline follows the released TripoSG flow closely: - it loads `BriaRMBG` from the separately downloaded `briaai/RMBG-1.4` dependency and feeds the image through the published `prepare_image(...)` foreground-isolation path - it runs the official `TripoSGPipeline.from_pretrained(...)` stack in `fp16` on CUDA with the published 50-step / 7.0-guidance defaults - it can optionally simplify the generated triangle mesh natively, and now automatically hands quad-dominant or `targetPolyCount` requests to MeshAnything V2 so downstream topology control is implemented instead of being warning-only intent The resolved result includes: - geometry-only mesh assets in `glb`, `obj`, or `ply` format - the prepared foreground image used for inference so callers can inspect the actual crop fed into TripoSG - per-run metadata for preprocessing, inference, and optional simplification timings plus mesh vertex / face counts - explicit warnings when callers supply prompt, negative-prompt, or texture controls the released runtime cannot honor directly, plus MeshAnything post-process metadata when topology cleanup was applied automatically ## TripoSG-Scribble Image-to-3D Pipeline The package also exposes a TripoSG-scribble pipeline for rough sketches and rapid concept iteration: ```ts import { Local3DModelRegistry, createLocal3DTripoSGScribbleImageTo3DPipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const scribblePipeline = createLocal3DTripoSGScribbleImageTo3DPipeline({ registry, runtimeRoot: '/opt/TripoSG', }); const result = await scribblePipeline.run({ images: [scribbleBuffer], prompt: 'a cat with wings', quality: 'standard', scribbleConfidence: 0.4, promptConfidence: 1.0, }); ``` This variant follows the published TripoSG-scribble release closely: - it downloads `VAST-AI/TripoSG-scribble`, which adds a tokenizer and CLIP text encoder on top of the TripoSG base image-conditioning stack - it keeps `guidance_scale=0` because the upstream model is explicitly CFG-distilled and instead exposes prompt/scribble weighting through cross-attention scaling - it preserves the official `use_flash_decoder=False` and 8/8 octree settings from the released inference script so the local runtime does not reintroduce the boundary artifacts noted upstream - it automatically routes quad-dominant or `targetPolyCount` requests through MeshAnything V2 after the released scribble-conditioned mesh path finishes The pipeline result includes: - geometry-only `glb`, `obj`, or `ply` outputs for rapid concept review - the prepared RGB sketch image that was actually fed into the model - per-run metadata for prompt text, confidence settings, denoising steps, timings, and mesh face / vertex counts - validation and warnings for unsupported negative-prompt and texturing controls, MeshAnything post-process metadata for topology cleanup requests, and guidance when scribble confidence moves outside the upstream-recommended 0.3-0.5 range ## TripoSG Multi-View Consistency Mode The package also exposes a TripoSG multi-view consistency pipeline for 2-8 reference views of the same object: ```ts import { Local3DModelRegistry, createLocal3DTripoSGMultiViewConsistencyPipeline, } from '@isis/3d-inference-local'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const pipeline = createLocal3DTripoSGMultiViewConsistencyPipeline({ registry, runtimeRoot: '/opt/TripoSG', }); const result = await pipeline.run({ images: [frontBuffer, leftBuffer, backBuffer, rightBuffer], quality: 'standard', consistencySampleCount: 2048, }); ``` This mode is intentionally honest about what upstream TripoSG does and does not ship: - the released `TripoSGPipeline` accepts list-image batches, but it still reconstructs one mesh per image instead of doing true fused multi-view conditioning - Oshun therefore uses shared-seed per-view generation and then computes a pairwise surface-distance agreement score across the candidate meshes - the final output is the geometric medoid, which is the candidate mesh with the lowest mean pairwise Chamfer-style distance to the other reconstructed views - once that medoid is selected, Oshun can now automatically apply MeshAnything V2 when the request asks for quad-dominant topology or a face-budget target The result includes: - the selected final `glb`, `obj`, or `ply` mesh - all prepared reference-view images used for conditioning - per-view consistency metrics, selected-view index, and cross-view agreement scores in metadata - automatic micro-batching when more than four views are supplied so Oshun stays inside the published TripoSG family’s practical VRAM envelope ## TripoSG vs TripoSR Comparison The package also exposes a comparison harness for validating the local TripoSG surface against the repo’s older TripoSR-style provider baseline: ```ts import { Local3DModelRegistry, createLocal3DTripoSGComparisonPipeline, } from '@isis/3d-inference-local'; import { createTripoProvider } from '@isis/3d-generation/providers'; const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d', }); const report = await createLocal3DTripoSGComparisonPipeline({ registry, runtimeRoot: '/opt/TripoSG', baselineProvider: createTripoProvider({ apiKey: process.env.TRIPO_API_KEY }), }).run({ images: [referenceImageBuffer], }); ``` The report is intentionally explicit about evidence quality: - it runs the local TripoSG image-to-3D path and the injected Tripo baseline in parallel on the same request - it marks whether topology improvement was validated, mixed, baseline- competitive, or inconclusive - it assigns a confidence level based on whether baseline topology metrics came from provider metadata or a custom measured extractor - it separates texture parity from the topology verdict so the current geometry-only TripoSG surface is not misrepresented as a texture-complete replacement