Self-hosted 3D inference primitives for the Isis domain.
This library is the local runtime kernel for Phase 71. It provides:
- Nx project metadata and build/test/lint wiring
- strongly typed contracts for local 3D model descriptors, GPU server config, queueing, warm-up, and lifecycle management
- provider interoperability helpers so self-hosted models can plug into the
existing
@isis/3d-generationprovider surface - disk-aware model weight caching with access tracking, cache inspection, and LRU eviction for large local model installs
- per-GPU VRAM budget management with pressure detection and automatic model offloading hooks for local multi-model serving
- capability-aware multi-model scheduling that selects, loads, reuses, and unloads text-to-3D, image-to-3D, and texturing models based on incoming request mix
- a starvation-aware request queue with interactive, batch, and background priority classes to prepare later batching and multi-GPU dispatch
- a batch aggregation layer that groups queue-compatible 3D requests into throughput-oriented activation batches using a configurable aggregation window
- a multi-GPU load balancer that routes requests across VRAM-aware schedulers while honoring soft affinity and hard per-model GPU pinning
- explicit VRAM-tier placement that keeps ~6GB models on consumer GPUs, routes 24GB-class workloads onto A100/H100-style accelerators when available, and skips GPUs that can never satisfy a model's VRAM floor
- a model prewarm manager that keeps hot models resident in VRAM, demotes cold models into disk-backed offloaded state for fast reloads, and plugs directly into the existing warmup-handler surface on the local GPU model server
- a queue-depth autoscaler that watches 3D generation backlog, chooses the highest-priority RunPod inference/shared pool with spare headroom, and emits threshold/cooldown-aware scale-up requests through a provisioner callback
- spot/preemptible-capacity-aware pool routing so batch/background 3D jobs prefer cheaper interruptible GPU pools with automatic on-demand fallback while interactive traffic stays on stable on-demand capacity by default
- a GPU utilization monitor that correlates time-windowed utilization samples with scheduler residency, raises lifecycle-aware alerts for underutilized GPUs and idle resident model instances, and exposes poll/reconcile APIs for external alert delivery
- a GPU node-pool manager that keeps 3D inference capacity isolated from training workloads, tracks Kubernetes/RunPod/manual pool membership, and exposes pool-aware node selection plus heartbeat-backed capacity snapshots
- Meshy-compatible SSE progress streams with resumable
Last-Event-IDreplay, task-scoped stream aliases, and typed progress publishing APIs - a unified provider-oriented REST API for capabilities, text/image job submission, status polling, cancellation, per-job streams, and result asset downloads
- package-local CUDA container assets for NVIDIA Container Toolkit deployment, including a Docker build, compose stack, and env-driven runtime bootstrap
- a package-local Helm chart for Kubernetes GPU deployments with NVIDIA node selectors, GPU resource requests/limits, persistent model/cache volumes, and optional runtime-module mounts
- RunPod serverless adapters that proxy the local Isis-compatible REST/SSE API
onto RunPod
/run,/runsync,/status,/cancel, and/healthendpoints for scale-to-zero operation - a built-in cost dashboard that tracks self-hosted cost per asset and compares it against configurable cloud API pricing profiles such as Meshy plans
- a dependency-aware TRELLIS.2 loader that registers the official Microsoft checkpoints, resolves the cross-repo sparse-structure decoder dependency, and produces bf16/fp16 runtime load plans with low-VRAM guidance
- a runnable TRELLIS text-to-3D mesh pipeline that patches the official Microsoft text-conditioned checkpoint bundle into a fully local runtime bundle, executes the upstream Python pipeline, and emits staged progress for text conditioning, sparse-structure sampling, SLat sampling, and mesh extraction
- a runnable TRELLIS.2 single-image reconstruction pipeline that patches the official Microsoft image-conditioned checkpoint bundle into a fully local runtime bundle and emits staged progress for preprocessing, image conditioning, sparse-structure sampling, shape SLat sampling, and mesh extraction
- a runnable TRELLIS.2 multi-image reconstruction pipeline that accepts 2-6 reference views, fuses the strongest views into one conditioning sequence, and returns raw mesh output with explicit selected-view metadata
- a runnable TRELLIS.2 PBR extraction pipeline that bakes
albedo,metallic,roughness,alpha, and combined metallic-roughness textures from the official TRELLIS.2 texturing pipeline - a runnable TRELLIS.2 Gaussian splatting pipeline that projects TRELLIS.2
mesh+appearance outputs into GraphDECO-compatible
.plysplats for real-time viewing workflows - a runnable TRELLIS.2 GLB/GLTF export pipeline that packages textured meshes as single-file GLB or embedded-resource GLTF assets
- preset-driven TRELLIS.2 quality controls that map
draft,standard,high, andultraonto released pipeline resolutions, extraction grids, and token/step budgets - a TRELLIS.2 prompt adapter that converts Isis prompt templates into object-centric reference-image prompts and structured guidance metadata for the released image-conditioned runtime
- deterministic TRELLIS.2 seed control with normalized replayable seeds across mesh, texturing, GLTF, and Gaussian export paths
- TRELLIS.2 negative prompt support that converts Isis exclusions into upstream reference-image avoidance guidance for the released image-conditioned runtime
- TRELLIS.2 style conditioning presets for
photorealistic,stylized,low-poly, andcartoonprompt-to-image guidance - a deterministic TRELLIS.2 benchmark suite that scores local quality against Meshy 6, Rodin Gen-1.5, and Tripo v2.5 commercial baselines with weighted case metrics and threshold gates
- TRELLIS.2 update automation that polls Microsoft GitHub, falls back from releases to tags to branch-head commits when needed, downloads fresh runtime source archives, and syncs the paired Hugging Face checkpoints
- a Hunyuan3D 2.1 shape loader that registers Tencent’s official
tencent/Hunyuan3D-2.1checkpoint bundle, restricts downloads to the shape subfolder, and resolves standard, low-VRAM, and 6GB mmgp load plans with offline runtime hints - a Hunyuan3D 2.1 MMGP coordinator that stages text, shape, paint, super-resolution, and bake phases through the existing GPU memory manager so 6GB-class consumer GPUs can execute the published Hunyuan flows sequentially
- a Hunyuan3D 2.1 Blender addon integration that installs as a single
.pyaddon, submits text/image jobs through the local provider API, polls job status, and imports completed models back into the active Blender scene - a Hunyuan3D 2.1 text-to-shape pipeline that follows Tencent’s published flow of HunyuanDiT prompt-to-image generation, optional background removal, and local Hunyuan3D 2.1 shape reconstruction with preview-image capture
- a Hunyuan3D 2.1 image-to-shape pipeline for direct image-conditioned shape reconstruction, including background-normalized preview capture, quality presets, and explicit anchor-image selection when callers provide multiple references to a single-image-conditioned runtime
- Hunyuan3D 2.1 batch generation for text-to-shape and image-to-shape runs, with compatibility-aware micro-batching on standard-memory GPUs and honest warm-sequential fallback when low-VRAM or mmgp staging is required
- Hunyuan3D 2.1 mesh topology control for geometry-only OBJ output, including a documented PyMeshLab quad-dominant post-process on top of Tencent’s released triangle mesh generation path
- a Hunyuan3D 2.1 style fine-tuning pipeline that prepares Tencent-compatible HunyuanDiT LoRA datasets, drives the official native DeepSpeed trainer, and emits reusable adapter manifests for local text-to-shape runs
- an SF3D loader that registers Stability AI’s gated
stable-fast-3dweights, resolves consumer-GPUfp16and nativebf16load plans, and documents the compiled UV unwrap / texture baking runtime surface needed for later sub-second image-to-3D execution - a SPAR3D loader that registers Stability AI’s gated
stable-point-aware-3dweights, resolves standard vs low-VRAM execution plans, and exposes the editable 512-pointxyzrgbintermediate representation used between point diffusion and final mesh extraction - a Sparc3D loader for operator-supplied private pretrained bundles, with local-filesystem or private Hugging Face mirror sources, bundle-manifest validation, and explicit 1024³ readiness checks instead of pretending public checkpoints exist
- a Sparc3D image-to-3D pipeline that executes an operator-supplied Python entrypoint from the imported private bundle, scales between 256³, 512³, and 1024³ resolution plans for quality-vs-speed tradeoffs, and returns provider-compatible GLB/OBJ/PLY assets
- strict Sparc3D watertight-manifold validation that independently inspects OBJ
and ASCII PLY topology, rejects meshes with holes or non-manifold edges in
surfaceMode: 'watertight', and records print-ready diagnostics for 3D printing workflows - explicit Sparc3D open-surface analysis that preserves thin sheets and disconnected shells as first-class outputs, requires topology sidecars for GLB exports, and records boundary/component diagnostics instead of forcing everything into closed solids
- a deterministic Sparc3D geometric-fidelity benchmark that compares the local Sparc3D stack against local TRELLIS.2 and Hunyuan3D 2.1 baselines on watertight, open-surface, thin-structure, and printable reconstruction cases
- a MeshAnything V2 topology-cleanup handoff for Sparc3D that retopologizes the
operator mesh through the released
main.pyCLI, preserves OBJ analysis sidecars for validation, and rechecks watertight/open-surface guarantees after cleanup instead of trusting the post-process blindly - a Kaolin FlexiCubes mesh-extraction path that can remesh finalized geometry from TRELLIS text/image/multi-image, Hunyuan3D 2.1 text/image/batch shape runs, TripoSG image/scribble/multi-view, and Sparc3D, while explicitly rejecting texture-first SF3D and SPAR3D requests where remeshing would discard baked materials
- automatic MeshAnything V2 post-processing across TRELLIS text/image/multi-image, Hunyuan3D 2.1 text/image/batch shape runs, Sparc3D cleanup handoffs, and the TripoSG family whenever callers ask for topology controls those native runtimes do not honestly expose on their own
- a TripoSG 1.5B loader that registers the official
VAST-AI/TripoSGrectified-flow weights plus the requiredbriaai/RMBG-1.4background-removal dependency, preserves the upstream 8GB CUDA floor, and models a more realistic ~12GB operating envelope for end-to-end local runs - a runnable TripoSG image-to-3D pipeline that follows the released RMBG
foreground-isolation and
TripoSGPipelinesampling path, supports step / guidance / seed / face-count control, and returns geometry-only GLB, OBJ, or PLY assets - a runnable TripoSG-scribble sketch-to-3D pipeline that follows the released CFG-distilled prompt+scribble conditioning path, exposes scribble/prompt confidence controls, and returns rapid-prototyping geometry assets without enabling the flash decoder that upstream flags for boundary artifacts
- a TripoSG multi-view consistency mode that honestly layers on top of the published single-image pipeline by batching per-view reconstructions and reranking them with geometric medoid scoring instead of claiming a non-public fused multi-view checkpoint
- a TripoSG vs TripoSR comparison harness that runs the local TripoSG surface against the repo’s legacy Tripo baseline and returns an explicit validation report with verdict and confidence instead of hand-wavy quality claims
- a runnable SPAR3D single-image pipeline that follows the released
image-to-point-cloud-to-mesh path, exports the intermediate
points.plyartifact alongside the textured GLB, and supports seed plus guidance control for the point diffusion stage - a runnable SPAR3D point-cloud editing pipeline that accepts generated or
uploaded
plypoint clouds, applies translation / scale / recolor edits to the released 512-pointxyzrgbrepresentation, and remeshes the edited cloud back into a textured GLB - a persistent SPAR3D preview loop that keeps the released runtime loaded, reuses the prepared image and current point cloud across iterations, and regenerates interactive preview meshes from edited point clouds without paying cold-start model load cost on every edit
- a runnable SF3D single-image pipeline that follows Stability AI’s released
background-removal, foreground-cropping,
run_image(...), and textured-GLB export path while surfacing precision downgrades and textured completion assets through the local provider contract - a speed-optimized SF3D batch image-to-3d pipeline that uses the released
list-image
run_image(...)path, compatibility-aware micro-batching, and VRAM-capped throughput heuristics to target 100+ images/minute on fast-path settings - a quality-vs-speed router that automatically keeps latency-sensitive image requests on SF3D, promotes fidelity-sensitive requests onto TRELLIS.2, and composes TRELLIS geometry with the released GLB/GLTF export stage so the quality route still returns delivery-ready assets
- an SF3D UV unwrap export path that derives a UV layout artifact directly from the official Stability AI unwrapped mesh and returns it as a provider texture asset alongside the textured GLB
- an SF3D illumination-disentangled texture export that surfaces the released albedo texture directly from the model’s material decoder, alongside normal, roughness, and metallic outputs needed for downstream PBR workflows
Subsequent Phase 71 tasks build on this package for containerization, GPU orchestration, model-specific runtime integrations, and cost tracking.
TRELLIS.2 Loader#
Local3DTrellis2ModelLoader is the first model-specific runtime integration in
the package. It wraps the existing Hugging Face-backed Local3DModelRegistry
and bakes in the official Microsoft upstream layout:
microsoft/TRELLIS.2-4Bfor the main image-to-3D and texturing checkpointsmicrosoft/TRELLIS-image-largefor the sparse-structure decoder still referenced bypipeline.json- runtime-side dependencies on
facebook/dinov3-vitl16-pretrain-lvd1689mandbriaai/RMBG-2.0
Example:
import {
Local3DModelRegistry,
createLocal3DTrellis2ModelLoader,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const trellis = createLocal3DTrellis2ModelLoader({ registry });
const plan = await trellis.resolveLoadPlan({
precision: 'fp16',
gpuDevice: {
id: 0,
computeCapability: '8.9',
vramBudgetMb: 24_576,
},
});
The resolved plan includes:
- the installed main and sparse-structure checkpoint locations
- pipeline metadata from
pipeline.jsonandtexturing_pipeline.json - per-component native precision and runtime precision
- automatic bf16 validation against GPU compute capability
flash-attnvsxformersattention backend selection- low-VRAM operating mode defaults and estimated runtime VRAM requirements
The default runtime profile is bf16 with an estimated 16GB VRAM requirement.
Requesting fp16 switches the loader into low-VRAM mode and targets an 8GB
operating profile by downcasting the bf16 flow checkpoints at load time while
keeping the native fp16 decoder components unchanged.
Hunyuan3D 2.1 Shape Loader#
The package now also exposes Local3DHunyuan3D21ModelLoader for Tencent’s
official Hunyuan3D 2.1 shape-generation runtime:
import {
Local3DModelRegistry,
createLocal3DHunyuan3D21ModelLoader,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const hunyuan = createLocal3DHunyuan3D21ModelLoader({ registry });
const plan = await hunyuan.resolveLoadPlan({
allowMemoryDowngrade: true,
enableFlashVdm: true,
gpuDevice: {
id: 0,
computeCapability: '8.9',
vramBudgetMb: 12_288,
},
});
This loader is grounded in Tencent’s published Hunyuan3D 2.1 sources and model card:
- the Hugging Face repo
tencent/Hunyuan3D-2.1publishes the shape DiT subfolder, optional ShapeVAE files, and the separate PBR paint bundle - the official runtime entrypoint is
Hunyuan3DDiTFlowMatchingPipeline.from_pretrained(...) - Tencent’s README states a 10GB VRAM target for shape generation, while the
upstream runtime also exposes
low_vram_modeand CPU offload hooks
The resolved load plan includes:
- selective download of only
hunyuan3d-dit-v2-1/config.yamlandhunyuan3d-dit-v2-1/model.fp16.ckptso the loader does not pull the 21GB paint bundle just to run shape inference - offline runtime hints for
from_pretrained, including the local model path, subfolder, config path, checkpoint path, and the upstreamconditioner->model->vaeCPU offload order - three memory profiles:
standard(10GB),low-vram(8GB withenable_model_cpu_offload), andmmgp(6GB staged profile coordinated byLocal3DHunyuan3D21MmgpCoordinator) - auxiliary artifact metadata for the published ShapeVAE files so later fine-tuning and advanced VAE-specific workflows can reuse the same registry install metadata
Hunyuan3D 2.1 Text-to-Shape Pipeline#
The package now also exposes a runnable local text-to-shape pipeline on top of the Hunyuan loader:
import {
Local3DModelRegistry,
createLocal3DHunyuan3D21TextToShapePipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const pipeline = createLocal3DHunyuan3D21TextToShapePipeline({
registry,
runtimeRoot: '/opt/Hunyuan3D-2.1',
allowMemoryDowngrade: true,
});
const result = await pipeline.run({
prompt: 'weathered bronze astrolabe',
negativePrompt: 'busy background',
meshTopology: 'quad-dominant',
styleAdapterPath:
'/models/hunyuan-style-loras/porcelain-style/adapter-manifest.json',
styleAdapterScale: 0.8,
quality: 'high',
seed: 7,
});
This matches Tencent’s published text workflow:
- the Hunyuan3D 2.1 gradio app enables text-to-3D by first generating a
reference image from
Tencent-Hunyuan/HunyuanDiT-v1.1-Diffusers-Distilled - the generated image then goes through optional background removal before
Hunyuan3DDiTFlowMatchingPipeline.from_pretrained(...)reconstructs the shape - the Diffusers HunyuanDiT pipeline supports direct
negative_prompt,num_inference_steps, andguidance_scalecontrols, which Oshun maps from the text request surface
At runtime the Node pipeline:
- downloads the distilled HunyuanDiT Diffusers bundle plus the previously
defined Hunyuan3D 2.1 shape checkpoint into
Local3DModelRegistry - resolves the shape loader first so low-VRAM and 6GB mmgp fallbacks carry into the text flow automatically
- invokes a package-local Python runner that emits staged progress for text-to-image loading, image generation, background removal, shape sampling, mesh extraction, and final asset writes
- can fuse Tencent-native HunyuanDiT style LoRA adapters into the Diffusers
text-to-image stage by pointing
styleAdapterPathat either a checkpoint directory containingadapter_model.safetensorsor the manifest emitted by the fine-tuning pipeline below - defaults quad-dominant requests to
objand runs PyMeshLabmeshing_tri_to_quad_dominantas an explicit post-process, because Tencent’s released runtime still emits triangle meshes - routes
targetPolyCountrequests through an automatic MeshAnything V2 post-process after shape extraction so face-budget control is enforced instead of being treated as metadata-only intent - returns both the generated mesh and the reference PNG used for reconstruction, so callers can persist the exact image that conditioned the final shape
Hunyuan3D 2.1 Style LoRA Fine-Tuning#
Oshun now also packages a native HunyuanDiT LoRA fine-tuning pipeline for
project-specific asset styles. This task is grounded in Tencent’s official
HunyuanDiT LoRA workflow rather than an ad-hoc Diffusers training loop:
import {
Local3DModelRegistry,
createLocal3DHunyuan3D21FineTuningPipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const pipeline = createLocal3DHunyuan3D21FineTuningPipeline({
registry,
hyditTrainingRoot: '/opt/HunyuanDiT',
});
const result = await pipeline.run({
adapterName: 'Porcelain Style',
triggerPhrase: 'porcelain style',
examples: trainingExamples,
validationPrompts: ['garden pavilion', 'lotus flower'],
rank: 64,
maxTrainingSteps: 2_000,
});
The fine-tuning pipeline:
- downloads Tencent’s native
Tencent-Hunyuan/HunyuanDiT-v1.1t2i/modelcheckpoint layout throughLocal3DModelRegistryso the official trainer sees the samepytorch_model_ema.ptandpytorch_model_module.ptstructure it expects - writes a HunyuanDiT-compatible CSV caption file, converts it into Arrow shards
via Tencent’s
hydit/data_loader/csv2arrow.py, and emits the matching YAML index consumed byhydit/train_deepspeed.py - launches Tencent’s official DeepSpeed LoRA trainer with the published v1.1
style-conditioning recipe (
--use-style-cond,--size-cond 1024 1024,--beta-end 0.03) instead of pretending the local Diffusers-distilled checkpoint can be trained natively - packages the latest checkpoint into an
adapter-manifest.jsonfile whosecheckpointPathcan be fed straight back intoLocal3DHunyuan3D21TextToShapePipelinethroughstyleAdapterPath
Hunyuan3D 2.1 Image-to-Shape Pipeline#
The package now also exposes a direct image-conditioned shape pipeline:
import {
Local3DModelRegistry,
createLocal3DHunyuan3D21ImageToShapePipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const pipeline = createLocal3DHunyuan3D21ImageToShapePipeline({
registry,
runtimeRoot: '/opt/Hunyuan3D-2.1',
allowMemoryDowngrade: true,
});
const result = await pipeline.run({
images: [inputImageBuffer],
meshTopology: 'quad-dominant',
quality: 'high',
seed: 7,
});
This follows the released Hunyuan3D 2.1 image-conditioned runtime directly:
- it resolves the same Hunyuan3D 2.1 shape loader introduced in
71.1.3.1 - it uses Tencent’s
Hunyuan3DDiTFlowMatchingPipeline.from_pretrained(...)image path, with the same optional CPU-offload andflashvdmhooks - it can post-process the generated triangle mesh into a quad-dominant
objusing PyMeshLab when callers request preserved face topology for downstream DCC workflows - it now also routes
targetPolyCountrequests through an automatic MeshAnything V2 remesh pass so face-count control works for image-conditioned runs and mixed-format outputs instead of remaining advisory - it emits the background-normalized preview image used for reconstruction so the exact conditioning input is preserved alongside the mesh
As of March 26, 2026, the published hunyuan3d-dit-v2-1/config.yaml still
targets SingleImageEncoder plus ImageProcessorV2, not Tencent’s separate
multi-view preprocessor. Oshun therefore handles multi-image requests by
selecting one anchor image deterministically (first-image or largest-image)
and records that decision in result metadata instead of pretending the released
2.1 checkpoint performs native multi-view fusion.
Hunyuan3D 2.1 Batch Generation#
The text-to-shape and image-to-shape pipelines now also expose runBatch(...)
for throughput-oriented asset generation:
const textBatch = await textPipeline.runBatch({
targetBatchSize: 4,
requests: [
{ prompt: 'bronze raven', outputFormat: 'glb', seed: 11 },
{ prompt: 'marble lion', meshTopology: 'quad-dominant', seed: 12 },
],
});
const imageBatch = await imagePipeline.runBatch({
targetBatchSize: 4,
requests: [
{ images: [frontImage], outputFormat: 'glb' },
{ images: [sideImage], quality: 'high' },
],
});
This batching path is grounded in Tencent’s released runtime rather than a fake queue-only abstraction:
- the text batch runner feeds prompt lists through the Diffusers
HunyuanDiTPipelineand then passes the resulting image list intoHunyuan3DDiTFlowMatchingPipeline - the image batch runner feeds a list of preprocessed conditioning images
directly into
Hunyuan3DDiTFlowMatchingPipeline, which Tencent’s published implementation already supports - Oshun groups requests by the inference settings that truly affect tensor compatibility, keeps one Python process and one loaded runtime per compatible cohort, and only splits batches when settings like guidance, octree resolution, style LoRA, or memory mode differ
- standard-memory execution uses a conservative honest cap of
2assets per micro-batch on 24GB-class GPUs and4on 48GB-class GPUs; low-VRAM and mmgp execution intentionally falls back to warm-sequential processing because CPU offload and staged residency erase the benefit of resident micro-batches - per-item output formats, preview PNGs, and optional quad-dominant PyMeshLab post-processing are still preserved after the shared batched inference pass
- per-item
targetPolyCountrequests are preserved too, with automatic MeshAnything V2 post-processing applied only to the specific batch items that asked for face-budget control
The Hunyuan3D-Paint texture stack remains a single-mesh runtime upstream, so these batch APIs currently optimize geometry throughput first and leave texturing as an explicitly separate stage.
Hunyuan3D-Paint Loader#
The package now also exposes a dedicated loader for the published
Hunyuan3D-Paint texture-generation stack:
import {
Local3DModelRegistry,
createLocal3DHunyuan3DPaintModelLoader,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const loader = createLocal3DHunyuan3DPaintModelLoader({
registry,
runtimeRoot: '/opt/Hunyuan3D-2.1',
});
const plan = await loader.resolveLoadPlan({
maxSelectedViews: 6,
resolution: 2048,
allowMemoryDowngrade: true,
});
This loader is grounded in Tencent’s published paint runtime and separates the different moving parts explicitly:
- it models Tencent’s documented baseline of roughly 21GB VRAM for 6 views at a
2048-map output profile, while exposing explicit
512,1024,2048, and4096texture-map targets through a runtime resolution plan - it selectively downloads only the
hunyuan3d-paintpbr-v2-1checkpoint files plus the required localfacebook/dinov2-giantdependency throughLocal3DModelRegistry - it surfaces the additional runtime prerequisites Tencent documents but does
not host in the model repo, including the
RealESRGAN_x4plus.pthcheckpoint, the custom rasterizer build, and the mesh-painter compile script - because Tencent does not publish a dedicated low-VRAM paint switch, Oshun’s
low-vramandmmgpmodes are exposed as explicit staged-runtime plans with warnings instead of being misrepresented as native upstream toggles - runtime hints include the paint config path, custom diffusers pipeline path, DINO path, RealESRGAN path, renderer entrypoints, component residency strategy, the selected view/bake/downsample profile for the requested texture size, and the albedo/MR material outputs expected by the released stack
Hunyuan3D-Paint PBR Texture Synthesis#
The package now also exposes a full offline texture-synthesis pipeline for the
published Hunyuan3D-Paint runtime:
import {
Local3DModelRegistry,
createLocal3DHunyuan3DPaintTextureSynthesisPipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const pipeline = createLocal3DHunyuan3DPaintTextureSynthesisPipeline({
registry,
runtimeRoot: '/opt/Hunyuan3D-2.1',
allowMemoryDowngrade: true,
});
const result = await pipeline.run({
mesh: inputMeshBuffer,
meshFormat: 'glb',
prompt: 'weathered brass with dark patina in recessed areas',
outputFormat: 'glb',
textureResolution: 1024,
});
This pipeline follows Tencent’s released texture stack closely while fixing the gaps required for self-hosted production use:
- it keeps the released
hunyuan3d-paintpbr-v2-1diffusion model fully offline by loading the already-registered local checkpoint directory instead of calling upstreamsnapshot_download(...)at inference time - it supports both image-guided and text-guided texture synthesis; when only a text prompt is supplied, Oshun uses a neutral white style canvas as the required image-conditioning anchor and records that decision in warnings and metadata
- it stages multiview diffusion, RealESRGAN enhancement, and UV baking as
separate phases so the
low-vramloader profile can offload components between steps instead of keeping the whole Tencent stack resident at once - it exposes true texture-map size control for
512,1024,2048, and4096outputs; lower sizes use an oversampled bake plus deterministic downsampling, while4096switches into a native 4K bake path - it auto-downloads Tencent’s required
RealESRGAN_x4plus.pthcheckpoint intohy3dpaint/ckpt/if the runtime checkout is missing it - it emits the textured mesh plus standalone
albedo,metallic,roughness,metallicRoughness, and geometry-derivednormalmaps through the standard provider completion contract - GLB output uses Tencent’s own published PBR conversion utility so the embedded material matches the exported standalone texture maps
Hunyuan3D 2.1 Combined Shape+Texture Pipeline#
The package now also exposes an end-to-end orchestration pipeline that chains
the released Hunyuan 2.1 shape stage into the local Hunyuan3D-Paint texture
stage:
import {
Local3DModelRegistry,
createLocal3DHunyuan3D21ShapeAndTexturePipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const pipeline = createLocal3DHunyuan3D21ShapeAndTexturePipeline({
registry,
runtimeRoot: '/opt/Hunyuan3D-2.1',
allowMemoryDowngrade: true,
});
const result = await pipeline.run({
prompt: 'weathered bronze astrolabe',
outputFormat: 'glb',
styleAdapterPath:
'/models/hunyuan-style-loras/porcelain-style/adapter-manifest.json',
});
This combined pipeline is grounded in Tencent’s published end-to-end demo flow while staying explicit about the runtime model:
- it supports both text-driven and image-driven entrypoints by delegating to the already-isolated Oshun shape pipelines and then feeding the exact emitted preview image into the paint stage as the texture-conditioning anchor
- it reports both Tencent’s published co-resident combined budget
(
LOCAL_3D_HUNYUAN3D21_COMBINED_STANDARD_VRAM_MB, 29GB class) and Oshun’s actual sequential peak budget so schedulers can reason about the official full stack versus the implemented staged execution path mmgpmode resolves to the documented 6GB-class sequential path by combining the existing shape and paint MMGP profiles, whilestandardmode preserves the published full-stack envelope in metadata- final provider completion metadata includes both the shape-stage and texture-stage metadata snapshots so downstream systems can audit the entire generation chain instead of only the last stage
- text-mode combined runs pass
styleAdapterPathandstyleAdapterScalethrough to the HunyuanDiT reference-image stage before texturing, so a custom style LoRA still influences the geometry bootstrap even though the texture bake remains triangle-based - the textured pipeline stays triangle-based by design: quad-dominant requests are rejected here because the released Hunyuan3D-Paint UV baking path re-triangulates meshes during texturing, so Oshun only exposes preserved quad output on the standalone geometry-only shape pipelines
Hunyuan3D 2.1 MMGP Coordinator#
Local3DHunyuan3D21MmgpCoordinator is the consumer-GPU orchestration layer for
the Hunyuan 6GB path. It sits above the existing shape and paint pipelines and
uses the shared Local3DGpuMemoryManager to keep only one heavy stage resident
at a time while tracing the exact stage sequence that ran:
import {
createLocal3DGpuDeviceConfig,
createLocal3DHunyuan3D21MmgpCoordinator,
createLocal3DHunyuan3D21ShapeAndTexturePipeline,
Local3DModelRegistry,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
modelsRoot: '/models',
cachePolicy: { maxDiskUsageGb: 400, minFreeDiskGb: 50 },
});
const pipeline = createLocal3DHunyuan3D21ShapeAndTexturePipeline({
registry,
runtimeRoot: '/opt/hunyuan3d',
});
const coordinator = createLocal3DHunyuan3D21MmgpCoordinator({
gpuDevices: [
createLocal3DGpuDeviceConfig(0, {
name: 'RTX 3060',
vramBudgetMb: 6_144,
}),
],
});
const execution = await coordinator.runShapeAndTexture(pipeline, {
prompt: 'weathered bronze astrolabe',
memoryMode: 'mmgp',
});
console.log(execution.plan.stages.map((stage) => stage.stageId));
console.log(execution.trace.peakReservedMb);
The coordinator exposes planning helpers for standalone text-to-shape, image-to-shape, texture synthesis, and combined shape+texture runs. For 6GB cards it automatically caps the default GPU reserve so the published MMGP envelope can use the whole device instead of inheriting the server-wide 2GB reserve intended for multi-tenant deployments.
Hunyuan3D 2.1 Blender Addon Integration#
The package also ships a Blender-side integration surface for direct viewport
generation. Oshun’s implementation follows Tencent’s official addon direction of
embedding generation into the sidebar, but targets the existing local provider
API instead of a one-off /generate route and uses job polling plus auto-import
instead of a single blocking request.
import { createLocal3DHunyuan3D21BlenderAddonIntegration } from '@isis/3d-inference-local';
const integration = createLocal3DHunyuan3D21BlenderAddonIntegration({
apiBaseUrl: 'http://127.0.0.1:4413',
});
const descriptor = await integration.describeInstall();
await integration.copyAddonScript('/tmp/hunyuan3d-blender-addon');
console.log(descriptor.scriptPath);
console.log(descriptor.installSteps);
The bundled addon:
- installs as a single
hunyuan3d_2_1_blender_addon.pyfile - targets
/v1/3d/capabilities,/v1/3d/text-to-3d/jobs, and/v1/3d/image-to-3d/jobs - submits
preferredModelKey: 'hunyuan3d-2.1'so the local server can route the request into the Hunyuan stack - exposes prompt, negative prompt, image reference, quality, output format, texturing, texture resolution, and background-removal controls directly in the sidebar
- polls job state in the background, supports cancellation, downloads the final
model asset, and imports
glb,gltf,obj, orplyoutput back into a dedicated Blender collection - can auto-select and frame the imported result so the finished mesh appears in the viewport with minimal operator work
Hunyuan3D 2.1 Multiview Reference Generation#
The package now also exposes a bridge pipeline that turns a single reference image into canonical front/left/back/right view images before later reconstruction stages:
import {
Local3DModelRegistry,
createLocal3DHunyuan3D21MultiviewImageGenerationPipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const pipeline = createLocal3DHunyuan3D21MultiviewImageGenerationPipeline({
registry,
runtimeRoot: '/opt/Hunyuan3D-2.1',
allowMemoryDowngrade: true,
});
const result = await pipeline.run({
images: [inputImageBuffer],
quality: 'high',
viewOrder: ['front', 'left', 'back', 'right'],
});
This pipeline is intentionally grounded in Tencent’s published 2.1 sources rather than inventing a fake native novel-view checkpoint:
- as of March 26, 2026, Tencent’s released 2.1 stack still ships single-image
shape reconstruction plus the separate
Hunyuan3D-Paintmultiview diffusion module, not a standalone single-imagefront/left/back/rightgenerator - Oshun therefore bootstraps a coarse shape mesh with the released
Hunyuan3DDiTFlowMatchingPipeline, renders normal and position control maps for the requested camera views, and then runs the publishedhunyuan3d-paintpbr-v2-1diffusion bundle to synthesize canonical reference images - the pipeline downloads the exact paint checkpoint files plus the local
facebook/dinov2-giantdependency throughLocal3DModelRegistry, keepsHF_HUB_OFFLINE=1for the runner, returns the synthesized views ascompletion.textures[]withreference-view-*types, and preserves the background-normalized input preview plus the bootstrap mesh path for later debugging
Because this flow uses Tencent’s published paint runtime, the runtime root still
needs a built Hunyuan3D 2.1 checkout with the renderer dependencies available
under hy3dpaint/.
TRELLIS Text Pipeline#
The package now also ships a text-to-3D mesh pipeline for the official Microsoft TRELLIS text-conditioned checkpoints:
import {
Local3DModelRegistry,
createLocal3DTrellisTextTo3DPipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const pipeline = createLocal3DTrellisTextTo3DPipeline({
registry,
runtimeRoot: '/opt/TRELLIS',
});
const result = await pipeline.run({
prompt: 'weathered bronze astrolabe',
outputFormat: 'obj',
seed: 7,
});
This pipeline is built from the current official upstream Microsoft sources:
microsoft/TRELLIS-text-largefor the text-conditioned flow checkpointsmicrosoft/TRELLIS-image-largefor the shared decoders required by the text pipeline- the original
microsoft/TRELLISPython runtime, which currently contains the official text-conditioned pipeline implementation
This is an inference from the current upstream repos and model cards: the
released TRELLIS.2 repo is image-conditioned today, while Microsoft’s
text-conditioned weights remain on the original TRELLIS runtime.
At runtime the Node pipeline:
- downloads the official checkpoint bundles into
Local3DModelRegistry - rewrites the upstream
pipeline.jsondependency references into a fully local bundle so inference can run offline from the downloaded weights - executes the official Python pipeline via a package-local runner script
- emits progress stages for runtime load, text encoding, sparse-structure sampling, SLat sampling, mesh extraction, and final mesh write-out
- automatically hands the extracted mesh into MeshAnything V2 when callers ask
for
meshTopologyortargetPolyCountcontrols beyond the native TRELLIS extractor surface
For this task the pipeline intentionally focuses on raw mesh extraction, so the
supported output formats are obj and ply. Textured/PBR exports and GLB/GLTF
packaging are implemented in later TRELLIS tasks.
TRELLIS.2 Image Pipeline#
For single-image reconstruction, the package now exposes a TRELLIS.2 image
pipeline on top of the official trellis2 runtime:
import {
Local3DModelRegistry,
createLocal3DTrellis2ImageTo3DPipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const pipeline = createLocal3DTrellis2ImageTo3DPipeline({
registry,
runtimeRoot: '/opt/TRELLIS.2',
});
const result = await pipeline.run({
images: [imageBuffer],
outputFormat: 'obj',
removeBackground: true,
pipelineType: '1024_cascade',
});
At runtime the Node wrapper:
- downloads
microsoft/TRELLIS.2-4Bplus the shared sparse-structure decoder frommicrosoft/TRELLIS-image-large - rewrites the upstream
pipeline.jsondependency reference into a local bundle so inference runs from registry-managed weights instead of remote downloads - executes only the geometry branch of the official TRELLIS.2 image pipeline for
this task, producing raw mesh output in
objorply - emits progress stages for image preprocessing, image feature extraction, sparse-structure sampling, shape SLat sampling, mesh extraction, and final write-out
- optionally accepts
meshExtractionResolutionin 64-step increments between64and512so callers can trade extraction fidelity against decode cost
This task intentionally stops at raw mesh reconstruction. Multi-image conditioning, PBR extraction, Gaussian splatting output, and GLB/GLTF export are implemented in later TRELLIS subtasks.
TRELLIS.2 Multi-Image Pipeline#
For higher-fidelity reconstruction from multiple references, the package now also exposes a dedicated TRELLIS.2 multi-image pipeline:
import {
Local3DModelRegistry,
createLocal3DTrellis2MultiImageTo3DPipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const pipeline = createLocal3DTrellis2MultiImageTo3DPipeline({
registry,
runtimeRoot: '/opt/TRELLIS.2',
});
const result = await pipeline.run({
images: [frontView, leftView, rightView, backView],
outputFormat: 'obj',
pipelineType: '1024_cascade',
conditioningMode: 'concat',
});
At runtime the Node wrapper:
- reuses the same patched local
microsoft/TRELLIS.2-4Bbundle as the single-image pipeline, so all checkpoint downloads stay registry-managed - accepts 2-6 input views, preprocesses each one independently, and passes them into a dedicated multi-image runner
- fuses multiple image-conditioning feature sets into one TRELLIS conditioning
sequence, with
concatas the default strategy andmeanas a lower-memory alternative - selects up to 4 conditioning views when more than 4 inputs are supplied, matching the current upstream training regime while still accepting 5-6 references from callers
- supports the same optional
meshExtractionResolutionoverride, allowing lower-cost 64³-512³ extraction grids without changing the underlying TRELLIS pipeline type - automatically runs MeshAnything V2 after reconstruction when callers request
meshTopologyortargetPolyCount, so multi-image runs get the same honest topology-control surface as the single-image path - emits selected-view metadata so downstream systems know which input views were actually used for conditioning
This is based on the current upstream TRELLIS.2 training sources: the official dataset code samples up to 4 conditioning views, while community multi-image forks demonstrate multi-view inference on top of the released TRELLIS.2 image weights. Oshun therefore accepts 2-6 inputs, then selects and fuses the strongest subset for the official checkpoint bundle.
TRELLIS.2 PBR Material Extraction#
The package now also exposes a dedicated material extraction pipeline built on
top of the official texturing_pipeline.json runtime:
import {
Local3DModelRegistry,
createLocal3DTrellis2PbrMaterialExtractionPipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const pipeline = createLocal3DTrellis2PbrMaterialExtractionPipeline({
registry,
runtimeRoot: '/opt/TRELLIS.2',
});
const result = await pipeline.run({
mesh,
meshFormat: 'obj',
image: referenceImage,
texturingResolution: 1024,
textureSize: 2048,
});
This pipeline:
- loads the official TRELLIS.2 texturing runtime via
texturing_pipeline.json - encodes the input mesh back into shape SLat, samples texture SLat from the reference image, decodes the PBR voxel field, and rasterizes UV textures
- returns typed texture assets for
albedo,metallic,roughness,alpha, and a combinedmetallicRoughnessmap ready for later GLTF/GLB packaging - reuses the official UV-unwrapping and rasterization path from Microsoft’s texturing pipeline so the extracted maps stay aligned with upstream behavior
TRELLIS.2 Gaussian Splatting#
The package now also exposes a TRELLIS.2 Gaussian splatting export pipeline for real-time viewing stacks that prefer splats over triangle meshes:
import {
Local3DModelRegistry,
createLocal3DTrellis2GaussianSplattingPipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const pipeline = createLocal3DTrellis2GaussianSplattingPipeline({
registry,
runtimeRoot: '/opt/TRELLIS.2',
});
const result = await pipeline.run({
mesh,
meshFormat: 'obj',
image: referenceImage,
texturingResolution: 1024,
gaussianCount: 32768,
});
This pipeline:
- reuses the official TRELLIS.2 texturing runtime to infer appearance from the reference image instead of inventing a separate local material model
- samples the normalized TRELLIS.2 mesh surface, queries the decoded PBR voxel field at each sampled point, and fits anisotropic surface-aligned Gaussian splats
- writes a GraphDECO-compatible binary
.plyasset with SH DC color, opacity, scale, and quaternion rotation fields so existing Gaussian viewers can ingest it directly - returns the asset through the standard model-completion contract using the
logical output format
gaussian-splatting
This is an inference from the current upstream sources, not a direct Microsoft
feature: official TRELLIS.2 ships mesh and texturing decoders today, while the
older microsoft/TRELLIS runtime defines the canonical Gaussian PLY schema and
viewer/export path. Oshun combines those two upstream pieces so TRELLIS.2
outputs can participate in real-time Gaussian workflows immediately.
TRELLIS.2 GLB/GLTF Export#
The package now also exposes a dedicated GLB/GLTF export pipeline for turning a TRELLIS.2 mesh plus reference image into a textured delivery asset:
import {
Local3DModelRegistry,
createLocal3DTrellis2GltfExportPipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const pipeline = createLocal3DTrellis2GltfExportPipeline({
registry,
runtimeRoot: '/opt/TRELLIS.2',
});
const result = await pipeline.run({
mesh,
meshFormat: 'obj',
image: referenceImage,
outputFormat: 'glb',
texturingResolution: 1024,
textureSize: 2048,
});
This pipeline:
- reuses the official TRELLIS.2 texturing runtime end-to-end, including mesh preprocessing, texture latent sampling, PBR voxel decode, and textured mesh postprocessing
- exports textured GLB directly through the same
trimesh-based path shown in the officialexample_texturing.py - postprocesses GLTF export bundles into a single
.gltffile by inlining emitted image and buffer resources as data URIs, so callers do not need sidecar texture files - returns the final model asset through the standard completion contract with
model/gltf-binaryormodel/gltf+jsoncontent typing
This is partly an inference from upstream exporter behavior: the official
TRELLIS.2 example demonstrates single-file GLB export directly, while
single-file embedded GLTF is assembled locally by inlining the trimesh
exported resources into one JSON payload.
TRELLIS.2 Quality Presets#
resolveLocal3DTrellis2QualityPreset exposes the preset table used by the
single-image and multi-image TRELLIS.2 pipelines. Explicit request fields still
win, but when callers provide only quality, Oshun now maps that preset onto a
concrete TRELLIS.2 runtime configuration:
draft:512pipeline,128³mesh extraction,8sparse-structure steps,8shape-SLat stepsstandard:1024_cascadepipeline,256³extraction, official released12-step samplershigh: direct1024pipeline,384³extraction,14-step samplersultra:1536_cascadepipeline,512³extraction,18-step samplers, and a larger token budget
This mapping is grounded in the released TRELLIS.2 resolution tiers and model
card speed table: Microsoft documents 512³, 1024³, and 1536³ inference
targets with roughly ~3s, ~17s, and ~60s runtimes on H100 hardware.
TRELLIS.2 Prompt Adapter#
The released TRELLIS.2 checkpoints are image-conditioned, so prompt engineering in Oshun works by translating Isis prompt templates into guidance for the upstream image-generation step rather than pretending the runtime consumes free text directly:
import { adaptLocal3DTrellis2TextRequestPrompt } from '@isis/3d-inference-local';
const adaptation = adaptLocal3DTrellis2TextRequestPrompt({
prompt: 'weathered bronze astrolabe with engraved markings',
category: 'prop',
style: 'photorealistic, cinematic',
quality: 'high',
generateTextures: true,
pbrTextures: true,
});
The adapter emits:
- a
referenceImagePrompttuned for clean single-object renders - a sectioned
structuredPromptsplit into subject, style, camera, lighting, background, and detail guidance - extracted material/style/detail tags so callers can inspect or postprocess the adaptation before handing it to an upstream image model
- negative-prompt adaptation helpers that emit
negativeReferenceImagePromptguidance for excluding unwanted features before TRELLIS.2 reconstruction
When callers pass prompt into the TRELLIS.2 single-image or multi-image mesh
pipelines, Oshun now surfaces the derived promptAdaptation metadata in the
result payload instead of silently discarding the text.
When callers pass negativePrompt, Oshun now surfaces a
negativePromptAdaptation payload that translates exclusions such as
busy background, extra limbs, or text watermark into upstream
reference-image avoidance guidance. This is an inference from the current
upstream runtime: the released TRELLIS.2 image checkpoints still use zero-image
neg_cond, so the negative prompt needs to act before reconstruction rather
than inside the model.
Style guidance is now normalized through
resolveLocal3DTrellis2StyleConditioning. The built-in presets map Isis style
values such as photorealistic, stylized, low-poly, and cartoon onto
consistent reference-image tags, lighting cues, and detail guidance so the
upstream image generation step stays aligned with the intended 3D art direction.
TRELLIS.2 Seed Control#
TRELLIS.2 requests now normalize seed handling through one shared resolver:
- omitting
seeduses deterministic default seed1 - passing an explicit integer seed replays that exact value
- passing
seed: -1requests a randomized seed, but Oshun records the resolved seed in result metadata so the run can still be replayed later
The normalized seed and seedSource fields are now attached to the
single-image, multi-image, PBR extraction, GLTF export, and Gaussian splatting
completion metadata.
TRELLIS.2 Benchmark Suite#
The package now also ships a deterministic benchmark harness for tracking whether the self-hosted TRELLIS.2 stack is actually surpassing commercial baselines on the kinds of assets where Oshun cares about quality:
import { runDefaultLocal3DTrellis2BenchmarkSuite } from '@isis/3d-inference-local';
const report = runDefaultLocal3DTrellis2BenchmarkSuite({
generatedAt: '2026-03-26T00:00:00.000Z',
});
The benchmark report includes:
- weighted per-case scoring across geometry fidelity, topology cleanliness, texture fidelity, prompt alignment, multi-view consistency, thin-structure preservation, and material realism
- provider profiles for
trellis2-local,meshy-6,rodin-gen-1.5, andtripo-v2.5, including first-party reference URLs that document the current commercial API surface used for calibration - case-level winner detection plus pairwise TRELLIS-vs-baseline win rates and average score margins
- threshold gates so CI can fail when TRELLIS.2 regresses against any baseline
The default dataset covers five representative asset classes:
- hard-surface product detail
- thin-structure preservation
- stylized character prompt fidelity
- multi-view product consistency
- organic photoreal asset quality
Each case is intentionally benchmarked with modality-specific weights instead of one flat score. Multi-image cases emphasize view consistency, thin-structure cases emphasize open-space preservation, and stylized cases emphasize prompt and surface fidelity.
TRELLIS.2 Update Automation#
Local3DTrellis2ModelUpdateAutomation adds an operations-oriented update loop
on top of the existing loader and registry:
import {
Local3DModelRegistry,
createLocal3DTrellis2ModelUpdateAutomation,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const automation = createLocal3DTrellis2ModelUpdateAutomation({
registry,
});
const check = await automation.checkForUpdates();
const result = await automation.pullLatest();
This automation now:
- checks Microsoft GitHub in priority order: latest release, latest tag, then branch-head commit when the repo has no formal releases or tags
- uses conditional
If-None-Matchpolling on the active detection endpoint so repeated checks can reuse the cached upstream snapshot without wasting GitHub API quota - persists update state under the registry manifests directory, including the last seen upstream snapshot, last applied snapshot, last pulled runtime archive, and last synced Hugging Face weight revisions
- downloads the upstream GitHub source tarball into a local runtime-archive cache whenever a new upstream snapshot is applied
- keeps Hugging Face weight sync explicit: the automation still downloads the
current TRELLIS.2 and sparse-structure checkpoint revisions through
Local3DModelRegistry, while warning that Microsoft does not currently expose a direct GitHub-release-to-Hugging-Face revision mapping
As of March 26, 2026, the live Microsoft TRELLIS.2 repository does not expose a
GitHub releases/latest payload or published tags through the public GitHub
API, so this automation currently tracks main branch head commits in practice
and upgrades automatically if formal releases or tags appear later.
Progress Streaming#
Local3DGpuModelServer now exposes progress publication and SSE fan-out:
const server = createLocal3DGpuModelServer();
server.publishProgress({
jobId: 'job-123',
status: 'running',
progressPct: 42,
stage: 'mesh-generation',
modelKey: 'trellis-2',
gpuId: 0,
});
Clients can subscribe on the configured binding.ssePath and optionally filter
by jobId:
GET /v1/3d/events?jobId=job-123
Last-Event-ID: 5
For Meshy-style task streams, the server also accepts:
GET /v1/3d/events/job-123/stream
The stream emits event: message payloads shaped like Meshy's task progress
updates, and terminal states include the richer task payload fields such as
type, timestamps, result, and task_error.
REST API#
Local3DProviderRestApi plugs into Local3DGpuModelServer so the same local
runtime can expose job-oriented HTTP endpoints:
const providerApi = createLocal3DProviderRestApi({
capabilities: createLocal3DProviderCapabilities([
'text-to-3d',
'image-to-3d',
]),
onSubmit(submission, api) {
api.updateTask(submission.id, {
status: 'processing',
progress: 10,
stage: 'bootstrapping-runtime',
});
},
});
const server = createLocal3DGpuModelServer({
providerApi,
});
Supported routes:
GET /v1/3d/capabilities
GET /v1/3d/costs/dashboard
GET /v1/3d/quality/hunyuan-vs-meshy/dashboard
GET /v1/3d/jobs
GET /v1/3d/jobs/:jobId
DELETE /v1/3d/jobs/:jobId
GET /v1/3d/jobs/:jobId/stream
GET /v1/3d/jobs/:jobId/assets/:assetId
POST /v1/3d/text-to-3d/jobs
POST /v1/3d/image-to-3d/jobs
Text submissions reuse the shared Isis-compatible request shape. Image
submissions accept base64 strings or data: URLs, which the controller decodes
back into Buffer[] payloads for the runtime hooks. Completed tasks can expose
in-memory model, thumbnail, and texture assets as downloadable URLs while still
publishing the same lifecycle over SSE.
Container Deployment#
The package now includes a package-scoped GPU container build under docker/.
The image compiles only the local inference runtime surface with TypeScript and
runs it on top of an NVIDIA CUDA runtime image with Node 22:
docker build -f docker/Dockerfile -t oshun/isis-3d-inference-local:latest .
docker run --rm --gpus all -p 4413:4413 oshun/isis-3d-inference-local:latest
For compose-based GPU startup, use:
docker compose -f docker/docker-compose.gpu.yml up --build
Runtime behavior is driven by environment variables such as
LOCAL_3D_CAPABILITIES, LOCAL_3D_GPU_DEVICES_JSON, and
LOCAL_3D_RUNTIME_MODULE. The container bootstrap can dynamically load a
runtime module from disk, so later model-specific integrations can be mounted
into the same base image without replacing the HTTP, SSE, and health-check
layers.
Kubernetes Helm Chart#
The package now ships with a GPU-oriented Helm chart under helm/ for cluster
deployments:
helm lint helm/
helm upgrade --install isis-3d-local helm/ \
--namespace isis-system \
--create-namespace
The chart defaults to:
- scheduling only onto GPU nodes with
node-type=gpuandnvidia.com/gpu.present=true - requesting and limiting one
nvidia.com/gpudevice per pod - provisioning persistent volumes for model weights and Hugging Face / Torch caches
- mounting
/dev/shmas memory-backed storage for inference workloads - exposing the runtime over a
ClusterIPservice on port4413
For shared storage or externally supplied runtime modules, override values such
as persistence.models.existingClaim, persistence.cache.existingClaim, and
runtimeModules.configMapName/runtimeModules.existingClaim.
Windows Portable Deployment#
createLocal3DHunyuan3D21WindowsPortableDeployment(...) assembles a
self-contained Windows Hunyuan3D 2.1 bundle around:
- a prepared
@isis/3d-inference-localapp dist - a portable Node runtime
- a portable Python runtime with the Hunyuan dependencies already installed
- a prepared Hunyuan3D 2.1 checkout
- optional preseeded model and Hugging Face caches
const deployment = createLocal3DHunyuan3D21WindowsPortableDeployment({
identity: {
bundleId: 'hunyuan3d-2-1-win-portable',
label: 'Hunyuan3D 2.1 Portable',
version: '0.1.0',
},
sources: {
appDistDir: 'dist/libs/isis/3d-inference-local',
nodeRuntimeDir: 'vendor/node-win-x64',
pythonRuntimeDir: 'vendor/python-win-x64',
hunyuanRuntimeDir: 'vendor/Hunyuan3D-2.1',
modelCacheDir: 'vendor/model-cache',
},
});
await deployment.assembleBundle('dist/windows/hunyuan3d-2-1-portable');
The generated bundle includes:
scripts/RUN-API.cmdandscripts/RUN-API.ps1to start the Isis-compatible/v1/3dserver with a generated Hunyuan runtime modulescripts/RUN-GRADIO.cmdandscripts/RUN-GRADIO.ps1to launch the upstream Hunyuan Gradio app whengradio_app.pyis present in the bundled runtimescripts/DOCTOR.cmdandscripts/DOCTOR.ps1to verify the portable runtime layout before launchportable-manifest.jsonandPORTABLE-README.mddocumenting the assembled artifact, runtime settings, launchers, and upstream references
The bundle runtime is backed by createLocal3DHunyuan3D21RuntimeHooks(...),
which routes local REST submissions into the text-to-shape, image-to-shape, or
combined shape-plus-PBR pipelines depending on whether the request asks for
textures. The launchers set the Windows-specific environment for model caches,
the prepared Hunyuan runtime root, portable Python, and the generated
runtime-module path so the package can boot without Docker or a system-wide
Python/Node installation.
This implementation is informed by the official Tencent-Hunyuan/Hunyuan3D-2.1
Windows guidance published with the June 13, 2025 open-source release and by the
existing YanWenKun/Hunyuan3D-2-WinPortable community pack, both reviewed on
2026-03-26.
SF3D Loader#
Local3DSf3DModelLoader covers Stability AI’s official Stable Fast 3D runtime
layout while keeping the package honest about the consumer-GPU constraints and
gated model access:
import {
Local3DModelRegistry,
createLocal3DSf3DModelLoader,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const sf3d = createLocal3DSf3DModelLoader({ registry });
const plan = await sf3d.resolveLoadPlan({
precision: 'fp16',
allowPrecisionDowngrade: true,
gpuDevice: {
id: 0,
computeCapability: '8.6',
vramBudgetMb: 8_192,
},
});
The loader is grounded in Stability AI’s published Stable Fast 3D sources as
reviewed on 2026-03-26:
- the gated Hugging Face repository
stabilityai/stable-fast-3dexposesconfig.yamlandmodel.safetensors - the official runtime entrypoint is
SF3D.from_pretrained(...) - the released
run.pyexample uses CUDAtorch.autocast(..., dtype=torch.bfloat16) - the official requirements include native
texture_bakeranduv_unwrapperpackages, pluspynanoinstantmeshesfor remeshing - Stability AI documents Windows support as experimental and recommends Visual Studio 2022 for extension builds
The resolved load plan includes:
- gated-model install metadata and an explicit warning that Hugging Face access must be approved before automated downloads can succeed in a real environment
- two half-precision operating profiles:
fp16for the ~4GB minimum consumer profile andbf16for the ~6GB recommended profile on Ampere-or-newer GPUs - a documented downgrade path from
bf16tofp16when the GPU lacks native bf16 support or enough VRAM, usingallowPrecisionDowngrade - runtime hints for
SF3D.from_pretrained(...), CUDA autocast, CPU fallback viaSF3D_USE_CPU, supported remesh modes, and the upstream--texture-resolutionCLI flag - explicit dependency manifests for the official repo, pinned Python packages,
and the compiled
texture_baker/uv_unwrapperextensions needed by later UV and texture-output tasks
The fp16 plan is intentionally labeled as an inference from the official
sources rather than a direct upstream default: Stability AI’s published example
uses bf16 autocast on CUDA, but the Phase 71 requirement is a half-precision
consumer-GPU loader that can operate in the 4GB-to-6GB range when later
pipelines are layered on top.
SPAR3D Loader#
Local3DSpar3DModelLoader extends the Stability AI integration with the newer
point-aware runtime and makes the editable point-cloud intermediate explicit in
the local model contract:
import {
Local3DModelRegistry,
createLocal3DSpar3DModelLoader,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const spar3d = createLocal3DSpar3DModelLoader({ registry });
const plan = await spar3d.resolveLoadPlan({
allowMemoryDowngrade: true,
gpuDevice: {
id: 0,
computeCapability: '8.9',
vramBudgetMb: 8_192,
},
});
The loader is grounded in Stability AI’s published SPAR3D sources and model tree
as reviewed on 2026-03-26:
- the gated Hugging Face repository
stabilityai/stable-point-aware-3dexposesconfig.yamlandmodel.safetensors - the official runtime entrypoint is
SPAR3D.from_pretrained(...) - the released
run.pyexample uses CUDAtorch.autocast(..., dtype=torch.bfloat16),SPAR3D_LOW_VRAM, andreturn_points=True - the official
spar3d.systempath first generates a denoisedpc_condpoint-cloud condition and then feeds that editable point cloud into the final mesh stage - the official Gradio demo allows users to export, edit, re-import, subsample, and pad the point cloud before final meshing
The resolved load plan includes:
- a standard runtime profile at roughly
10.5GBVRAM and a published low-VRAM fallback at roughly7GB, with automatic downgrade viaallowMemoryDowngrade - explicit runtime hints for
SPAR3D.from_pretrained(...),SPAR3D_LOW_VRAM, CPU fallback viaSPAR3D_USE_CPU, and experimental Apple Silicon fallback viaPYTORCH_ENABLE_MPS_FALLBACK - a typed
pointCloudIntermediatecontract describing the releasedxyzrgbrepresentation: 512 points, 6 channels,pc_condas the batch key,pointcloudas therun_image(...)argument, andnormalize_pc_bboxas the normalization step before meshing - low-VRAM dynamic-module paging metadata that mirrors the released point-diffusion, main-mesh, and estimator groups unloaded and reloaded by the upstream runtime
- dependency manifests for the official repo, CLIP / AlphaCLIP, optional remesh
packages, optional Gradio point-cloud editor packages, and the compiled
texture_baker/uv_unwrapperextensions
The plan also preserves one upstream ambiguity instead of hiding it: Stability
AI’s current README mentions both ~10.5GB default VRAM and ~6GB single-image
manual inference. Oshun treats the explicit low-VRAM section as authoritative
for fallback planning, keeps the discrepancy in warnings, and leaves later
SPAR3D inference tasks to validate the true end-to-end footprint under the
selected output settings.
Sparc3D Loader#
Local3DSparc3DModelLoader adds an honest loader contract for Sparc3D even
though the public project still does not ship downloadable runtime weights:
import {
Local3DModelRegistry,
createLocal3DSparc3DModelLoader,
createLocal3DSparc3DOperatorBundleSource,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const sparc3d = createLocal3DSparc3DModelLoader({
registry,
weightsSource: createLocal3DSparc3DOperatorBundleSource({
rootDir: '/models/private/sparc3d',
}),
});
const plan = await sparc3d.resolveLoadPlan({
gpuDevice: {
id: 0,
computeCapability: '8.9',
vramBudgetMb: 24_576,
},
});
This loader is grounded in the live public Sparc3D materials reviewed on
2026-03-26:
- the project page still says code may be released only upon approval of Math Magic, so Oshun does not pretend an official inference runtime or weight tree is publicly downloadable yet
- the public Hugging Face space is demo-facing and does not expose a checkpoint
bundle that
Local3DModelRegistrycan download directly - the paper and project page still establish the product direction: sparse
high-resolution
1024^3geometry, Sparconv-VAE, and Sparcubes decoding
The resolved load plan therefore treats Sparc3D as a validated private bundle format instead of a fake public repo:
Local3DModelRegistrycan import the pretrained bundle from a local filesystem path or from a private mirrored Hugging Face repository- the bundle root must contain
bundle.manifest.json, and that manifest must declare theimage-encoder,sparconv-vae,latent-diffusion, andsparcubes-decodercheckpoint entries - the manifest must also advertise
1024inruntime.supportedVoxelResolutions, so later high-resolution image-to-3D stages are blocked early if the wrong checkpoint family is staged - runtime precision, VRAM planning, latency, batch size, supported surface
modes, and Python package requirements are all read from the bundle manifest;
unknown values stay at
0with an explicit warning instead of being guessed
This keeps Sparc3D usable inside the local model-registry flow today while remaining accurate about the current upstream publication gap.
Sparc3D Image-to-3D Pipeline#
Local3DSparc3DImageTo3DPipeline turns that private-bundle contract into an
actual runnable image-to-3d stage with explicit 256^3, 512^3, and 1024^3
resolution plans:
import {
Local3DModelRegistry,
createLocal3DSparc3DImageTo3DPipeline,
createLocal3DSparc3DOperatorBundleSource,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const sparc3d = createLocal3DSparc3DImageTo3DPipeline({
registry,
weightsSource: createLocal3DSparc3DOperatorBundleSource({
rootDir: '/models/private/sparc3d',
}),
});
const result = await sparc3d.run({
images: [imageBuffer],
outputFormat: 'glb',
quality: 'standard',
surfaceMode: 'watertight',
seed: 42,
});
This pipeline is grounded in the same public Sparc3D sources rechecked on
2026-03-27:
- the public GitHub README still exposes only the paper/project links and the
1024^3positioning, not a published inference runtime - the project page still says code may be released only upon approval of Math Magic, so the executable path remains operator-supplied rather than upstream downloadable
The runnable stage therefore uses a generic but explicit operator contract:
Local3DModelRegistryimports the full private bundle, including checkpoint files and the Python module named byruntime.packageRoot- the bundled runner loads
bundle.manifest.json, resolvesruntime.entrypoint, imports that module from the staged bundle, and invokes the callable with the image path, output path, requested voxel resolution, surface mode, seed, bundle manifest path, and resolved component paths - single-image conditioning is enforced up front; multi-image requests fail before runtime startup because the current Sparc3D task is specifically the single-image path
- quality presets now map onto actual voxel plans:
draft -> 256^3,standard -> 512^3, andhigh/ultra -> 1024^3 - explicit
voxelResolutionoverrides are supported when the operator bundle advertises that resolution inruntime.supportedVoxelResolutions
The result surface matches the local provider conventions used elsewhere in this
package: one model asset (glb, obj, or ply), an optional prepared-image
thumbnail, and structured metadata carrying the bundle revision, selected voxel
resolution, scaled VRAM estimate, selected surface mode, seed, output format,
runtime warnings, and any operator-reported mesh statistics.
Watertight Manifold Enforcement#
surfaceMode: 'watertight' now does more than forward a hint into the private
bundle runtime:
- Oshun independently parses the returned mesh topology for
objand ASCIIplyoutputs and computes boundary-edge, non-manifold-edge, degenerate-face, and connected-component counts before accepting the result as print-ready glbexports remain supported, but the operator bundle must also emit an OBJ or ASCII-PLY analysis sidecar throughanalysis_mesh_path, because the final GLB container is not treated as a trustworthy topology-analysis source inside this library- if the mesh is not closed and manifold, the request fails instead of quietly returning a supposedly watertight asset
- completion metadata now records
watertightMesh,repairActions, andprintableSolid, which makes the 3D-printing contract explicit to downstream orchestration
This is intentionally strict. Sparc3D's public paper and project page position
the method around high-fidelity 1024^3 watertight surfaces, so Oshun treats a
failed watertight verification as a hard correctness error rather than a soft
warning.
Open-Surface Support#
surfaceMode: 'open-surface' now has a distinct validation path aimed at thin
surfaces and disconnected parts:
- Oshun uses the same OBJ / ASCII-PLY topology analysis flow to inspect boundary edges, connected components, degenerate faces, and non-manifold edges for open-surface requests
- boundary edges and multiple connected components are treated as valid signals that thin sheets or floating sub-meshes survived reconstruction, instead of being misclassified as failures
glbexports must still include an analysis sidecar so the library can inspect whether open boundaries or loose parts were preserved- completion metadata now records
openSurfaceMesh, includingpreservesOpenBoundaries,hasDisconnectedComponents, andsupportsThinStructures
Open-surface mode still rejects clearly broken geometry such as non-manifold edges or degenerate faces. The difference from watertight mode is that boundary loops and disconnected components are allowed and explicitly surfaced instead of being treated as defects.
Geometric Fidelity Benchmark#
runDefaultLocal3DSparc3DGeometricFidelityBenchmark() adds a deterministic
geometry-only comparison surface for the local Sparc3D, TRELLIS.2, and Hunyuan3D
2.1 stacks:
import { runDefaultLocal3DSparc3DGeometricFidelityBenchmark } from '@isis/3d-inference-local';
const report = runDefaultLocal3DSparc3DGeometricFidelityBenchmark();
console.log(report.summary);
The benchmark intentionally avoids texture-centric scoring and instead focuses on the geometric claims that make Sparc3D interesting inside Oshun:
geometryFidelityandsurfaceDetailRetentionweight silhouette accuracy and dense relief retentionwatertightReliabilitymeasures printable-solid performance on closed-shape casesopenSurfacePreservationandthinStructurePreservationkeep the open-surface and delicate-geometry paths first-class instead of forcing every provider into a watertight-only rubric- pairwise report rows expose Sparc3D's win rate and worst-case margin against the local TRELLIS.2 and Hunyuan3D 2.1 baselines
Like the existing TRELLIS and Hunyuan dashboards, this is a deterministic evaluation dataset checked into the package, which keeps CI coverage reliable and makes geometry-regression thresholds explicit without requiring live model runs during tests.
MeshAnything V2 Cleanup#
The package now also exposes a dedicated MeshAnything V2 model loader so the official Hugging Face weights and runtime caveats can be planned explicitly before any cleanup or remeshing workflow runs:
import {
Local3DModelRegistry,
createLocal3DMeshAnythingV2ModelLoader,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d' });
const loader = createLocal3DMeshAnythingV2ModelLoader({ registry });
const plan = await loader.resolveLoadPlan();
console.log(plan.install.installDir);
console.log(plan.runtime.maxOutputFaces);
console.log(plan.warnings);
This loader is grounded in the public MeshAnything V2 release reviewed on
2026-03-27:
- the canonical official Hugging Face repository is
Yiwen-ntu/MeshAnythingV2and the published weight set currently includesconfig.json,model.safetensors, and350m.pth - Oshun models MeshAnything V2 as a
remeshingfamily member even though the Hugging Face card usesimage-to-3d, because the released runtime actually consumes dense meshes or point clouds throughmain.py - the published operational envelope is about
8GBVRAM and45sper mesh on an A6000, while Oshun recommends extra headroom for Marching Cubes preprocessing and flash-attn allocation spikes - the upstream model class still bootstraps config from
facebook/opt-350m, so fully offline deployments must pre-cache that config or patch the runtime to stay air-gapped - the Git checkout remains part of the runtime contract because the released
entrypoint imports
mesh_to_pc.pyplusMeshAnything/miche/shapevae-256.yamlfrom the repository tree - outputs remain within the current upstream
<=1600face artist-mesh regime and should be treated as topology cleanup / controlled retopology rather than unlimited mesh refinement
Sparc3D can now hand its topology-analysis OBJ into the released MeshAnything V2 CLI as an explicit post-process step:
import {
Local3DModelRegistry,
createLocal3DSparc3DImageTo3DPipeline,
createLocal3DSparc3DOperatorBundleSource,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d' });
const sparc3d = createLocal3DSparc3DImageTo3DPipeline({
registry,
weightsSource: createLocal3DSparc3DOperatorBundleSource({
rootDir: '/models/private/sparc3d',
}),
meshAnythingRuntimeRoot: '/models/runtime/MeshAnythingV2',
});
const result = await sparc3d.run({
images: [imageBuffer],
outputFormat: 'glb',
surfaceMode: 'watertight',
topologyCleanup: 'meshanything-v2',
quality: 'high',
});
This integration is deliberately grounded in the current upstream MeshAnything
V2 surface reviewed on 2026-03-27:
- the official repository exposes
main.pyfor mesh-inference rather than a stable Python library API, so Oshun drives that CLI through a dedicated runner - upstream recommends
--mcpreprocessing for non-Marching-Cubes meshes; Oshun enables it by default for watertight Sparc3D cleanup and disables it forsurfaceMode: 'open-surface'so intended boundary loops are not sealed away - the current upstream model targets an artist-mesh envelope capped by the
released
<=1600face regime, so cleanup is treated as topology improvement with an expected density reduction rather than a fidelity-preserving no-op
The returned metadata now includes topologyCleanup with before/after face and
vertex counts, boundary / manifold deltas, Marching Cubes settings, and cleanup
warnings. Oshun then reruns the same watertight or open-surface verification on
the cleaned mesh before returning it, so a post-process cannot silently break
the geometry contract established by the original Sparc3D request.
The same MeshAnything V2 runtime can now also be reused as an automatic post-process helper across the other generation families in this package:
- TRELLIS text, TRELLIS.2 single-image, and TRELLIS.2 multi-image now
automatically remesh when callers request
meshTopologyortargetPolyCount - Hunyuan3D 2.1 text/image single runs, both shape batch APIs, and the combined
shape+texture orchestrator now enforce
targetPolyCountthrough the same post-process helper - TripoSG image, scribble, and multi-view consistency runs automatically hand quad/topology-sensitive requests to MeshAnything V2 instead of leaving them as warning-only intent
- Sparc3D automatically activates its MeshAnything cleanup handoff when callers ask for face-budget or topology controls, while surfacing that the dedicated animation-ready edge-flow pass currently lives on the MeshAnything remeshing API rather than Sparc3D's own request layer
FlexiCubes Mesh Extraction#
meshExtractionMethod: 'flexicubes' now requests a Kaolin-backed FlexiCubes
remesh across the geometry-first generation families in this package:
import {
Local3DModelRegistry,
createLocal3DTripoSGImageTo3DPipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d' });
const pipeline = createLocal3DTripoSGImageTo3DPipeline({
registry,
runtimeRoot: '/opt/TripoSG',
});
const result = await pipeline.run({
images: [inputImageBuffer],
outputFormat: 'obj',
quality: 'high',
meshExtractionMethod: 'flexicubes',
});
This feature is intentionally honest about the current upstream boundaries:
- the official NVIDIA Kaolin surface reviewed on
2026-03-27iskaolin.non_commercial.flexicubes.FlexiCubes, so Oshun now drives the real Kaolin extractor instead of pretending native model outputs were already FlexiCubes meshes - most released 3D runtimes used here return finalized meshes rather than caller-controlled signed-distance fields, so Oshun reconstructs a temporary scalar field from nearest-surface point-and-normal projection before invoking FlexiCubes
- TRELLIS text/image/multi-image, Hunyuan3D 2.1 text/image/batch shape runs,
TripoSG image/scribble/multi-view, and Sparc3D all expose this as a real
post-extraction path with per-run
flexiCubesPostProcessmetadata - Sparc3D keeps an OBJ or ASCII-PLY analysis sidecar when FlexiCubes is used so open-surface / watertight validation and optional MeshAnything cleanup still inspect the remeshed geometry instead of stale native output
- SF3D and SPAR3D now reject
meshExtractionMethod: 'flexicubes'explicitly because those released Stability AI pipelines bake textured GLB assets from their native mesh path and Oshun cannot re-extract geometry there without discarding baked materials
FlexiCubes now also defaults to adaptive resolution on top of Kaolin's public
regular voxel-grid API. Oshun keeps the real construct_voxel_grid(...)
topology exposed by Kaolin, then applies a bounded axis warp derived from
face-dihedral and open-boundary complexity histograms before reconstructing the
scalar field. That gives flatter regions coarser cell spacing while
redistributing density toward thin, high-curvature, and boundary-heavy areas
without inventing an upstream octree API that Kaolin does not currently expose.
Grid selection is still preset-driven at the coarse level: draft maps to
128, mid-tier requests map to 192, and higher-detail plans map to 256
unless a family already exposes a stronger native resolution signal such as
Sparc3D voxel resolution. Low-level callers can still set
adaptivityMode: 'uniform' on the FlexiCubes postprocessor when they need a
strictly uniform reconstruction grid for controlled experiments or comparisons.
The low-level FlexiCubes postprocessor also now exposes an optional
differentiableRefinement stage for iterative mesh cleanup:
import { createLocal3DFlexiCubesAutomaticPostProcessor } from '@isis/3d-inference-local';
const processor = createLocal3DFlexiCubesAutomaticPostProcessor({
workDir: '/tmp/oshun-flexicubes',
});
const refined = await processor.run({
sourceLabel: 'Sparc3D image-to-3d generation',
mesh: inputMeshBuffer,
meshFormat: 'obj',
outputFormat: 'obj',
gridResolution: 256,
differentiableRefinement: {
iterations: 48,
cameraBatchSize: 4,
trainResolution: 256,
},
});
This path follows the official FlexiCubes optimization workflow rather than a
made-up post-smoothing pass: Oshun initializes the FlexiCubes SDF from the input
mesh, renders mask/depth supervision from that source mesh with nvdiffrast,
and then optimizes FlexiCubes SDF values, voxel deformations, and per-cube
weights before re-extracting the final surface. Because this is a real
differentiable optimization loop, it is intentionally opt-in on the low-level
API instead of silently running on every FlexiCubes request.
For regression tracking, the package now also exposes a deterministic
FlexiCubes-vs-Marching-Cubes benchmark harness through
runDefaultLocal3DFlexiCubesVsMarchingCubesBenchmark(...). The default dataset
captures five equal-budget extraction cases across hard-surface, thin-structure,
photogrammetry, organic, and optimization-oriented workloads, and scores both
extractors on geometry fidelity, sharp feature retention, triangle quality,
topology cleanliness, thin-structure preservation, and optimization readiness.
That keeps the quality claim tied to an explicit, versioned benchmark surface
instead of ad hoc eyeballing.
When CUDA is available, the FlexiCubes postprocessor now also uses Kaolin's
mesh-ops stack directly for the heavy mesh-prep stages instead of relying only
on trimesh and a CPU KD-tree. Surface sampling runs through
kaolin.ops.mesh.sample_points, nearest-face normals come from
kaolin.ops.mesh.face_normals, the scalar field is built with
kaolin.metrics.trianglemesh.point_to_mesh_distance, and watertight sources use
kaolin.ops.mesh.check_sign for sign classification. Open meshes still get the
accelerated Kaolin distance path, but switch sign inference to a
nearest-face-normal heuristic, and full CPU fallback is recorded explicitly in
the returned kaolinMeshOps report metadata and warning stream.
MeshAnything V2 Remeshing Pipeline#
The package now also exposes a dedicated remeshing pipeline for direct dense mesh cleanup:
import {
Local3DModelRegistry,
createLocal3DMeshAnythingV2RemeshingPipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d' });
const pipeline = createLocal3DMeshAnythingV2RemeshingPipeline({
registry,
runtimeRoot: '/models/runtime/MeshAnythingV2',
});
const result = await pipeline.run({
mesh: inputDenseMeshBuffer,
meshFormat: 'obj',
meshTopology: 'quad-dominant',
animationReadyTopology: true,
animationRigProfile: 'humanoid',
edgeFlowStrength: 'balanced',
});
This pipeline is grounded in the current upstream MeshAnything V2 runtime, but it closes the practical gaps needed for local production use:
- it resolves the MeshAnything V2 Hugging Face weights through the local model registry and passes the downloaded weight directory into the runtime instead of pretending a hard-coded remote model id is sufficient
- the Python runner now drives the upstream
mesh_to_pc.pyplusMeshAnythingV2.from_pretrained(...)path directly, which keeps the released geometry logic while avoiding the earlier remote-only CLI handoff - Oshun patches the upstream
facebook/opt-350mconfig bootstrap to use the localconfig.jsonwhen a local model directory is supplied, so the remesh path can run without a hidden extra network fetch meshTopology: 'quad-dominant'is implemented as a real two-stage cleanup: MeshAnything V2 first produces the artist-mesh triangle result, then Oshun runs a quad-dominant conversion pass and preserves the result as OBJanimationReadyTopology: trueadds an explicit edge-flow optimization stage between the artist mesh and the quad conversion pass; Oshun uses articulation band heuristics plus documented PyMeshLab remeshing/edge-flip filters so likely neck, shoulder, elbow, hip, knee, and ankle regions can be biased toward cleaner rigging loops forgeneric-biped,humanoid, andquadrupedassets- face-count control now snaps requests onto the supported
200,400,800, and1600artist-mesh budgets and applies that simplification before any optional quad conversion, so later topology stages work from the requested density envelope instead of a fixed upstream output - the pipeline exports both the normalized preprocessed input mesh and the MeshAnything artist-mesh OBJ sidecar so topology analysis and later quality scoring can compare before/after structure instead of only the final file
- returned metadata includes normalized input, artist-mesh, and final-output topology summaries, plus the Marching Cubes setting, seed, and warnings that affected the cleanup path
- animation-ready runs also return an
edgeFlowOptimizationreport with the optimized articulation bands, quad coverage, valence-4 ratio, edge-length consistency, and a heuristic closed-loop score for the final OBJ topology - every remesh run now also returns a
topologyQualityComparisonreport that scores the normalized pre-remesh mesh against the post-remesh topology on boundary-edge ratio, non-manifold/degenerate penalties, connected-component count, face-regularity, valence variance, and target-topology preference so Oshun can quantify whether the remesh materially improved topology quality
MeshAnything V2 LOD Chain Generation#
The package now also exposes a dedicated MeshAnything V2 LOD-chain generator for game-oriented asset packaging:
import {
Local3DModelRegistry,
createLocal3DMeshAnythingV2LodChainGenerator,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d' });
const generator = createLocal3DMeshAnythingV2LodChainGenerator({
registry,
runtimeRoot: '/models/runtime/MeshAnythingV2',
});
const result = await generator.run({
mesh: inputDenseMeshBuffer,
meshFormat: 'obj',
outputFormat: 'glb',
});
This chain generator is grounded in the same upstream MeshAnything V2 limits and
the published MSFT_lod ordering model:
- Oshun emits
LOD0as the highest-detail MeshAnything cleanup pass, then derivesLOD1throughLOD4from the supported1600,800,400, and200face budgets - the returned manifest includes descending screen-coverage thresholds aligned
with the high-to-low ordering used by the Khronos
MSFT_lodextension, so downstream engines can wire the exported files into their own LOD switching policy without reverse-engineering the chain order - each LOD preserves the same topology preferences as the source remesh request, including quad-dominant conversion and the optional animation-ready edge-flow pass, instead of silently dropping those constraints on lower-detail levels
- Oshun currently exports the chain as separate OBJ/PLY/GLB assets plus a JSON manifest rather than merging them into a single glTF container, which keeps the output honest and compatible with non-glTF game build pipelines
MeshAnything V2 Batch Remeshing#
The package now also exposes a dedicated MeshAnything V2 bulk-cleanup pipeline for asset-library remediation:
import {
Local3DModelRegistry,
createLocal3DMeshAnythingV2BatchRemeshingPipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({ rootDir: '/models/isis-3d' });
const pipeline = createLocal3DMeshAnythingV2BatchRemeshingPipeline({
registry,
runtimeRoot: '/models/runtime/MeshAnythingV2',
});
const result = await pipeline.runBatch(
[
{
mesh: firstDenseMeshBuffer,
meshFormat: 'obj',
outputFormat: 'glb',
meshTopology: 'triangle',
},
{
mesh: secondDenseMeshBuffer,
meshFormat: 'obj',
animationReadyTopology: true,
animationRigProfile: 'humanoid',
targetFaceCount: 800,
},
],
{ requestedBatchSize: 2 }
);
This batch pipeline is grounded in the released MeshAnything V2 runtime and the upstream folder-input execution model, but it fills in the production gaps for real library cleanup:
- the public
main.py --input_dir ...path is treated as the honest upstream batching surface, and Oshun converts that into manifest-driven CUDA micro-batches instead of pretending the original single-file CLI was already a full provider-ready batch API - compatible jobs share one MeshAnything V2 model load and one batched forward pass while still preserving per-item output formats, face-count simplification, normalized-input sidecars, and artist-mesh analysis OBJs
- compatibility grouping is explicit: Marching Cubes enablement/resolution, the sampling flag, and stochastic sampling seeds must match before assets are placed into the same micro-batch
- Oshun keeps the VRAM envelope honest by defaulting to micro-batch size
2when no GPU budget is supplied, allowing2on typical24GBcards, and only exposing4on48GB+plans;256-resolution Marching Cubes and stochastic sampling stay capped at2 - post-remesh topology work still happens per asset, so quad-dominant conversion, animation-ready edge-flow optimization, topology-quality scoring, and provider-compatible completion payloads survive the batch path instead of being silently dropped for throughput
- returned batch metadata now reports compatibility-group count, micro-batch count, effective execution mode, and measured throughput in items per minute so operators can reason about whether the cleanup job actually delivered library-scale gains
SPAR3D Image-to-3D Pipeline#
Local3DSpar3DImageTo3DPipeline turns the released SPAR3D point-aware runtime
into a provider-compatible single-image workflow and keeps the intermediate
point cloud visible instead of hiding it behind the final mesh:
import {
Local3DModelRegistry,
createLocal3DSpar3DImageTo3DPipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const pipeline = createLocal3DSpar3DImageTo3DPipeline({
registry,
runtimeRoot: '/opt/stable-point-aware-3d',
});
const result = await pipeline.run({
images: [inputImageBuffer],
quality: 'standard',
guidanceScale: 3,
seed: 1234,
});
This pipeline is grounded in the same official SPAR3D sources reviewed on
2026-03-26, but it intentionally follows the lower-level Gradio flow instead
of the one-line CLI helper so the editable point cloud remains available:
- image preprocessing still uses the released
transparent-backgroundforeground-removal path plusforeground_crop(...) - point diffusion is run explicitly through
forward_pdiff_cond(...)andsampler.sample_batch_progressive(...)soguidanceScaleandseedare honored for the intermediatepc_cond - the intermediate 512-point
xyzrgbcloud is exported aspoints.plyalongside the final texturedmesh.glb - final meshing still uses the released
generate_mesh(...)path with optionalnone/triangle/quadremeshing and texture baking
The returned provider completion includes:
- a primary textured
glbasset - a secondary
plymodel asset containing the intermediate point cloud - a thumbnail of the processed RGBA input used for reconstruction
- metadata for point-cloud generation time, mesh generation time, point count, selected memory mode, guidance scale, seed, and the emitted runtime keys
standard quality maps to the released 1024 texture-resolution default and
stays closest to the published fast path. high and ultra remain available,
but the pipeline warns when those settings move the run away from the fastest
advertised SPAR3D profile.
SPAR3D Point-Cloud Editing#
Local3DSpar3DPointCloudEditingPipeline exposes the same editable intermediate
that Stability AI’s published Gradio demo uses and turns it into a provider-safe
local workflow:
import {
Local3DModelRegistry,
createLocal3DSpar3DPointCloudEditingPipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const pipeline = createLocal3DSpar3DPointCloudEditingPipeline({
registry,
runtimeRoot: '/opt/stable-point-aware-3d',
});
const result = await pipeline.run({
images: [inputImageBuffer],
quality: 'standard',
editOperations: [
{ kind: 'translate', pointIndices: [0, 1, 2], delta: [0.05, 0.0, -0.03] },
{ kind: 'recolor', pointIndices: [0, 1, 2], color: [255, 180, 32] },
],
});
This pipeline is grounded in the same official SPAR3D sources reviewed on
2026-03-26, including the published Gradio demo and runtime internals:
- when
pointCloudDatais omitted, Oshun follows the released image-to-point diffusion path to generate the editablepc_cond - when
pointCloudDatais supplied, the pipeline accepts an uploadedplypoint cloud, normalizes it to SPAR3D’s 512-point conditioning size through padding or subsampling, and bypasses point-diffusion-only controls such asguidanceScaleandseed - edit operations are applied sequentially to the
xyzrgbcloud beforegenerate_mesh(...)reconstructs the final textured mesh - the supported edit primitives match the concrete editing affordances the official demo advertises directly: point translation, point scaling, and point recoloring
- the edited point cloud is exported as
plyalongside the final texturedglb, so callers can round-trip into external tools such as MeshLab or Blender for more advanced edits and then re-run meshing locally
The returned provider completion includes:
- a primary textured
glbasset generated from the edited point cloud - a secondary
plyasset containing the final edited conditioning cloud - a thumbnail of the processed RGBA input image used for conditioning
- metadata for point-cloud source, edit operation count, applied operation kinds, edit time, mesh time, point count, and the selected memory mode
standard quality stays aligned with the released 1024-texture fast path.
Higher quality tiers remain available for better texture fidelity, but the
pipeline preserves the same warnings when those settings move the run away from
the advertised low-latency SPAR3D profile.
SPAR3D Preview Loop#
Local3DSpar3DPreviewLoop turns SPAR3D’s published “edit point cloud, then
re-run meshing” demo pattern into a real warmed local session instead of a
cold-start wrapper:
import {
Local3DModelRegistry,
createLocal3DSpar3DPreviewLoop,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const loop = createLocal3DSpar3DPreviewLoop({
registry,
runtimeRoot: '/opt/stable-point-aware-3d',
});
const { session } = await loop.start({
images: [inputImageBuffer],
quality: 'high',
meshTopology: 'quad-dominant',
targetVertexCount: 3200,
});
const fastPreview = await session.preview({
editOperations: [
{ kind: 'translate', pointIndices: [0, 1, 2], delta: [0.04, 0, -0.02] },
],
});
const finalMesh = await session.finalize();
await session.close();
This preview loop is grounded in the same official SPAR3D sources reviewed on
2026-03-26, especially the published Gradio demo’s “Re-run with point cloud”
flow and the released generate_mesh(...) runtime internals:
- the Python worker loads
SPAR3D.from_pretrained(...)once, preprocesses the conditioning image once, and keeps the current 512-pointpc_condresident between renders session.preview(...)defaults to the real fast-path settings for interactive feedback:512texture resolution andremesh: 'none'session.finalize(...)uses the higher-quality defaults captured at session start, so callers can iterate quickly and then produce the final mesh without reloading the modelsession.reset()restores the original generated or uploaded point cloud so a preview loop can recover from bad edits without re-running image conditioning- uploaded replacement point clouds still go through the same 512-point normalization that the official demo applies before meshing
Each preview/final render returns:
- a textured
glbmesh for the current edited point cloud - a
plyexport of the current conditioning cloud after any new edits - the prepared conditioning thumbnail
- per-render metadata for edit counts, render mode, mesh timings, point-cloud source, and texture/remesh settings
The preview loop is intentionally conservative about what counts as
“interactive”: raising texture resolution above 512 or enabling remeshing
during session.preview(...) is allowed, but surfaced as a warning because it
moves the session away from the released fast edit path.
SF3D Image-to-3D Pipeline#
Local3DSf3DImageTo3DPipeline wraps the official Stable Fast 3D single-image
runtime into the Oshun provider surface and keeps the released preprocessing and
export path intact:
import {
Local3DModelRegistry,
createLocal3DSf3DImageTo3DPipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const pipeline = createLocal3DSf3DImageTo3DPipeline({
registry,
runtimeRoot: '/opt/stable-fast-3d',
defaultPrecision: 'fp16',
allowPrecisionDowngrade: true,
});
const result = await pipeline.run({
images: [inputImageBuffer],
quality: 'standard',
meshTopology: 'quad-dominant',
targetVertexCount: 12000,
});
This implementation is based on Stability AI’s released code paths reviewed on
2026-03-26:
run.pyremoves the background, resizes the foreground crop, runsSF3D.from_pretrained(...).run_image(...), and exportsmesh.glbgradio_app.pyuses the sameforeground_ratiopreprocessing and drives the directgenerate_mesh(...)path with a textured GLB previewsf3d/system.pyhandles UV unwrap, texture baking, and PBR material assembly during mesh generation rather than as a post-export step
The pipeline surface intentionally reflects those upstream behaviors:
- only a single image is accepted for this task because the released SF3D runtime is single-image conditioned
- output is restricted to textured
glbso the provider completion preserves the official textured-mesh artifact instead of discarding materials qualitymaps onto texture-atlas size:draft=512,standard=1024,high=2048,ultra=4096meshTopologymaps onto the official remesh options:triangle -> triangle,quad-dominant -> quad, unset ->none- the completion payload includes both the textured GLB and the processed input preview used for reconstruction
At runtime the Node wrapper:
- resolves the previously introduced SF3D load plan so gated-model warnings, precision downgrades, and VRAM estimates stay aligned with the loader
- writes the incoming image buffer to disk, launches the bundled Python runner, and streams JSON progress for runtime load, preprocessing, mesh generation, texture bake, and final export
- preserves the upstream
remove_backgroundandresize_foregroundflow, surfaces peak memory and generation duration metadata from the runner, and returns a provider completion asset withmodel/gltf-binarycontent
SF3D Batch Image-to-3D Pipeline#
Local3DSf3DBatchImageTo3DPipeline exposes the real SF3D batched inference path
from Stability AI’s published CLI instead of simulating batching at the queue
layer:
import {
Local3DModelRegistry,
createLocal3DSf3DBatchImageTo3DPipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const pipeline = createLocal3DSf3DBatchImageTo3DPipeline({
registry,
runtimeRoot: '/opt/stable-fast-3d',
gpuDevice: {
id: 0,
computeCapability: '8.9',
vramBudgetMb: 24_576,
},
});
const result = await pipeline.runBatch(
[{ images: [chairImage] }, { images: [lampImage] }, { images: [shoeImage] }],
{ requestedBatchSize: 8 }
);
This batch pipeline is grounded in the same official SF3D sources reviewed on
2026-03-26:
- Stability AI’s published
run.pyalready accepts--batch_sizeand passes a list of prepared images intomodel.run_image([...]) - the bundled batch runner preserves the official preprocessing sequence for
every item: RGBA load, optional
rembg, andresize_foreground(...) - compatible requests are grouped by precision, background-removal settings, foreground ratio, texture resolution, remesh mode, and target vertex count so each micro-batch maps cleanly onto one native SF3D invocation
- the micro-batch size is capped by a conservative VRAM heuristic derived from
the already-documented 4GB
fp16and 6GBbf16runtime envelopes - remeshing requests intentionally fall back to warm-sequential execution because the per-mesh CPU remesh stage is not the released high-throughput path
The batch result includes:
- one provider-compatible
Local3DSf3DImageTo3DResultper input request - batch metadata for compatibility groups, micro-batch count, effective batch size, total duration, and measured items-per-minute throughput
- a
targetThroughputAchievedflag checked against the Phase 71 goal of100images per minute
draft and standard quality with remesh: 'none' are the intended throughput
settings. Higher texture resolutions or remeshing remain available, but the
pipeline warns when those controls move a run away from the fastest official
SF3D path.
Quality-vs-Speed Routing#
Local3DQualityVsSpeedRouter promotes the first automatic model-selection layer
inside the local 3D stack. It is intentionally narrow and honest: SF3D remains
the default speed path for single-image preview-style requests, while TRELLIS.2
takes over when the request is clearly quality-driven or needs a delivery format
that the current SF3D surface does not expose.
import {
Local3DModelRegistry,
createLocal3DQualityVsSpeedRouter,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const router = createLocal3DQualityVsSpeedRouter({
registry,
sf3dRuntimeRoot: '/opt/stable-fast-3d',
trellisRuntimeRoot: '/opt/TRELLIS.2',
});
const speedResult = await router.runImage({
images: [thumbnailImage],
quality: 'draft',
});
const qualityResult = await router.runImage({
images: [heroReference],
quality: 'ultra',
outputFormat: 'glb',
});
The router is grounded in the same upstream implementations reviewed on
2026-03-26:
- Stability AI’s released SF3D stack is the fast single-image textured GLB path
with published sub-second latency and native batched
run_image([...]) - Microsoft’s released TRELLIS.2 runtime is the higher-fidelity path with
documented 512³ / 1024³ / 1536³ quality tiers and a first-party GLB export
flow via
o_voxel.postprocess.to_glb(...) - Oshun’s TRELLIS route composes those existing local TRELLIS geometry and GLB export stages so a quality-routed request does not regress from “delivery asset” back to “raw mesh intermediate”
Current routing rules:
- single-image
draftand latency-sensitivestandardrequests default to SF3D high/ultraquality, prompt-guided image requests, and non-GLB delivery requests route to TRELLIS.2- multi-image requests stay on TRELLIS.2 because SF3D remains single-image
- explicit quad-dominant topology requests can stay on SF3D when that topology control matters more than TRELLIS fidelity heuristics
The router exposes selectImageRoute(...), runImage(...), runBatch(...),
and runText(...). Mixed batches are supported: speed-route items are collapsed
into the SF3D batch pipeline when available, while quality-route outliers run
through TRELLIS sequentially with per-item routing metadata preserved.
Intelligent Model Routing#
Local3DIntelligentModelRouter is the broader model-selection layer for
Phase 71. It keeps the existing quality-vs-speed router intact, but expands the
decision space across the local 3D families already implemented in this package:
- organic and character-leaning prompts favor Hunyuan3D 2.1
- hard-surface, architectural, and multi-image requests favor TRELLIS.2
- speed-critical single-image preview jobs favor SF3D
- high-resolution, printable, or thin-structure geometry jobs favor Sparc3D
- sketch and doodle-flavored image prompts favor TripoSG
import {
Local3DModelRegistry,
createLocal3DIntelligentModelRouter,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const router = createLocal3DIntelligentModelRouter({
registry,
sf3dRuntimeRoot: '/opt/stable-fast-3d',
trellisRuntimeRoot: '/opt/TRELLIS.2',
hunyuanRuntimeRoot: '/opt/hunyuan3d',
triposgRuntimeRoot: '/opt/TripoSG',
});
const decision = router.selectImageRoute({
images: [referenceImage],
prompt: 'high-resolution printable ornate statue with fine filigree',
quality: 'ultra',
outputFormat: 'obj',
});
const result = await router.runImage({
images: [referenceImage],
prompt: 'hard-surface mech cockpit with industrial panels',
outputFormat: 'glb',
});
The routing heuristics were aligned against the released upstream capability
surfaces reviewed on 2026-03-27 and then mapped onto Oshun’s existing local
pipelines:
- Hunyuan3D 2.1 is the strongest local path for organic, production-style asset generation and direct GLB delivery
- TRELLIS.2 remains the broadest geometry-plus-texturing path for hard-surface and multi-image reconstruction
- SF3D is still the fastest released single-image textured preview route
- Sparc3D is the geometry-first route when watertight, printable, or high-resolution reconstruction matters most
- TripoSG is the best fit for sketch-like inputs and stylized reference images
Every routed result preserves a structured routing block in provider
completion metadata with the selected family, selected model key, execution
mode, candidate scores, prompt-analysis evidence, and human-readable reasons.
TRELLIS image routes also preserve the existing geometry-plus-GLB/GLTF export
composition so a routed request can still come back as a delivery-ready asset
instead of an intermediate mesh only.
Quality-Aware Model Routing#
Local3DQualityAwareModelRouter sits on top of the intelligent router and
executes multiple compatible candidates before selecting a winner. The current
implementation is benchmark-backed rather than hand-wavy:
- candidate families are shortlisted from the existing intelligent-router prompt analysis and compatibility scores
- compatible candidates are executed in parallel or in bounded parallel cohorts
depending on
maxConcurrentCandidates - each successful candidate is scored from a weighted mix of benchmark prior, prompt-fit prior, artifact-quality signals, delivery fitness, latency efficiency, and warning penalties
- benchmark priors come from the existing TRELLIS.2 benchmark suite, the Hunyuan3D 2.1 quality dashboard, and the Sparc3D geometric-fidelity benchmark; SF3D and TripoSG use explicit local heuristics because Phase 71 does not yet include equivalent comparative dashboards for them
- failed candidates are preserved in the quality-routing report and excluded from ranking instead of aborting the whole request
import {
Local3DModelRegistry,
createLocal3DQualityAwareModelRouter,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const router = createLocal3DQualityAwareModelRouter({
registry,
trellisRuntimeRoot: '/opt/TRELLIS.2',
hunyuanRuntimeRoot: '/opt/hunyuan3d',
triposgRuntimeRoot: '/opt/TripoSG',
maxConcurrentCandidates: 2,
});
const result = await router.runImage({
images: [referenceImage],
prompt: 'printable cathedral ornament with fine filigree detail',
outputFormat: 'obj',
quality: 'ultra',
});
console.log(result.decision.selectedFamily);
console.log(
result.candidates.map((candidate) => ({
family: candidate.family,
rank: candidate.rank,
score: candidate.score?.totalScore,
}))
);
Cost-Aware Model Routing#
Local3DCostAwareModelRouter keeps self-hosted generation as the default path
and only falls back to configured cloud providers once the selected local route
is genuinely blocked by GPU saturation or hard self-hosted capacity limits.
- the router reuses the existing intelligent local-routing decision instead of inventing a second local selection stack
- scheduler snapshots are inspected to determine whether the selected local model can run immediately, whether every eligible GPU is saturated, and how much queue delay the request is likely to inherit
- queue-delay budgets are priority-aware by default (
interactive,batch,background) and can be overridden per request - cloud providers are filtered by request compatibility before selection; exact output-format support, texture-resolution limits, multi-image support, PBR support, variation limits, and advanced mesh-control safety are all checked
- when cloud fallback is needed, the router ranks compatible providers by known cloud price profile first and latency second
- advanced local-only controls such as
meshExtractionMethod,meshTopology, andquadDominantOptimizationkeep the request on the self-hosted path even under saturation rather than silently degrading them
The cloud-provider interface is intentionally structural, so existing provider
instances that expose generateFromText(...), generateFromImage(...), and
getCapabilities() can be passed in directly. That keeps
@isis/3d-inference-local buildable without taking a hard compile-time
dependency on the broader hosted-provider package surface.
import { createLocal3DCostAwareModelRouter } from '@isis/3d-inference-local';
const router = createLocal3DCostAwareModelRouter({
intelligentRouter,
scheduler,
queue,
costDashboard,
cloudProviders: [
{ id: 'meshy', provider: meshyProvider },
{ id: 'rodin', provider: rodinProvider },
],
});
const result = await router.runImage({
images: [referenceImage],
prompt: 'high-detail production prop',
quality: 'high',
outputFormat: 'glb',
priority: 'interactive',
});
console.log(result.decision.selectedTarget);
console.log(result.decision.loadAssessment.estimatedQueueDelayMs);
console.log(result.completion.metadata?.costRouting);
Cloud-fallback executions are normalized back into the same provider-completion
shape as local executions, persisted into the configured workDir, and tagged
with a costRouting metadata block that preserves the selected target, selected
local model key, selected cloud provider, queue/load evidence, and the estimated
self-hosted versus cloud cost comparison.
The result surface keeps the winning routed output at the top level, preserves
the full winning intelligent-route result under winner, and attaches a
structured qualityRouting metadata block to provider completion metadata with
candidate ranks, score margins, benchmark sources, and the scoring weights used
for the final decision.
Generation A/B Testing#
Local3DGenerationABTestingFramework provides a deterministic experiment layer
for Phase 71 model-routing work. It is purpose-built for comparing the three
surfaces Oshun actually changes in this package:
- model-version upgrades
- prompt-strategy rewrites
- post-processing pipeline swaps
The framework keeps the statistical surface explicit instead of hiding it behind "experiment" marketing language:
- subjects are assigned to variants deterministically with sticky hashing
- every experiment has an explicit control variant plus a declared primary metric
success-rateuses a two-sample z-test for proportionsquality-score,latency-ms,cost-usd, andgpu-utilization-pctuse a Welch-style two-sample t-test- the snapshot reports confidence intervals, p-values, effect sizes, sample-size sufficiency, and a recommendation that stays conservative until the minimum sample target is reached
import {
createLocal3DGenerationABTestingFramework,
createLocal3DModelVersionExperimentDefinition,
} from '@isis/3d-inference-local';
const experiments = createLocal3DGenerationABTestingFramework();
experiments.createExperiment(
createLocal3DModelVersionExperimentDefinition({
id: 'hunyuan-2-1-patch',
name: 'Hunyuan 2.1 patch validation',
primaryMetric: {
id: 'quality-score',
},
secondaryMetrics: [
{
id: 'latency-ms',
},
{
id: 'cost-usd',
},
],
minimumSampleSizePerVariant: 24,
variants: [
{
id: 'control',
label: 'Hunyuan 2.1.0',
modelKey: 'hunyuan3d-2.1',
modelVersion: '2.1.0',
isControl: true,
},
{
id: 'candidate',
label: 'Hunyuan 2.1.1',
modelKey: 'hunyuan3d-2.1',
modelVersion: '2.1.1',
},
],
})
);
experiments.startExperiment('hunyuan-2-1-patch');
const assignment = experiments.assignVariant('hunyuan-2-1-patch', 'subject-42');
experiments.recordOutcome('hunyuan-2-1-patch', {
subjectId: 'subject-42',
variantId: assignment.variantId,
success: true,
qualityScore: 0.87,
latencyMs: 18400,
costUsd: 0.21,
});
const snapshot = experiments.getSnapshot('hunyuan-2-1-patch');
console.log(snapshot.summary.winningVariantId);
console.log(snapshot.primaryMetricComparisons[0]?.pValue);
Generation Analytics Dashboard#
Local3DGenerationAnalyticsDashboard is the operational dashboard surface for
the same routing stack. It tracks the signals that matter when model routing is
making production decisions instead of benchmark-only ones:
- per-model success and failure rates
- quality scores extracted from routed result metadata
- average, p50, and p95 latency
- GPU utilization from live sample capture or embedded task telemetry
- self-hosted cost plus cloud-price comparison deltas
- optional experiment summaries when the A/B framework is attached
import {
createLocal3DGenerationABTestingFramework,
createLocal3DGenerationAnalyticsDashboard,
createLocal3DProviderRestApi,
} from '@isis/3d-inference-local';
const experiments = createLocal3DGenerationABTestingFramework();
const analytics = createLocal3DGenerationAnalyticsDashboard({
experimentFramework: experiments,
});
analytics.recordGpuUtilizationSample({
gpuId: 0,
utilizationPct: 72,
memoryUtilizationPct: 58,
activeModelKeys: ['hunyuan3d-2.1'],
});
const providerApi = createLocal3DProviderRestApi({
capabilities,
generationAnalyticsDashboard: analytics,
});
const snapshot = analytics.getSnapshot({
cloudProfileIds: ['meshy-pro'],
});
console.log(snapshot.summary.successRate);
console.log(snapshot.modelBreakdown[0]?.averageLatencyMs);
console.log(snapshot.gpuBreakdown[0]?.latestUtilizationPct);
console.log(
analytics.getGpuUtilizationSamples({
gpuId: 0,
since: new Date(Date.now() - 15 * 60_000),
}).length
);
When the dashboard is attached to Local3DProviderRestApi, it is also exposed
as a REST snapshot:
GET /v1/3d/analytics/generations/dashboard- filters:
kind,status,modelKey,gpuId,limit,selfHostedProfileId,cloudProfileIds, andexperimentStatus
GPU Utilization Monitor#
Local3DGpuUtilizationMonitor consumes those sampled GPU traces plus the
multi-model scheduler snapshot and turns them into actionable cluster-hygiene
alerts:
- underutilized GPUs are flagged when a recent utilization window stays below a configurable threshold
- idle resident model instances are flagged when they keep holding VRAM without active assignments for too long
- active and resolved alerts keep timestamps and stable identifiers so external automation can forward only state changes
import {
Local3DGpuUtilizationMonitor,
createLocal3DGenerationAnalyticsDashboard,
createLocal3DMultiModelScheduler,
} from '@isis/3d-inference-local';
const analytics = createLocal3DGenerationAnalyticsDashboard();
const scheduler = createLocal3DMultiModelScheduler({
gpuDevices,
models,
runtime,
});
const monitor = new Local3DGpuUtilizationMonitor({
analytics,
scheduler,
gpuObservationWindowMs: 15 * 60_000,
idleModelThresholdMs: 10 * 60_000,
onAlertEvent: ({ transition, alert }) => {
console.log(transition, alert.kind, alert.message);
},
});
const cycle = await monitor.reconcile();
console.log(cycle.activeAlerts.map((alert) => alert.id));
console.log(cycle.idleModelObservations[0]?.idleDurationMs);
Model Sharding Across Multiple GPUs#
Local3DModelShardingManager adds tensor-parallel planning and reservation
tracking for models that do not fit on a single GPU:
- it estimates full-model VRAM with the same scheduling heuristic already used by the single-GPU scheduler
- it chooses the smallest viable shard count that fits the cluster, while preferring homogeneous GPU sets over mixed hardware groups
- it produces an explicit tensor-parallel plan with split dimension, collective ops, per-shard reservations, and load/unload hooks for the runtime layer
import {
Local3DModelShardingManager,
type Local3DModelShardingRuntime,
} from '@isis/3d-inference-local';
const runtime: Local3DModelShardingRuntime = {
async loadShardedModel(model, context) {
console.log(model.key, context.plan.shardCount, context.plan.gpuIds);
},
async unloadShardedModel(model, context) {
console.log('unload', model.key, context.reason);
},
};
const manager = new Local3DModelShardingManager({
gpuDevices: [
{
id: 0,
name: 'RTX 4090',
vramBudgetMb: 24_576,
reserveMb: 0,
modelAffinity: [],
},
{
id: 1,
name: 'RTX 4090',
vramBudgetMb: 24_576,
reserveMb: 0,
modelAffinity: [],
},
{
id: 2,
name: 'RTX 4090',
vramBudgetMb: 24_576,
reserveMb: 0,
modelAffinity: [],
},
],
models: [largeModelDescriptor],
runtime,
});
const decision = await manager.schedule({
modelKey: largeModelDescriptor.key,
request: {
prompt: 'polished brass astrolabe',
quality: 'high',
},
});
console.log(decision.plan.parallelismMode);
console.log(decision.plan.shardCount);
console.log(
decision.shards.map((shard) => [shard.gpuId, shard.estimatedVramMb])
);
Model Quantization Planning#
Local3DModelQuantizationPlanner adds shared INT8 and INT4 planning across the
self-hosted model families without forcing each loader to reinvent quantization
rules:
- it understands every built-in 3D family shipped by the package
- it models calibration requirements, preserved modules, artifact outputs, and backend dependencies
- it produces quantized runtime environment wiring plus VRAM and throughput estimates on top of an existing self-hosted load plan
import {
Local3DModelQuantizationPlanner,
Local3DModelRegistry,
Local3DTrellis2ModelLoader,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const loader = new Local3DTrellis2ModelLoader({ registry });
const basePlan = await loader.resolveLoadPlan({
precision: 'bf16',
lowVram: false,
});
const quantizer = new Local3DModelQuantizationPlanner();
const int8Plan = quantizer.plan(basePlan, {
precision: 'int8',
calibrationSampleCount: 256,
});
console.log(int8Plan.estimatedRuntimeVramMb);
console.log(int8Plan.quantization.artifacts[0]?.absolutePath);
console.log(int8Plan.environment.LOCAL_3D_QUANTIZATION_BACKEND);
TensorRT Optimization Planning#
Local3DModelTensorRTPlanner adds a shared NVIDIA TensorRT planning surface for
the self-hosted 3D runtimes instead of scattering ad hoc compile settings across
individual model loaders:
- it models family-specific dynamic-shape profiles for image, texturing, and remeshing paths
- it plans persistent Torch-TensorRT program, engine-cache, timing-cache, and calibration-cache artifacts per GPU architecture
- it exposes compile settings, environment wiring, VRAM reduction, and expected speedup on top of an existing load plan
import {
Local3DModelRegistry,
Local3DModelTensorRTPlanner,
Local3DTrellis2ModelLoader,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const loader = new Local3DTrellis2ModelLoader({ registry });
const basePlan = await loader.resolveLoadPlan({
precision: 'bf16',
});
const tensorrt = new Local3DModelTensorRTPlanner();
const optimizedPlan = tensorrt.plan(basePlan, {
precision: 'fp16',
capability: 'image-to-3d',
targetGpu: {
id: 0,
name: 'NVIDIA L40S',
computeCapability: '8.9',
vramBudgetMb: 49_152,
},
});
console.log(optimizedPlan.tensorrt.estimatedSpeedup);
console.log(optimizedPlan.tensorrt.compileSettings.engineCacheDir);
console.log(optimizedPlan.environment.LOCAL_3D_TENSORRT_PROGRAM_PATH);
ONNX Runtime Planning#
Local3DModelOnnxRuntimePlanner adds a cross-platform ONNX Runtime planning
surface for the self-hosted 3D model families:
- it maps each supported capability onto platform-valid execution providers for CUDA, DirectML, CoreML, and CPU fallback
- it plans optimized ORT artifacts plus serialized session/provider profiles for deployment reuse
- it exposes provider-aware precision fallback, graph optimization defaults, runtime dependencies, and environment wiring on top of an existing load plan
import {
Local3DModelOnnxRuntimePlanner,
Local3DModelRegistry,
Local3DSf3DModelLoader,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const loader = new Local3DSf3DModelLoader({ registry });
const basePlan = await loader.resolveLoadPlan({
precision: 'fp16',
});
const onnx = new Local3DModelOnnxRuntimePlanner();
const optimizedPlan = onnx.plan(basePlan, {
capability: 'image-to-3d',
targetOs: 'windows',
targetArch: 'x64',
});
console.log(optimizedPlan.onnxruntime.provider);
console.log(optimizedPlan.onnxruntime.sessionOptions.graphOptimizationLevel);
console.log(optimizedPlan.environment.LOCAL_3D_ONNX_OPTIMIZED_MODEL_PATH);
Compile Optimization Planning#
Local3DModelCompileOptimizationPlanner centralizes torch.compile planning
for the PyTorch-native generation families and exposes the specialized
stable-fast path for SF3D:
- it limits compile planning to generation-oriented model families instead of pretending remeshing-only runtimes should be compiled the same way
- it reuses existing runner wiring like Hunyuan3D 2.1’s
--compileswitch where the repo already exposes one - it models TorchInductor cache directories, stable-fast runtime dependencies, warmup costs, and expected steady-state latency gains on top of a load plan
import {
Local3DModelCompileOptimizationPlanner,
Local3DModelRegistry,
Local3DSf3DModelLoader,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const loader = new Local3DSf3DModelLoader({ registry });
const basePlan = await loader.resolveLoadPlan({
precision: 'fp16',
});
const compiler = new Local3DModelCompileOptimizationPlanner();
const optimizedPlan = compiler.plan(basePlan, {
backend: 'stable-fast',
capability: 'image-to-3d',
});
console.log(optimizedPlan.compilation.estimatedSpeedup);
console.log(optimizedPlan.environment.LOCAL_3D_COMPILE_BACKEND);
console.log(optimizedPlan.runtimeDependencies.map((dep) => dep.dependencyId));
MeshAnything V2 Speculative Decoding#
Local3DMeshAnythingV2SpeculativeDecodingPlanner adds MeshAnything-specific
draft/verify planning for the autoregressive retopology path:
- it pairs the published MeshAnything V2 verifier with the same
facebook/opt-350mbootstrap model family as a lightweight draft decoder - it estimates token budgets from the supported 200/400/800/1600-face buckets and emits draft-window, verification-batch, and acceptance-threshold settings
- it exposes reusable artifact locations plus deterministic prefix-reconciliation helpers for verifier rollback handling
import {
Local3DMeshAnythingV2SpeculativeDecodingPlanner,
Local3DModelRegistry,
Local3DMeshAnythingV2ModelLoader,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const loader = new Local3DMeshAnythingV2ModelLoader({ registry });
const basePlan = await loader.resolveLoadPlan();
const planner = new Local3DMeshAnythingV2SpeculativeDecodingPlanner();
const speculative = planner.plan(basePlan, {
targetFaceCount: 1600,
});
console.log(speculative.speculativeDecoding.estimatedSpeedup);
console.log(speculative.environment.LOCAL_3D_SPECULATIVE_DRAFT_MODEL);
console.log(speculative.speculativeDecoding.artifacts[0]?.absolutePath);
Inference Result Caching#
Local3DInferenceResultCache caches completed local 3D inference outputs so
repeated requests can short-circuit expensive generation work:
- it hashes normalized request inputs together with model key, model version, optional family, and cache salt so prompt-identical runs only hit when the runtime identity also matches
- it supports both text-to-3D and image-to-3D requests, hashing image bytes rather than Buffer identity for deterministic cache keys
- it stores full completion payloads with TTL, byte-budget, and LRU eviction semantics so repeated results can be returned without rerunning inference
import {
Local3DInferenceResultCache,
buildLocal3DInferenceCacheKey,
} from '@isis/3d-inference-local';
const cache = new Local3DInferenceResultCache({
defaultTtlMs: 30 * 60 * 1000,
});
const cacheKey = buildLocal3DInferenceCacheKey({
modelKey: 'trellis-2',
modelVersion: '2.0.1',
request: {
prompt: 'ornate shield',
quality: 'high',
},
});
const hit = cache.get({
modelKey: 'trellis-2',
modelVersion: '2.0.1',
request: {
prompt: 'ornate shield',
quality: 'high',
},
});
console.log(cacheKey);
console.log(hit.hit);
Progressive Generation With Early Preview#
Local3DProgressiveGenerationCoordinator adds a shared two-stage orchestration
surface for returning a draft preview quickly and then refining to the final
asset:
- it applies a conservative preview profile by default, using
draftquality,glboutput, a400-face target,512texture resolution, and textures disabled so the preview path stays on the fast side of a 2-second target - it preserves the original request for the refinement pass, emits typed progress updates for preview preparation, preview readiness, refinement, and final completion, and records whether the preview met the requested deadline
- it merges preview affordances such as
previewUrland thumbnails into the final completion payload so clients can keep showing the early artifact while higher-quality geometry and textures arrive
import { Local3DProgressiveGenerationCoordinator } from '@isis/3d-inference-local';
const coordinator = new Local3DProgressiveGenerationCoordinator({
previewRunner: async ({ request }) => ({
previewUrl: `https://cdn.local/previews/${request.quality}.glb`,
}),
finalRunner: async () => ({
models: [{ data: Buffer.from('final-glb'), format: 'glb' }],
}),
});
const result = await coordinator.run(
{
jobId: 'job-42',
request: {
prompt: 'ornate bronze helmet',
quality: 'high',
textureResolution: 2048,
generateTextures: true,
},
},
{
onProgress(update) {
console.log(update.stage, update.progressPct);
},
}
);
console.log(result.firstPreviewLatencyMs);
console.log(result.mergedCompletion.previewUrl);
console.log(result.final.completion.models?.[0]?.format);
SF3D UV Unwrap Output#
Local3DSf3DUvUnwrapPipeline exposes the released SF3D UV unwrap as an explicit
artifact instead of leaving it implicit inside the textured GLB:
import {
Local3DModelRegistry,
createLocal3DSf3DUvUnwrapPipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const pipeline = createLocal3DSf3DUvUnwrapPipeline({
registry,
runtimeRoot: '/opt/stable-fast-3d',
});
const result = await pipeline.run({
images: [inputImageBuffer],
meshTopology: 'quad-dominant',
textureResolution: 2048,
});
This task is grounded in the official Stability AI UV path reviewed on
2026-03-26:
sf3d/models/mesh.pyimports the nativeuv_unwrapperpackage and callsMesh.unwrap_uv(...)- the unwrap step duplicates seam vertices and stores per-vertex UVs on
mesh.v_tex sf3d/system.pythen bakes textures against those UVs before constructing the texturedtrimeshexport
The Oshun UV export keeps that official unwrap intact and adds explicit output artifacts on top:
- the pipeline still returns the textured
glbproduced by the released SF3D runtime so downstream tools keep the same mesh/material representation - it also renders a
uv-layout.pngartifact directly from the official mesh UVs and exposes it as a providertextureasset withtextureType=uv-layout - metadata includes UV vertex count, UV face count, UV bounds, and a UV area coverage estimate so later tasks can validate unwrap quality and atlas use
This is intentionally different from a generic downstream UV pass. The UV layout
is derived from the exact Stability AI mesh that already passed through
uv_unwrapper, which is the main differentiator called out in Phase 71 versus
weaker image-to-3D baselines that skip a production-grade unwrap surface.
SF3D Illumination-Disentangled Texture Output#
Local3DSf3DIlluminationTexturePipeline promotes the released SF3D material
channels into explicit provider assets so the lighting-neutral texture is
available without scraping the final GLB:
import {
Local3DModelRegistry,
createLocal3DSf3DIlluminationTexturePipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const pipeline = createLocal3DSf3DIlluminationTexturePipeline({
registry,
runtimeRoot: '/opt/stable-fast-3d',
});
const result = await pipeline.run({
images: [inputImageBuffer],
textureResolution: 2048,
});
This task is grounded in the official Stability AI material assembly reviewed on
2026-03-26:
sf3d/system.pyassignsdecoded["features"]tomat_out["albedo"]- the same code path computes
roughness,metallic, and tangent-space normal data before constructingtrimesh.visual.material.PBRMaterial - the final GLB therefore already contains an illumination-neutral base color texture rather than a simple lit screenshot projection
The Oshun export surfaces those channels directly:
albedois saved as a standalone texture asset and marked in metadata asalbedoLightingDisentangled=truenormalis exported when present from the released material stack- scalar
roughnessFactorandmetallicFactorare preserved in metadata for downstream material authoring or validation - the textured
glbremains part of the completion payload so the mesh and texture package stay consistent with the official SF3D runtime
This gives downstream tooling a proper lighting-neutral texture handoff, which is the Phase 71 differentiator here: not just “a textured mesh”, but the actual released albedo channel that SF3D decodes before material assembly.
Cost Dashboard#
Local3DCostDashboard can ingest tracked jobs from Local3DProviderRestApi and
expose an API-first unit-economics snapshot:
const costDashboard = createLocal3DCostDashboard();
const providerApi = createLocal3DProviderRestApi({
capabilities: createLocal3DProviderCapabilities([
'text-to-3d',
'image-to-3d',
]),
costDashboard,
});
Once jobs are flowing, query the dashboard at:
GET /v1/3d/costs/dashboard?selfHostedProfileId=runpod-l4-flex&cloudProfileIds=meshy-pro,meshy-studio
The default dashboard ships with:
- self-hosted deployment profiles based on current RunPod serverless GPU pricing
such as
runpod-l4-flexandrunpod-a40-flex - cloud API comparison profiles for Meshy Pro and Meshy Studio using current stage-credit pricing
- per-task and aggregate comparisons showing self-hosted total cost, average cost per asset, estimated external API spend, and savings deltas
You can override both the self-hosted profiles and cloud pricing catalog when constructing the dashboard, so teams using on-prem GPUs, different storage retention windows, or other commercial APIs can plug in their own economics model without changing the REST contract.
Hunyuan Quality Dashboard#
Local3DHunyuan3D21QualityComparisonDashboard provides a deterministic,
API-first benchmark snapshot for comparing the local Hunyuan3D 2.1
shape-plus-PBR stack against the current Meshy-6 PBR baseline surface:
const hunyuanQualityDashboard =
createLocal3DHunyuan3D21QualityComparisonDashboard();
const providerApi = createLocal3DProviderRestApi({
capabilities: createLocal3DProviderCapabilities([
'text-to-3d',
'image-to-3d',
]),
hunyuanQualityDashboard,
});
Query it at:
GET /v1/3d/quality/hunyuan-vs-meshy/dashboard?category=hard-surface&modality=image-to-3d-pbr&limit=3
The default snapshot ships with:
- six curated text-to-3d + PBR and image-to-3d + PBR benchmark cases
- weighted scoring across geometry fidelity, topology cleanliness, texture fidelity, material realism, prompt alignment, PBR completeness, lighting neutrality, and asset readiness
- threshold gates that summarize whether the local Hunyuan stack still clears the target lead over Meshy-6 in average score, win rate, and PBR/material deltas
- provider source references captured on
2026-03-26from the official Tencent Hunyuan3D repository and current Meshy retexture API documentation
The dashboard is intentionally deterministic rather than a live online eval. It is designed for release gating, regression tracking, and API-level reporting, while still allowing teams to swap in their own benchmark dataset and scoring thresholds.
RunPod Serverless Proxy Mode#
For bursty workloads that should scale to zero between requests, the same local
HTTP surface can now proxy jobs to a RunPod Serverless endpoint while keeping
the Isis-compatible /v1/3d REST API and SSE progress stream online.
Set the following container environment variables:
LOCAL_3D_RUNPOD_ENDPOINT_ID=your-endpoint-id
LOCAL_3D_RUNPOD_API_KEY=your-runpod-api-key
LOCAL_3D_RUNPOD_SYNC_MODE=interactive
LOCAL_3D_RUNPOD_SYNC_TIMEOUT_SECONDS=90
LOCAL_3D_RUNPOD_EXECUTION_TIMEOUT=600
LOCAL_3D_RUNPOD_TTL=3600
When RunPod proxy mode is enabled:
- interactive jobs can use
/runsyncwhile batch/background jobs use/runplus/statuspolling - local task IDs remain stable even though the adapter tracks a separate remote RunPod job ID internally
- completed RunPod outputs can return encoded model/texture assets that the adapter hydrates back into local downloadable result assets
- health checks treat zero-worker idle states as healthy, which matches RunPod's scale-to-zero behavior instead of reporting false failures
The adapter also exports request/result serializers
(createLocal3DRunpodJobInput, parseLocal3DRunpodJobInput, and completion
encode/decode helpers) so a RunPod worker can speak the same envelope format as
the local gateway.
TripoSG Loader#
The package now also exposes Local3DTripoSGModelLoader for Tripo’s published
1.5B rectified-flow image-to-3D runtime:
import {
Local3DModelRegistry,
createLocal3DTripoSGModelLoader,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const triposg = createLocal3DTripoSGModelLoader({ registry });
const plan = await triposg.resolveLoadPlan({
gpuDevice: {
id: 0,
computeCapability: '8.9',
vramBudgetMb: 12_288,
},
});
This loader is grounded in Tripo’s published repo and Hugging Face artifacts:
VAST-AI/TripoSGpublishes the diffusersTripoSGPipelinebundle with the Dinov2 encoder, rectified-flow transformer, scheduler, and VAE checkpoints- the official
scripts.inference_triposgentrypoint also downloadsbriaai/RMBG-1.4and runsBriaRMBG.from_pretrained(...)beforeprepare_image(...) - the upstream README publishes an 8GB CUDA minimum; Oshun keeps that as the hard floor and models ~12GB as an operational headroom target for the full TripoSG+RMBG runtime
The resolved load plan includes:
- local paths for
model_index.json, transformer, VAE, Dinov2 encoder, scheduler, and RMBG checkpoints - runtime defaults for
num_inference_steps=50,guidance_scale=7.0,num_tokens=2048, and the published octree / flash-decoder settings - mixed-source dependency metadata covering the MIT TripoSG weights plus the separate BRIA RMBG 1.4 background-removal license
- an explicit note that the underlying pipeline accepts list-image batches even though the released CLI only exposes a single-image entrypoint today
TripoSG Image-to-3D Pipeline#
The package now also exposes a runnable local TripoSG image-to-3D pipeline on top of the loader:
import {
Local3DModelRegistry,
createLocal3DTripoSGImageTo3DPipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const triposgPipeline = createLocal3DTripoSGImageTo3DPipeline({
registry,
runtimeRoot: '/opt/TripoSG',
});
const result = await triposgPipeline.run({
images: [inputImageBuffer],
quality: 'standard',
outputFormat: 'glb',
guidanceScale: 7,
seed: 42,
});
This pipeline follows the released TripoSG flow closely:
- it loads
BriaRMBGfrom the separately downloadedbriaai/RMBG-1.4dependency and feeds the image through the publishedprepare_image(...)foreground-isolation path - it runs the official
TripoSGPipeline.from_pretrained(...)stack infp16on CUDA with the published 50-step / 7.0-guidance defaults - it can optionally simplify the generated triangle mesh natively, and now
automatically hands quad-dominant or
targetPolyCountrequests to MeshAnything V2 so downstream topology control is implemented instead of being warning-only intent
The resolved result includes:
- geometry-only mesh assets in
glb,obj, orplyformat - the prepared foreground image used for inference so callers can inspect the actual crop fed into TripoSG
- per-run metadata for preprocessing, inference, and optional simplification timings plus mesh vertex / face counts
- explicit warnings when callers supply prompt, negative-prompt, or texture controls the released runtime cannot honor directly, plus MeshAnything post-process metadata when topology cleanup was applied automatically
TripoSG-Scribble Image-to-3D Pipeline#
The package also exposes a TripoSG-scribble pipeline for rough sketches and rapid concept iteration:
import {
Local3DModelRegistry,
createLocal3DTripoSGScribbleImageTo3DPipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const scribblePipeline = createLocal3DTripoSGScribbleImageTo3DPipeline({
registry,
runtimeRoot: '/opt/TripoSG',
});
const result = await scribblePipeline.run({
images: [scribbleBuffer],
prompt: 'a cat with wings',
quality: 'standard',
scribbleConfidence: 0.4,
promptConfidence: 1.0,
});
This variant follows the published TripoSG-scribble release closely:
- it downloads
VAST-AI/TripoSG-scribble, which adds a tokenizer and CLIP text encoder on top of the TripoSG base image-conditioning stack - it keeps
guidance_scale=0because the upstream model is explicitly CFG-distilled and instead exposes prompt/scribble weighting through cross-attention scaling - it preserves the official
use_flash_decoder=Falseand 8/8 octree settings from the released inference script so the local runtime does not reintroduce the boundary artifacts noted upstream - it automatically routes quad-dominant or
targetPolyCountrequests through MeshAnything V2 after the released scribble-conditioned mesh path finishes
The pipeline result includes:
- geometry-only
glb,obj, orplyoutputs for rapid concept review - the prepared RGB sketch image that was actually fed into the model
- per-run metadata for prompt text, confidence settings, denoising steps, timings, and mesh face / vertex counts
- validation and warnings for unsupported negative-prompt and texturing controls, MeshAnything post-process metadata for topology cleanup requests, and guidance when scribble confidence moves outside the upstream-recommended 0.3-0.5 range
TripoSG Multi-View Consistency Mode#
The package also exposes a TripoSG multi-view consistency pipeline for 2-8 reference views of the same object:
import {
Local3DModelRegistry,
createLocal3DTripoSGMultiViewConsistencyPipeline,
} from '@isis/3d-inference-local';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const pipeline = createLocal3DTripoSGMultiViewConsistencyPipeline({
registry,
runtimeRoot: '/opt/TripoSG',
});
const result = await pipeline.run({
images: [frontBuffer, leftBuffer, backBuffer, rightBuffer],
quality: 'standard',
consistencySampleCount: 2048,
});
This mode is intentionally honest about what upstream TripoSG does and does not ship:
- the released
TripoSGPipelineaccepts list-image batches, but it still reconstructs one mesh per image instead of doing true fused multi-view conditioning - Oshun therefore uses shared-seed per-view generation and then computes a pairwise surface-distance agreement score across the candidate meshes
- the final output is the geometric medoid, which is the candidate mesh with the lowest mean pairwise Chamfer-style distance to the other reconstructed views
- once that medoid is selected, Oshun can now automatically apply MeshAnything V2 when the request asks for quad-dominant topology or a face-budget target
The result includes:
- the selected final
glb,obj, orplymesh - all prepared reference-view images used for conditioning
- per-view consistency metrics, selected-view index, and cross-view agreement scores in metadata
- automatic micro-batching when more than four views are supplied so Oshun stays inside the published TripoSG family’s practical VRAM envelope
TripoSG vs TripoSR Comparison#
The package also exposes a comparison harness for validating the local TripoSG surface against the repo’s older TripoSR-style provider baseline:
import {
Local3DModelRegistry,
createLocal3DTripoSGComparisonPipeline,
} from '@isis/3d-inference-local';
import { createTripoProvider } from '@isis/3d-generation/providers';
const registry = new Local3DModelRegistry({
rootDir: '/models/isis-3d',
});
const report = await createLocal3DTripoSGComparisonPipeline({
registry,
runtimeRoot: '/opt/TripoSG',
baselineProvider: createTripoProvider({ apiKey: process.env.TRIPO_API_KEY }),
}).run({
images: [referenceImageBuffer],
});
The report is intentionally explicit about evidence quality:
- it runs the local TripoSG image-to-3D path and the injected Tripo baseline in parallel on the same request
- it marks whether topology improvement was validated, mixed, baseline- competitive, or inconclusive
- it assigns a confidence level based on whether baseline topology metrics came from provider metadata or a custom measured extractor
- it separates texture parity from the topology verdict so the current geometry-only TripoSG surface is not misrepresented as a texture-complete replacement