Historical task source. The Eve task board holds current task state.
Phase 71: 3D Generation SOTA — Surpass Meshy.ai & Commercial Platforms#
This phase closes every identified gap between the Oshun ecosystem and commercial 3D generation platforms (Meshy.ai, Tripo, Rodin, 3DFY.AI). It establishes Oshun as the industry-leading 3D generation ecosystem by self-hosting open-source SOTA models, building AI-native texture pipelines, universal auto-rigging, browser-based generation, video-to-mesh reconstruction, and a complete 3D asset marketplace. The result: Meshy-quality (or better) generation at zero per-asset cost, with capabilities no commercial platform offers.
Dependency: Phase 5 (Isis core), Phase 8 (Bellona DCC bridges), Phase 19 (GPU infrastructure). Many tasks can proceed independently.
Key open-source models referenced: TRELLIS.2 (Microsoft, MIT), Hunyuan3D 2.1 (Tencent, Mixed Source), SF3D/SPAR3D (Stability AI, Community), TripoSG (Tripo AI, Open), MeshAnything V2 (Open), Sparc3D (Open), Hunyuan3D-Paint (Tencent), FlexiCubes (NVIDIA).
71.1 @isis/3d-inference-local — Self-Hosted Open-Source 3D Model Inference#
Eliminate per-asset cost by self-hosting SOTA open-source 3D generation models on own GPU infrastructure. This is the single highest-impact gap closure.
71.1.1 Model Serving Infrastructure#
- 71.1.1.1 Create
libs/isis/3d-inference-local/project with project.json (scope:isis, layer:domain) - 71.1.1.2 Implement GPU model server framework with health checks, warm-up, and graceful shutdown
- 71.1.1.3 Create model registry for discovering, downloading, and versioning local model weights (HuggingFace Hub integration)
- 71.1.1.4 Implement model weight caching with LRU eviction (manage disk space across multiple large models)
- 71.1.1.5 Create GPU memory manager with VRAM budget tracking and automatic model offloading when memory pressure is high
- 71.1.1.6 Implement multi-model scheduling — load/unload models based on incoming request types (text-to-3D vs image-to-3D vs texturing)
- 71.1.1.7 Create request queue with priority levels (real-time interactive, batch, background)
- 71.1.1.8 Implement batch inference aggregation — combine compatible requests for throughput optimization
- 71.1.1.9 Create multi-GPU load balancing with affinity (pin specific models to specific GPUs)
- 71.1.1.10 Implement inference progress streaming via SSE (match Meshy's real-time progress API)
- 71.1.1.11 Create unified REST API matching Isis provider interface so self-hosted models are drop-in replacements for cloud APIs
- 71.1.1.12 Implement Docker containerization with NVIDIA Container Toolkit for reproducible deployment
- 71.1.1.13 Create Kubernetes Helm charts with GPU node selectors and resource limits
- 71.1.1.14 Implement RunPod serverless endpoint adapters (scale to zero when idle, spin up on demand)
- 71.1.1.15 Create cost tracking dashboard — compare self-hosted cost per asset vs cloud API cost per asset
71.1.2 TRELLIS.2 Integration (Microsoft — MIT License)#
Best-in-class geometry quality, clean topology, thin surfaces, complex structures. 4B parameters. MIT license = fully permissive commercial use.
- 71.1.2.1 Implement TRELLIS.2 model loader with fp16/bf16 precision options (16GB VRAM default, 8GB with fp16 mode)
- 71.1.2.2 Create TRELLIS.2 text-to-3D inference pipeline (text prompt → SLatS representation → mesh extraction)
- 71.1.2.3 Implement TRELLIS.2 image-to-3D inference pipeline (single image → multi-view → 3D reconstruction)
- 71.1.2.4 Create TRELLIS.2 multi-image conditioning mode (2-6 reference images for higher fidelity)
- 71.1.2.5 Implement TRELLIS.2 mesh extraction with configurable resolution (64³ to 512³ marching cubes)
- 71.1.2.6 Create TRELLIS.2 PBR material extraction (albedo, roughness, metallic from SLatS)
- 71.1.2.7 Implement TRELLIS.2 Gaussian Splatting output mode (alternative to mesh for real-time viewing)
- 71.1.2.8 Create TRELLIS.2 GLB/GLTF export with embedded PBR textures
- 71.1.2.9 Implement TRELLIS.2 quality presets (draft: low-res fast, standard: balanced, ultra: maximum quality)
- 71.1.2.10 Create TRELLIS.2 prompt engineering adapter — translate Isis prompt templates to TRELLIS-optimal format
- 71.1.2.11 Implement TRELLIS.2 seed control for reproducible generation
- 71.1.2.12 Create TRELLIS.2 negative prompt support for excluding unwanted features
- 71.1.2.13 Implement TRELLIS.2 style conditioning (photorealistic, stylized, low-poly, cartoon)
- 71.1.2.14 Create TRELLIS.2 benchmark suite comparing output quality against Meshy-6, Rodin, and Tripo
- 71.1.2.15 Implement TRELLIS.2 model update automation — detect and pull new releases from Microsoft GitHub
71.1.3 Hunyuan3D 2.1 Integration (Tencent — Mixed Source License)#
Near-commercial quality with full PBR. Only top-tier model releasing both weights AND training code. Runs on 6GB VRAM with optimization.
- 71.1.3.1 Implement Hunyuan3D 2.1 shape generation model loader (10GB VRAM, 6GB with mmgp memory management)
- 71.1.3.2 Create Hunyuan3D 2.1 text-to-shape inference pipeline (text → 3D shape in 10-25 seconds on high-end GPU)
- 71.1.3.3 Implement Hunyuan3D 2.1 image-to-shape inference pipeline (single/multi-image → 3D shape)
- 71.1.3.4 Create Hunyuan3D 2.1 multi-view image generation (generate front/side/back reference views from single image before 3D reconstruction)
- 71.1.3.5 Implement Hunyuan3D-Paint texture generation model loader (21GB VRAM, 6GB with mmgp)
- 71.1.3.6 Create Hunyuan3D-Paint PBR texture synthesis pipeline (albedo, normal, metallic, roughness maps from mesh + text prompt)
- 71.1.3.7 Implement Hunyuan3D 2.1 combined shape+texture pipeline (29GB VRAM full, 6GB sequential with mmgp)
- 71.1.3.8 Create Hunyuan3D 2.1 mmgp (Multi-Model GPU Parallelism) memory optimization for consumer GPUs
- 71.1.3.9 Implement Hunyuan3D 2.1 Blender addon integration (direct generation inside Blender viewport)
- 71.1.3.10 Create Hunyuan3D 2.1 texture resolution control (512, 1024, 2048, 4096 texture maps)
- 71.1.3.11 Implement Hunyuan3D 2.1 mesh topology control (triangle, quad-dominant output)
- 71.1.3.12 Create Hunyuan3D 2.1 fine-tuning pipeline — train custom LoRA adapters on project-specific asset styles
- 71.1.3.13 Implement Hunyuan3D 2.1 batch generation with throughput optimization
- 71.1.3.14 Create Hunyuan3D 2.1 quality comparison dashboard vs Meshy-6 PBR output
- 71.1.3.15 Implement Hunyuan3D 2.1 Windows portable deployment (self-contained package with all dependencies)
71.1.4 SF3D / SPAR3D Integration (Stability AI — Community License)#
Sub-second 3D generation. SF3D: 0.5s on consumer GPU. SPAR3D: 0.7s with interactive point cloud editing. Unmatched speed.
- 71.1.4.1 Implement SF3D model loader with half-precision support (~4GB VRAM minimum, 6GB recommended)
- 71.1.4.2 Create SF3D image-to-3D inference pipeline (single image → textured mesh in <0.5 seconds)
- 71.1.4.3 Implement SF3D UV unwrapping output (proper UV layout unlike TripoSR)
- 71.1.4.4 Create SF3D illumination-disentangled texture output (albedo without baked lighting)
- 71.1.4.5 Implement SPAR3D model loader with point cloud intermediate representation
- 71.1.4.6 Create SPAR3D image-to-3D inference pipeline (single image → point cloud → mesh in 0.7 seconds)
- 71.1.4.7 Implement SPAR3D interactive point cloud editing (user adjusts points before final mesh extraction)
- 71.1.4.8 Create SPAR3D real-time preview loop (edit point cloud → instantly see updated mesh)
- 71.1.4.9 Implement speed-optimized batch pipeline — process 100+ images to 3D models per minute using SF3D
- 71.1.4.10 Create quality-vs-speed routing — automatically select SF3D for speed-critical requests, TRELLIS for quality-critical
71.1.5 TripoSG Integration (Tripo AI — Open License)#
1.5B parameter rectified flow model. Significant quality improvement over TripoSR. Includes TripoSG-scribble variant for sketch-to-3D.
- 71.1.5.1 Implement TripoSG 1.5B model loader (~12GB VRAM)
- 71.1.5.2 Create TripoSG image-to-3D inference pipeline with improved topology over TripoSR
- 71.1.5.3 Implement TripoSG-scribble variant for sketch/doodle to 3D generation (rapid prototyping from rough drawings)
- 71.1.5.4 Create TripoSG multi-view consistency mode
- 71.1.5.5 Implement TripoSG output comparison against existing TripoSR provider to validate quality improvement
71.1.6 Sparc3D Integration (Open Weights)#
State-of-the-art reconstruction fidelity. 1024³ resolution watertight meshes. Novel Sparcubes + Sparconv-VAE architecture. 4x faster training than prior methods.
- 71.1.6.1 Implement Sparc3D model loader with pretrained weights
- 71.1.6.2 Create Sparc3D image-to-3D inference pipeline at 1024³ resolution
- 71.1.6.3 Implement Sparc3D watertight mesh output (guaranteed manifold, no holes — ideal for 3D printing)
- 71.1.6.4 Create Sparc3D open surface support (thin surfaces, disconnected components)
- 71.1.6.5 Implement Sparc3D resolution scaling (256³, 512³, 1024³) for quality-vs-speed tradeoff
- 71.1.6.6 Create Sparc3D benchmark against TRELLIS.2 and Hunyuan3D for geometric fidelity comparison
- 71.1.6.7 Implement Sparc3D integration with MeshAnything V2 for post-generation topology cleanup
71.1.7 MeshAnything V2 Integration (Open Source)#
Convert AI-generated meshes into artist-quality topology using autoregressive transformers. Adjacent Mesh Tokenization reduces token length by 50%. Up to 1600 faces.
- 71.1.7.1 Implement MeshAnything V2 model loader (~8GB VRAM)
- 71.1.7.2 Create MeshAnything V2 remeshing pipeline (AI-generated triangle soup → clean quad-dominant topology)
- 71.1.7.3 Implement MeshAnything V2 face count control (target 200, 400, 800, 1600 faces)
- 71.1.7.4 Create MeshAnything V2 integration as automatic post-processing step in all generation pipelines
- 71.1.7.5 Implement MeshAnything V2 edge flow optimization for animation-ready topology (proper edge loops around joints)
- 71.1.7.6 Create MeshAnything V2 LOD chain generation — run at multiple face counts to produce LOD0-LOD4 from single input
- 71.1.7.7 Implement MeshAnything V2 quality metrics — compare topology quality scores before/after remeshing
- 71.1.7.8 Create MeshAnything V2 batch processing for bulk asset library cleanup
71.1.8 FlexiCubes Integration (NVIDIA — Kaolin)#
High-quality isosurface extraction for gradient-based mesh optimization. Drop-in improvement over Marching Cubes for all generation pipelines.
- 71.1.8.1 Implement FlexiCubes mesh extraction as alternative to Marching Cubes in all generation pipelines
- 71.1.8.2 Create FlexiCubes adaptive resolution — higher detail in complex regions, lower in flat areas
- 71.1.8.3 Implement FlexiCubes differentiable rendering integration for iterative mesh refinement
- 71.1.8.4 Create FlexiCubes vs Marching Cubes quality comparison benchmark
- 71.1.8.5 Implement FlexiCubes integration with NVIDIA Kaolin library for GPU-accelerated mesh operations
71.1.9 Intelligent Model Routing#
- 71.1.9.1 Create model routing engine — select optimal model per request based on prompt analysis (organic → Hunyuan3D, hard surface → TRELLIS, speed-critical → SF3D, high-res → Sparc3D)
- 71.1.9.2 Implement quality-aware routing — run generation on multiple models in parallel, score outputs, return best result
- 71.1.9.3 Create cost-aware routing — prefer self-hosted models, fall back to cloud APIs only when local GPUs are fully loaded
- 71.1.9.4 Implement A/B testing framework — compare model versions, prompt strategies, and post-processing pipelines with statistical significance
- 71.1.9.5 Create generation analytics dashboard — track per-model success rates, quality scores, latency, GPU utilization, and cost
71.2 @isis/ai-texturing — AI-Native PBR Texture Generation & Refinement#
Close Meshy's remaining quality edge: polished PBR textures. Build a complete AI-native texturing pipeline using open-source models.
71.2.1 Hunyuan3D-Paint Integration#
- 71.2.1.1 Create
libs/isis/ai-texturing/project with project.json (scope:isis, layer:domain) - 71.2.1.2 Implement Hunyuan3D-Paint standalone texture generation (apply PBR textures to any existing mesh, not just Hunyuan-generated)
- 71.2.1.3 Create text-guided PBR texture generation (describe desired material → generate albedo, normal, metallic, roughness maps)
- 71.2.1.4 Implement image-guided texture generation (reference photo → extract material and apply to mesh)
- 71.2.1.5 Create multi-view consistent texture projection (no seams between front/back/side views)
- 71.2.1.6 Implement texture resolution scaling (512, 1024, 2048, 4096, 8192 px per map)
- 71.2.1.7 Create texture map completeness validation (detect missing coverage, UV islands without texture data)
- 71.2.1.8 Implement re-texturing — replace textures on existing models while preserving geometry and UVs (Meshy AI Retexture equivalent)
71.2.2 Depth-Conditioned Texture Painting#
TEXTure/Text2Tex approach enhanced with modern diffusion models.
- 71.2.2.1 Implement depth-conditioned diffusion texture painting (render depth map from mesh viewpoint → use as ControlNet condition → generate texture from target viewpoint)
- 71.2.2.2 Create iterative multi-view texture painting (front → side → back → top → bottom, each view inpaints unseen regions)
- 71.2.2.3 Implement view-consistency enforcement using cross-attention between generated views
- 71.2.2.4 Create lighting-disentangled texture generation (generate flat albedo without baked shadows or highlights)
- 71.2.2.5 Implement seam-aware texture blending at UV boundaries
- 71.2.2.6 Create normal map generation from depth and diffuse maps (derive surface detail from color variation)
- 71.2.2.7 Implement roughness and metallic map estimation from diffuse texture (ML-based material property inference)
- 71.2.2.8 Create ambient occlusion baking from mesh geometry
- 71.2.2.9 Implement displacement map generation from diffuse texture detail
- 71.2.2.10 Create texture atlas optimization — repack UV islands for maximum texel density utilization
71.2.3 AI Material Graph Generation#
Go beyond Meshy — generate full procedural material definitions, not just static textures.
- 71.2.3.1 Implement text-to-material-graph generation (describe material → generate Substance-style node graph)
- 71.2.3.2 Create material graph parameter extraction (identify tweakable parameters: color, scale, roughness range, wear amount)
- 71.2.3.3 Implement material tiling control (generate seamlessly tileable materials at any scale)
- 71.2.3.4 Create material variation generation (10 variations of "weathered concrete" with different stain patterns, crack density)
- 71.2.3.5 Implement material blending recipes (combine materials: "mossy stone" = stone base + moss overlay with height-based blend)
- 71.2.3.6 Create material library builder — batch-generate material sets for environments (urban, forest, desert, sci-fi, medieval)
- 71.2.3.7 Implement Substance Designer .sbs export from generated material graphs
- 71.2.3.8 Create OpenUSD MaterialX export for generated materials
71.2.4 Texture Super-Resolution & Enhancement#
- 71.2.4.1 Implement PBR-aware texture super-resolution (upscale albedo, normal, roughness, metallic maps with channel-appropriate models)
- 71.2.4.2 Create texture denoising that preserves material detail (remove compression artifacts without blurring micro-surface features)
- 71.2.4.3 Implement texture detail synthesis (add fine-grained detail to low-res textures: pores on skin, grain on wood, scratches on metal)
- 71.2.4.4 Create texture style transfer (apply reference material style to generated texture while preserving UV-specific detail)
- 71.2.4.5 Implement UDIM-aware texture processing (handle multi-tile UV layouts for high-detail characters and environments)
- 71.2.4.6 Create texture compression optimization (generate BC7/ASTC/ETC2-friendly textures that compress well for real-time use)
- 71.2.4.7 Implement AI-based texture baking (bake high-poly detail to low-poly normal maps using neural rendering instead of ray casting)
71.3 @isis/universal-rigging — Auto-Rigging for Any Topology#
Surpass Meshy's humanoid-only limitation. Support arbitrary mesh rigging through ML-based skeleton prediction.
71.3.1 ML-Based Skeleton Prediction#
- 71.3.1.1 Create
libs/isis/universal-rigging/project with project.json (scope:isis, layer:domain) - 71.3.1.2 Implement ML skeleton prediction model (input: mesh geometry → output: joint positions and hierarchy) — train on Mixamo, RigNet, and open rigging datasets
- 71.3.1.3 Create topology-agnostic joint placement (predict skeleton from mesh shape regardless of poly count or topology quality)
- 71.3.1.4 Implement bone hierarchy inference (parent-child relationships, joint orientation)
- 71.3.1.5 Create automatic skin weight computation (smooth binding with ML-predicted influence weights)
- 71.3.1.6 Implement joint limit inference from mesh proportions (predict natural range of motion per joint)
- 71.3.1.7 Create IK chain setup automation (set up IK solvers for limbs, spine, feet based on predicted skeleton)
- 71.3.1.8 Implement rigging confidence scoring (indicate areas where auto-rig may need manual adjustment)
71.3.2 Humanoid Rigging Enhancements#
- 71.3.2.1 Implement A-pose and T-pose auto-detection and conversion
- 71.3.2.2 Create facial rig generation with blend shapes (52 ARKit blend shapes + FACS action units)
- 71.3.2.3 Implement hand rig generation with individual finger articulation (21 joints per hand)
- 71.3.2.4 Create tongue and jaw rig for speech animation
- 71.3.2.5 Implement eye rig with look-at targets and eyelid controls
- 71.3.2.6 Create corrective blend shapes for shoulder, elbow, knee deformation
- 71.3.2.7 Implement clothing rig binding (bind garment mesh to character skeleton with offset)
- 71.3.2.8 Create Mixamo-compatible skeleton output (direct compatibility with Mixamo animation library)
- 71.3.2.9 Implement VRM-compatible skeleton output for VTuber/social VR applications
- 71.3.2.10 Create MetaHuman-compatible skeleton mapping
71.3.3 Non-Humanoid Rigging#
- 71.3.3.1 Implement quadruped auto-rigging (dogs, horses, cats, deer — 4-leg skeleton with spine flexibility)
- 71.3.3.2 Create bird auto-rigging (wings with feather spread, legs, head, tail)
- 71.3.3.3 Implement insect/arachnid auto-rigging (6-8 legs, mandibles, antennae, wings)
- 71.3.3.4 Create fish/marine auto-rigging (spine chain, fins, tail, jaw)
- 71.3.3.5 Implement serpent/snake auto-rigging (long spine chain with IK, jaw)
- 71.3.3.6 Create multi-limbed creature rigging (centaurs, multi-armed beings, tentacled creatures)
- 71.3.3.7 Implement winged creature rigging (dragons, bats, angels — wings + body + tail)
- 71.3.3.8 Create plant/tree rigging (trunk, branches, leaves with wind response)
- 71.3.3.9 Implement mechanical/robot rigging (joints, pistons, actuators, treads, turrets)
- 71.3.3.10 Create vehicle rigging (wheels with suspension, doors, hood, trunk, steering)
- 71.3.3.11 Implement furniture/prop rigging (drawers, doors, lids, levers, hinges)
- 71.3.3.12 Create custom creature template system (user defines skeleton topology, ML predicts optimal placement on mesh)
71.3.4 Animation Library & Retargeting#
- 71.3.4.1 Create humanoid animation library (1000+ motions: idle, walk, run, jump, combat, dance, gesture, emote) sourced from Mixamo + open mocap datasets
- 71.3.4.2 Implement quadruped animation library (walk, trot, gallop, sit, lie down, eat, shake, play)
- 71.3.4.3 Create animation retargeting engine — apply humanoid animations to any humanoid-rigged mesh regardless of proportions
- 71.3.4.4 Implement cross-species retargeting (human motion → quadruped motion with gait adaptation)
- 71.3.4.5 Create animation blending and transition graph for generated assets
- 71.3.4.6 Implement Mixamo animation import pipeline (download and apply Mixamo animations to auto-rigged meshes)
- 71.3.4.7 Create animation preview renderer (quick turntable + animation preview for quality validation)
- 71.3.4.8 Implement FBX, GLB, and BVH animation export
71.3.5 Assisted Rigging UX & Motion Library Productization#
- 71.3.5.1 Create guided landmark placement fallback for failed or low-confidence auto-rig attempts (chin, wrists, elbows, knees, ankles, spine root, shoulders)
- 71.3.5.2 Implement joint placement correction UI with before/after preview, symmetry snapping, and confidence heatmap overlay
- 71.3.5.3 Create guided skin-weight refinement flow for problematic regions (shoulders, hips, fingers, wings, tails, cloth attachments)
- 71.3.5.4 Implement rig validation playbook with one-click test poses, deformation stress tests, and auto-generated issue summaries
- 71.3.5.5 Create searchable motion catalog metadata (tags, locomotion type, emotion, speed, interaction class, loopability, root-motion status)
- 71.3.5.6 Implement one-click motion application UX with trim, mirror, in-place, root-motion, playback speed, and loop toggles
- 71.3.5.7 Create motion licensing and provenance records per clip so imported/open-library animations have clear usage metadata
- 71.3.5.8 Implement clip quality scoring and preview generation so low-quality or poorly retargeting motions are flagged before export
- 71.3.5.9 Create recommended-motion matching based on detected rig type, proportions, gait class, and project style profile
71.4 @isis/video-to-mesh — Video & Multi-View 3D Reconstruction#
Open new input modality that Meshy doesn't offer. Convert video footage directly into textured 3D meshes.
71.4.1 Monocular Video to 3D#
- 71.4.1.1 Create
libs/isis/video-to-mesh/project with project.json (scope:isis, layer:domain) - 71.4.1.2 Implement monocular depth estimation pipeline (single video → per-frame depth maps using Depth Anything V2 or equivalent)
- 71.4.1.3 Create camera pose estimation from video (structure from motion on video frames)
- 71.4.1.4 Implement multi-frame depth fusion (aggregate depth maps across frames into consistent 3D point cloud)
- 71.4.1.5 Create point cloud to mesh conversion (Poisson surface reconstruction or screened Poisson)
- 71.4.1.6 Implement texture mapping from video frames onto reconstructed mesh
- 71.4.1.7 Create object segmentation from video (isolate target object from background before reconstruction)
- 71.4.1.8 Implement turntable video reconstruction (optimize for common capture pattern: object on turntable)
- 71.4.1.9 Create quality assessment for input video (detect insufficient coverage, motion blur, poor lighting)
- 71.4.1.10 Implement progressive reconstruction with real-time preview (show mesh improving as more frames are processed)
71.4.2 Multi-View Reconstruction#
- 71.4.2.1 Implement COLMAP integration for structure-from-motion (multi-view images → camera poses + sparse point cloud)
- 71.4.2.2 Create dense multi-view stereo reconstruction (sparse points → dense point cloud → mesh)
- 71.4.2.3 Implement neural multi-view reconstruction using InstantMesh or equivalent (Apache 2.0, multi-view → 3D in ~10 seconds)
- 71.4.2.4 Create guided capture assistant (instruct user: "rotate 15 degrees left", "tilt down", "capture top view" for optimal coverage)
- 71.4.2.5 Implement smartphone capture integration (phone camera → stream frames → real-time reconstruction)
71.4.3 4D Gaussian Splatting to Animated Mesh#
- 71.4.3.1 Implement 4DGS temporal Gaussian field to animated mesh export (convert 4D Gaussians into mesh sequence or skeletal animation)
- 71.4.3.2 Create Gaussian mesh extraction with temporal coherence (consistent topology across frames)
- 71.4.3.3 Implement 4DGS-to-blend-shape conversion (extract facial or body deformation as blend shapes from Gaussian temporal field)
- 71.4.3.4 Create 4DGS compression for streaming animated content
- 71.4.3.5 Implement 4DGS editing tools for animated scene modification (move, scale, delete temporal Gaussians)
71.4.4 SLAM-Based 3D Scanning#
- 71.4.4.1 Implement visual SLAM pipeline for real-time camera tracking and sparse mapping
- 71.4.4.2 Create LiDAR integration for iPhone/iPad Pro (use ARKit LiDAR data for dense depth)
- 71.4.4.3 Implement SLAM-to-Gaussian pipeline (convert SLAM map to 3D Gaussian Splatting representation)
- 71.4.4.4 Create room-scale scanning (walk through a room → reconstruct full 3D environment)
- 71.4.4.5 Implement object-scale scanning with automatic background removal and mesh cleanup
71.5 @isis/3d-browser — Browser-Based 3D Generation & Viewing#
Match Meshy's browser-based UX while adding self-hosted power. WebGPU-native viewer, editor, and inference.
71.5.1 WebGPU 3D Viewer#
- 71.5.1.1 Create
libs/isis/3d-browser/project with project.json (scope:isis, layer:domain) - 71.5.1.2 Implement WebGPU PBR renderer for generated 3D assets (environment lighting, shadows, reflections)
- 71.5.1.3 Create 3D model turntable viewer with orbit controls, zoom, and pan
- 71.5.1.4 Implement wireframe overlay toggle (view mesh topology in browser)
- 71.5.1.5 Create UV layout visualization overlay
- 71.5.1.6 Implement PBR channel isolation (view albedo only, normal only, roughness only, metallic only)
- 71.5.1.7 Create polygon count and mesh statistics display
- 71.5.1.8 Implement model comparison mode (side-by-side or overlay two generated variants)
- 71.5.1.9 Create AR Quick Look integration (view generated model in real-world via phone camera)
- 71.5.1.10 Implement progressive mesh streaming (display low-LOD immediately, stream full quality progressively)
71.5.2 WebGPU 3D Gaussian Splatting Viewer#
- 71.5.2.1 Implement WebGPU 3DGS renderer (sort and splat Gaussians at 60+ fps in browser)
- 71.5.2.2 Create 3DGS scene loading from .ply and .splat formats
- 71.5.2.3 Implement 3DGS level-of-detail (reduce splat count at distance)
- 71.5.2.4 Create 3DGS scene compression for fast network transfer
- 71.5.2.5 Implement 3DGS-to-mesh conversion in browser (extract mesh from Gaussians client-side)
- 71.5.2.6 Create 4DGS animated scene playback in browser
71.5.3 Browser-Based 3D Editing#
- 71.5.3.1 Implement browser-based mesh sculpting (push, pull, smooth, flatten brushes via WebGPU compute)
- 71.5.3.2 Create browser-based texture painting (paint directly on 3D model in browser)
- 71.5.3.3 Implement vertex color painting for quick material assignment
- 71.5.3.4 Create basic mesh Boolean operations in browser (union, subtract, intersect)
- 71.5.3.5 Implement mesh decimation/simplification in browser (reduce poly count client-side)
- 71.5.3.6 Create UV unwrapping preview and manual adjustment in browser
- 71.5.3.7 Implement undo/redo history for all editing operations
- 71.5.3.8 Create export from browser editor (download GLB, FBX, OBJ, USDZ directly)
71.5.4 Browser-Based Inference (WASM)#
- 71.5.4.1 Implement SF3D WASM module — compile model to WebAssembly for sub-second image-to-3D in browser without server
- 71.5.4.2 Create TripoSR WASM module — lightweight 3D generation entirely client-side
- 71.5.4.3 Implement WebGPU inference backend for ONNX models (run quantized 3D generation models using WebGPU compute shaders)
- 71.5.4.4 Create model download and caching in IndexedDB (download model once, reuse across sessions)
- 71.5.4.5 Implement hybrid client/server inference (start generation client-side, offload to server if GPU is insufficient)
71.6 @isis/3d-asset-library — Searchable 3D Asset Marketplace#
Build ecosystem stickiness with a comprehensive, searchable library of pre-generated and community-contributed 3D assets.
71.6.1 Asset Library Core#
- 71.6.1.1 Create
libs/isis/3d-asset-library/project with project.json (scope:isis, layer:domain) - 71.6.1.2 Implement asset metadata schema (name, description, tags, category, poly count, texture resolution, format, license, generation parameters, source model, creation date)
- 71.6.1.3 Create asset ingestion pipeline (upload mesh → auto-extract metadata → generate thumbnails → index for search)
- 71.6.1.4 Implement asset thumbnail renderer (generate 6-view turntable thumbnail set per asset)
- 71.6.1.5 Create animated turntable GIF/video for each asset preview
- 71.6.1.6 Implement full-text search over asset metadata (name, description, tags)
- 71.6.1.7 Create semantic search (CLIP-based: search by text description or reference image)
- 71.6.1.8 Implement category taxonomy (characters, vehicles, architecture, furniture, nature, props, weapons, food, clothing, etc.)
- 71.6.1.9 Create filter system (poly count range, texture resolution, format, license, style, generation model)
- 71.6.1.10 Implement similar asset search (find assets visually similar to a given model)
71.6.2 Asset Variation & Remix#
- 71.6.2.1 Implement asset variation generation (given a library asset, generate N similar variants with controlled differences)
- 71.6.2.2 Create style transfer across assets (apply the material style of one asset to the geometry of another)
- 71.6.2.3 Implement asset color palette remapping (change color scheme while preserving material properties)
- 71.6.2.4 Create asset scale normalization (normalize all assets to real-world scale for consistent scene composition)
- 71.6.2.5 Implement asset kit generation (generate matching asset sets: "medieval tavern kit" with table, chairs, mugs, candles, barrels)
71.6.3 Style Consistency Engine#
- 71.6.3.1 Implement project style profile (define a project's visual identity: color palette, material properties, poly budget, texture style)
- 71.6.3.2 Create style-constrained generation (generate new assets matching an existing project style profile)
- 71.6.3.3 Implement style transfer post-processing (take any generated asset and adapt it to match project style)
- 71.6.3.4 Create style consistency scoring (measure how well a generated asset matches the project style profile)
- 71.6.3.5 Implement reference sheet extraction (analyze concept art or style guides to create style profiles automatically)
71.6.4 Asset Provenance & Licensing#
- 71.6.4.1 Implement generation provenance tracking (record exact prompt, model, seed, parameters, post-processing steps for every asset)
- 71.6.4.2 Create license assignment system (CC0, CC-BY, commercial, non-commercial, project-restricted)
- 71.6.4.3 Implement asset usage tracking (which projects use which assets, for license compliance)
- 71.6.4.4 Create C2PA content provenance metadata embedding (embed AI generation metadata per C2PA standard)
- 71.6.4.5 Implement asset export with embedded provenance (GLB/USD files carry generation metadata)
71.6.5 Community & Sharing#
- 71.6.5.1 Implement asset upload pipeline with automatic quality validation (reject malformed meshes, missing textures, oversized files)
- 71.6.5.2 Create asset rating and review system
- 71.6.5.3 Implement asset collection/bundle creation (curate sets of related assets)
- 71.6.5.4 Create asset download API with format conversion on download (request GLB, get FBX conversion server-side)
- 71.6.5.5 Implement usage analytics (most downloaded, trending, top rated per category)
71.7 @isis/3d-generation-benchmarks — Quality Benchmarking Suite#
Continuous, automated quality measurement to prove SOTA leadership. No commercial platform offers this level of transparency.
71.7.1 Benchmark Infrastructure#
- 71.7.1.1 Create
libs/isis/3d-generation-benchmarks/project with project.json (scope:isis, layer:domain) - 71.7.1.2 Implement standardized prompt test suite (200+ prompts across categories: organic, hard-surface, architectural, character, vehicle, nature, abstract, stylized, low-poly, photorealistic)
- 71.7.1.3 Create standardized image test suite (200+ reference images for image-to-3D benchmarking)
- 71.7.1.4 Implement automated generation runner (run all prompts through all providers/models, collect outputs)
- 71.7.1.5 Create benchmark result storage with versioned history (track quality over time as models are updated)
71.7.2 Quality Metrics#
- 71.7.2.1 Implement geometric quality metrics (watertight check, manifold validation, self-intersection count, degenerate triangle count, thin triangle ratio)
- 71.7.2.2 Create topology quality metrics (quad percentage, edge flow score, pole count, n-gon count, edge loop continuity)
- 71.7.2.3 Implement texture quality metrics (texel density uniformity, seam visibility score, UV overlap percentage, PBR channel completeness)
- 71.7.2.4 Create prompt adherence scoring (CLIP similarity between prompt and rendered views of generated mesh)
- 71.7.2.5 Implement image adherence scoring for image-to-3D (CLIP similarity + structural similarity between input image and generated mesh renders)
- 71.7.2.6 Create multi-view consistency scoring (render from 8+ viewpoints, measure inter-view coherence)
- 71.7.2.7 Implement Chamfer distance and Hausdorff distance against reference meshes (where ground truth is available)
- 71.7.2.8 Create human preference prediction model (train on human ratings to predict quality without manual review)
71.7.3 Competitive Comparison#
- 71.7.3.1 Implement Meshy-6 API benchmark runner (generate same prompts
via Meshy API for direct comparison) Retired 2026-09-15, not done:
Meshy's terms of use §2.6 (iv) forbid competitive analysis, and the owner
made Meshy an operator-only production lane
(
ISIS_CHROMA_RUNPOD_MVP_TODOS_2026-09-11.mdT.22, T.01.15), so the runnermeshy-api-benchmark-runner.tswas deleted andcompetitive-analysis-exclusions.spec.tsfails if Meshy returns as a comparison target. Reopens only with Meshy's written permission. 2026-09-18, board tag: retired by owner decision; it reopens only on Meshy's written permission, as the note above says.blocked:external - 71.7.3.2 Create Tripo API benchmark runner
- 71.7.3.3 Implement Rodin API benchmark runner
- 71.7.3.4 Create side-by-side comparison renderer (generate turntable videos of same prompt across all providers)
- 71.7.3.5 Implement automated benchmark report generation (weekly report with quality trends, regressions, improvements per model)
- 71.7.3.6 Create public benchmark leaderboard (optional: publish results to demonstrate SOTA leadership)
71.8 @isis/3d-post-pipeline — Unified Post-Processing Pipeline#
Combine all post-processing steps into one configurable pipeline that runs automatically after any generation.
71.8.1 Pipeline Orchestration#
- 71.8.1.1 Create
libs/isis/3d-post-pipeline/project with project.json (scope:isis, layer:domain) - 71.8.1.2 Implement configurable post-processing pipeline graph (define ordered steps: generate → remesh → UV → texture → rig → animate → export)
- 71.8.1.3 Create pipeline presets per use case: - Game-ready: generate → MeshAnything remesh → auto-UV → PBR texture → LOD chain → engine export (GLB/FBX) - 3D print: generate → Sparc3D watertight → manifold repair → STL/3MF - Film VFX: generate → high-res texture → UDIM layout → OpenUSD export - Stylized/Low-poly: generate → decimate → vertex color → GLB - Avatar: generate → humanoid rig → blend shapes → animation → VRM
- 71.8.1.4 Implement pipeline step skip/override (user can skip remeshing, force specific texture resolution, etc.)
- 71.8.1.5 Create pipeline execution monitoring with per-step timing, GPU memory usage, and intermediate result preview
71.8.2 Mesh Optimization Steps#
- 71.8.2.1 Implement automated manifold repair (close holes, fix non-manifold edges, remove interior faces)
- 71.8.2.2 Create mesh decimation with feature preservation (reduce poly count while preserving sharp edges, creases, and silhouette)
- 71.8.2.3 Implement automatic UV unwrapping with configurable algorithm (angle-based, least-squares conformal, ABF++)
- 71.8.2.4 Create UV packing optimization (maximize texture space utilization, configurable padding)
- 71.8.2.5 Implement LOD chain generation (auto-generate LOD0-LOD4 with configurable reduction ratios)
- 71.8.2.6 Create mesh symmetry enforcement (detect near-symmetric meshes and enforce perfect symmetry)
- 71.8.2.7 Implement mesh centering and orientation normalization (consistent up-axis, centered origin, grounded on Y=0)
71.8.3 Format Export#
- 71.8.3.1 Implement GLB/GLTF export with Draco mesh compression and KTX2 texture compression
- 71.8.3.2 Create FBX export with embedded textures and skeleton
- 71.8.3.3 Implement OpenUSD/USDA/USDC export with MaterialX materials
- 71.8.3.4 Create OBJ + MTL export with PBR material maps
- 71.8.3.5 Implement STL export (binary and ASCII) for 3D printing
- 71.8.3.6 Create 3MF export with multi-color support for 3D printing
- 71.8.3.7 Implement USDZ export for Apple AR Quick Look
- 71.8.3.8 Create BLEND export (Blender native format with materials and modifiers)
- 71.8.3.9 Implement Unity package export (.unitypackage with prefab, materials, textures)
- 71.8.3.10 Create Unreal asset export (FBX with Unreal material instance setup metadata)
- 71.8.3.11 Implement Godot resource export (.tres/.tscn with materials and mesh resources)
- 71.8.3.12 Create Roblox mesh export (RBXM format with Roblox material properties)
- 71.8.3.13 Implement VRM 1.0 export with humanoid metadata, spring bones, expression/blend-shape bindings, and VTuber-ready material mapping
71.9 @isis/3d-comfyui-nodes — ComfyUI 3D Generation Nodes#
Leverage the ComfyUI-3D-Pack ecosystem for node-based 3D workflows. Bring all self-hosted models into ComfyUI as native nodes.
71.9.1 ComfyUI Node Development#
- 71.9.1.1 Create
libs/isis/3d-comfyui-nodes/project with project.json (scope:isis, layer:domain) - 71.9.1.2 Implement TRELLIS.2 ComfyUI node (text-to-3D and image-to-3D with all provider options exposed)
- 71.9.1.3 Create Hunyuan3D 2.1 ComfyUI node (shape generation + texture generation as separate chainable nodes)
- 71.9.1.4 Implement SF3D ComfyUI node (sub-second image-to-3D)
- 71.9.1.5 Create SPAR3D ComfyUI node with interactive point cloud editing widget
- 71.9.1.6 Implement TripoSG ComfyUI node
- 71.9.1.7 Create Sparc3D ComfyUI node (high-resolution reconstruction)
- 71.9.1.8 Implement MeshAnything V2 ComfyUI node (topology optimization as post-processing node)
- 71.9.1.9 Create Hunyuan3D-Paint ComfyUI node (standalone texture generation for any mesh)
- 71.9.1.10 Implement mesh export ComfyUI node (GLB, FBX, USDZ, OBJ output with format selection)
71.9.2 ComfyUI Workflow Templates#
- 71.9.2.1 Create "Text-to-Game-Asset" workflow (text → TRELLIS.2 → MeshAnything V2 → Hunyuan3D-Paint → GLB export)
- 71.9.2.2 Create "Image-to-3D-Print" workflow (image → Sparc3D → manifold repair → STL export)
- 71.9.2.3 Create "Photo-to-Avatar" workflow (face photo → Hunyuan3D → auto-rig → blend shapes → VRM export)
- 71.9.2.4 Create "Sketch-to-3D" workflow (rough sketch → TripoSG- scribble → SF3D refinement → texture → GLB)
- 71.9.2.5 Create "Batch Asset Kit" workflow (style reference + asset list → generate matching set with style consistency)
71.10 GPU Infrastructure for Self-Hosted 3D Generation#
Ensure the infrastructure exists to run all self-hosted models efficiently. Extends Phase 19 (AWS ECS + RunPod) with 3D-specific GPU provisioning.
71.10.1 GPU Cluster Management#
- 71.10.1.1 Implement GPU node pool management for 3D inference (separate from training workloads)
- 71.10.1.2 Create GPU VRAM-aware scheduling (route 6GB models to consumer GPUs, 24GB+ models to A100/H100)
- 71.10.1.3 Implement model pre-warming (keep hot models loaded in VRAM, cold models on disk with fast swap)
- 71.10.1.4 Create auto-scaling based on 3D generation queue depth (spin up RunPod instances when queue exceeds threshold)
- 71.10.1.5 Implement GPU utilization monitoring and alerting (detect underutilized GPUs, idle model instances)
- 71.10.1.6 Create spot/preemptible GPU instance support for batch generation (50-70% cost reduction)
- 71.10.1.7 Implement model sharding across multiple GPUs for models exceeding single GPU VRAM (tensor parallelism)
71.10.2 Inference Optimization#
- 71.10.2.1 Implement model quantization (INT8, INT4) for all self-hosted models to reduce VRAM and increase throughput
- 71.10.2.2 Create TensorRT optimization for NVIDIA GPUs (2-4x inference speedup)
- 71.10.2.3 Implement torch.compile optimization with stable-fast for generation models
- 71.10.2.4 Create ONNX runtime integration for cross-platform inference (CUDA, DirectML, CoreML backends)
- 71.10.2.5 Implement speculative decoding for autoregressive 3D models (MeshAnything V2 speedup)
- 71.10.2.6 Create inference caching (hash prompt + model version → return cached result for repeated requests)
- 71.10.2.7 Implement progressive generation with early preview (return low-quality preview in 2 seconds, refine to full quality)
71.11 @isis/3d-semantic-editing — Part-Aware Segmentation, Local Edit & Iterative Refinement#
Close the remaining iterative editing gap against Tripo, Meshy, and DCC-grade workflows. Support semantic part decomposition, localized geometry/material edits, part replacement, merge/weld, and reversible refinement loops.
71.11.1 Semantic Part Decomposition#
- 71.11.1.1 Create
libs/isis/3d-semantic-editing/project with project.json (scope:isis, layer:domain) - 71.11.1.2 Implement semantic part segmentation pipeline (characters, props, vehicles, architecture, furniture, creatures)
- 71.11.1.3 Create instance-aware part labeling (left/right doors, repeated wheels, mirrored limbs, multi-seat furniture components)
- 71.11.1.4 Implement editable part masks across mesh space, UV space, and texture space for unified localized editing
- 71.11.1.5 Create hierarchical part tree representation (asset → assembly → subassembly → part → material region)
- 71.11.1.6 Implement automatic connector and joint detection (hinges, sockets, weld seams, articulation pivots, hard-surface mating surfaces)
- 71.11.1.7 Create cross-format part metadata embedding for GLB, USD, and BLEND handoff so semantic edits survive round-trips
- 71.11.1.8 Implement segmentation confidence scoring with low-confidence region highlighting and manual correction recommendations
71.11.2 Localized Editing Workflows#
- 71.11.2.1 Implement local geometry edit prompting scoped to selected parts only ("replace blade", "thicken handle", "open visor")
- 71.11.2.2 Create localized texture repaint and material override workflows scoped to selected parts or UV islands
- 71.11.2.3 Implement protected-region editing locks so untouched parts preserve geometry, UVs, materials, and rig bindings
- 71.11.2.4 Create part replacement flow using generated candidates, marketplace assets, or project-approved donor parts
- 71.11.2.5 Implement localized remesh and topology cleanup that only affects the edited region and preserves surrounding edge flow
- 71.11.2.6 Create masked geometry infill for missing, occluded, or damaged regions discovered during edit and review loops
- 71.11.2.7 Implement smart edit propagation for symmetric parts and repeated instances (mirrored limbs, wheel sets, windows, bolts)
- 71.11.2.8 Create before/after diff visualization per part across mesh, texture, material, and rig state
- 71.11.2.9 Implement iterative edit sessions with branch history, rollback, and non-destructive variant exploration
- 71.11.2.10 Create domain-specific edit recipes ("replace wheel", "change jacket fabric", "convert to battle damage", "turn chair into stool")
71.11.3 Part Assembly, Merge & Validation#
- 71.11.3.1 Implement one-click part separation export for downstream DCC tools and marketplace packaging
- 71.11.3.2 Create part merge and weld pipeline with seam cleanup, tangent continuity repair, and material boundary preservation
- 71.11.3.3 Implement connector compatibility system for sockets, pegs, rails, hinges, mounting points, and modular architecture joins
- 71.11.3.4 Create snap-and-align constraints using pivots, normals, bounding features, sockets, and semantic landmarks
- 71.11.3.5 Implement kitbash-to-final-asset bake workflow (merge parts, repack UVs, rebake textures, rebuild LODs, regenerate metadata)
- 71.11.3.6 Create semantic merge conflict resolver when multiple edits target overlapping parts, connectors, or material regions
- 71.11.3.7 Implement assembly validation for floating parts, self-intersections, inverted normals, orphan materials, and rig disconnects
- 71.11.3.8 Create reusable part library indexing and search with connector compatibility, semantic tags, and style profile awareness
71.12 @bellona/blender-agent — Blender MCP Server & DCC Parity#
Architecture: Python MCP server running inside Blender. The server exposes
bpyoperations as MCP tools. Any MCP client (Claude Code, custom agents) can drive Blender directly — no custom RPC protocol or NL planner needed, as the LLM handles tool selection and sequencing natively via the MCP protocol. Domain-specific capabilities (Geometry Nodes, materials, rigging, scene assembly) are implemented as MCP tool groups within the server.
71.12.1 Blender MCP Server Runtime#
- 71.12.1.1 Create
libs/bellona/blender-agent/project with project.json (scope:bellona, layer:domain) - 71.12.1.2 Implement Python MCP server running inside Blender with
stdio and streamable-HTTP transports, exposing
bpyoperations as MCP tools for both interactive and headless execution - 71.12.1.3 Create MCP tool definitions per domain — object, mesh, armature, modifier, material, node, camera, light, and render operations — each with typed input schemas and structured result output
- 71.12.1.4
Implement natural-language-to-action plannerRemoved — the LLM handles planning natively via MCP tool calling - 71.12.1.5 Create semantic selection resolver ("select all door
handles", "isolate left forearm", "find emissive materials") as an MCP
tool (
resolve_selection) - 71.12.1.6 Implement current-scene context capture (collections, visibility, active object, modifiers, materials, constraints, linked files) as an MCP resource
- 71.12.1.7 Create reversible transaction layer with undo checkpoints, dry-run mode, and operation preview before destructive edits
- 71.12.1.8 Implement headless batch execution mode for render farms and automated post-processing jobs (MCP stdio transport)
- 71.12.1.9 Create execution audit log capturing exact
bpycalls, operator parameters, file diffs, and timing - 71.12.1.10 Implement Blender-side permission policy for destructive actions (delete, apply modifiers, overwrite materials, unlink collections)
71.12.2 Geometry Nodes, Materials, Rigging & Physics Automation#
- 71.12.2.1 Implement Geometry Nodes graph authoring from structured intents (scatter, instancing, bevel, paneling, terrain dressing, growth)
- 71.12.2.2 Create Geometry Nodes template library for common procedural setups (foliage scatter, cables, kitbash panels, debris, modular facades)
- 71.12.2.3 Implement node graph diff and merge for generated Geometry Nodes changes so users can inspect and accept individual edits
- 71.12.2.4 Create shader and material graph authoring targeting Principled BSDF with MaterialX-compatible parameter mapping
- 71.12.2.5 Implement UV, weighted normals, remesh, decimate, shrinkwrap, bevel, boolean, and modifier-stack cleanup macros callable by the agent
- 71.12.2.6 Create rig generation and rig editing macros for armatures, constraints, drivers, IK/FK setup, custom controls, and corrective shapes
- 71.12.2.7 Implement animation blocking and polish macros covering keyframe creation, NLA track setup, retargeting, and preview playblasts
- 71.12.2.8 Create groom, hair, fur, and curve-based generation macros for characters, creatures, foliage, and stylized assets
- 71.12.2.9 Implement cloth, rigid-body, and basic soft-body setup macros for garments, capes, props, and destruction-ready previews
- 71.12.2.10 Create validation rules ensuring generated node graphs, rigs, and physics setups remain editable, named, and production-readable
71.12.3 Scene Assembly, Review & Delivery Inside Blender#
- 71.12.3.1 Implement scene graph authoring from prompts or structured scene specs (collections, parent-child relationships, naming conventions)
- 71.12.3.2 Create asset placement workflow with collision checks, real-world scale normalization, orientation rules, and snap targets
- 71.12.3.3 Implement camera and lighting staging presets for turntable, product render, hero shot, animation blocking, and review dailies
- 71.12.3.4 Create Blender Asset Browser integration and kitbash assembly workflows backed by Oshun asset library metadata
- 71.12.3.5 Implement render preview and review sheet generation from Blender (contact sheet, clay render, wireframe, UV preview, material IDs)
- 71.12.3.6 Create
.blendpackaging workflow with linked assets, dependency manifest, missing-file checks, and reproducible rebuild data - 71.12.3.7 Implement exporter validation inside Blender for GLB, FBX, USD, VRM, and engine-target handoff presets
- 71.12.3.8 Create publish-to-library flow from Blender back into Isis asset library with provenance, thumbnails, and benchmark metadata
- 71.12.3.9 Implement operator suggestion and recovery hints when the agent's execution confidence is low or a Blender operator fails
- 71.12.3.10 Create CLI and UI smoke tests for Blender addon panels, headless workflows, and representative agent execution scenarios
71.12.4 Addon Productization, Onboarding & In-Editor Asset Workflows#
- 71.12.4.1 Create distributable Blender addon packaging with signed release bundles, version metadata, compatibility matrix, and rollback path
- 71.12.4.2 Implement first-run onboarding flow with connection test, auth/session setup, workspace defaults, and safe-mode explanation
- 71.12.4.3 Create addon preferences panel for endpoint selection, model routing defaults, GPU/offload options, privacy settings, and logs
- 71.12.4.4 Implement resilient auth and reconnect UX for expired sessions, dropped bridge connections, and headless/background execution
- 71.12.4.5 Create in-editor asset browser for personal, team, and marketplace assets with search, filters, previews, and provenance badges
- 71.12.4.6 Implement one-click send/import workflows between Blender and Isis/Bellona for generate, fetch, update, publish, and replace operations
- 71.12.4.7 Create update-channel management (stable, beta, nightly) and in-addon upgrade notifications with migration notes
- 71.12.4.8 Implement diagnostics panel covering bridge health, auth status, dependency checks, exporter readiness, and recent operation logs
- 71.12.4.9 Create guided recovery and support bundle export for failed addon actions (collect logs, environment details, traceback, scene summary)
71.13 @isis/3d-quality-gates — Release Gates, Human Review & Active Learning#
Turn the benchmark suite into a shipping-quality governance system with hard release gates, review queues, failure taxonomies, automated repair policies, and active learning loops for continuous improvement.
71.13.1 Failure Taxonomy & Human Review Operations#
- 71.13.1.1 Create failure taxonomy schema covering geometry, topology, UVs, textures, materials, rigging, animation, export, compliance, and UX
- 71.13.1.2 Implement review queue with severity, SLA, ownership, benchmark category, and affected provider/model version fields
- 71.13.1.3 Create annotation tools for defect marking on mesh regions, UV islands, texture maps, and turntable review frames
- 71.13.1.4 Implement human review assignment, consensus review, and tie-break workflow for disputed quality judgments
- 71.13.1.5 Create gold-standard benchmark corpus with approved exemplar outputs per category and use case
- 71.13.1.6 Implement root-cause capture linking each failure to prompt, model, post-step, export path, and environment conditions
- 71.13.1.7 Create escalation workflow for recurrent failure clusters, degraded benchmark trends, and regressions after model updates
- 71.13.1.8 Implement reviewer calibration, rubric training, and inter-rater agreement tracking for benchmark and marketplace review teams
71.13.2 Automated Gates, Retry Policies & Delivery Certification#
- 71.13.2.1 Implement per-use-case release gates (game-ready, 3D print, film/VFX, avatar, marketplace listing, browser viewer compatibility)
- 71.13.2.2 Create hard failure thresholds that block delivery when manifold, UV, texture, rig, or export invariants are violated
- 71.13.2.3 Implement soft score thresholds that trigger auto-retry, alternate model routing, or post-processing repair instead of silent ship
- 71.13.2.4 Create stepwise repair policy engine (remesh, UV repack, texture repaint, rerig, re-export, re-benchmark) with bounded retries
- 71.13.2.5 Implement canary benchmark runs for every new model version, prompt adapter update, and exporter change before promotion
- 71.13.2.6 Create regression diff reports with explicit pass/fail signoff requirement before benchmark regressions are accepted
- 71.13.2.7 Implement output quarantine for assets that fail quality, compliance, or provenance gates
- 71.13.2.8 Create delivery readiness certificate embedded into asset metadata with benchmark scores, gate results, and validation timestamps
- 71.13.2.9 Implement continuous evaluation in CI/cron with alerts for drift, latency regressions, export breakage, and benchmark score drops
- 71.13.2.10 Create customer-visible quality status indicators so users can see when an asset is draft, repaired, validated, or benchmark-approved
71.13.3 Active Learning, Dataset Curation & Model Improvement#
- 71.13.3.1 Implement sampling strategy that prioritizes low-confidence outputs, recurring failures, and underrepresented prompt categories
- 71.13.3.2 Create corrected data capture pipeline from reviewer annotations and accepted repair outcomes
- 71.13.3.3 Implement fine-tuning candidate dataset builder grouped by failure mode, style profile, and target use case
- 71.13.3.4 Create prompt template optimization loop driven by benchmark failures and human-reviewed successful repairs
- 71.13.3.5 Implement synthetic data augmentation for sparse categories (non-humanoids, modular kits, hard-surface interiors, stylized props)
- 71.13.3.6 Create active learning dashboard showing coverage, failure density, retraining yield, and quality uplift over time
- 71.13.3.7 Implement shadow testing for retrained models and adapters before they can replace existing production routes
- 71.13.3.8 Create rollback policy for any quality regression detected after model promotion, exporter changes, or routing-rule updates
71.14 @isis/model-governance-3d — Open-Source License, Commercial Policy & Deployment Compliance#
Make the self-hosted stack legally and operationally safe by tracking model licenses, enforcing commercial-use policy, blocking unauthorized deployment, and auditing model lineage for every generated asset.
71.14.1 Model License Registry & Policy Enforcement#
- 71.14.1.1 Create
libs/isis/model-governance-3d/project with project.json (scope:isis, layer:domain) - 71.14.1.2 Implement model license registry with normalized fields for weights, code, derivatives, redistribution, fine-tuning, and output rights
- 71.14.1.3 Create commercial-use policy matrix per model, adapter, fine-tune, and deployment topology
- 71.14.1.4 Implement allow, deny, and warn rules for mixed-source, community, research-only, and changeable upstream licenses
- 71.14.1.5 Create dependency inheritance tracking for derived LoRAs, merged checkpoints, quantized weights, and converted runtimes
- 71.14.1.6 Implement policy-aware routing that excludes restricted models based on tenant, project type, region, and commercial intent
- 71.14.1.7 Create user-facing license disclosure in API and UI before generation or export begins
- 71.14.1.8 Implement upstream model card and license change monitor with alerts whenever a referenced model changes terms or repository metadata
71.14.2 Deployment Guardrails & Auditability#
- 71.14.2.1 Implement signed model manifest containing source URL, hash, version, license snapshot, approval state, and benchmark status
- 71.14.2.2 Create deployment admission checks that block unapproved or tampered weights from loading into inference infrastructure
- 71.14.2.3 Implement region and jurisdiction deployment policy rules for self-hosted clusters, serverless endpoints, and customer-managed installs
- 71.14.2.4 Create training-data provenance capture fields where available so legal/compliance teams can assess upstream dataset claims
- 71.14.2.5 Implement audit log for every generation capturing model lineage, adapter lineage, quantization mode, and exporter lineage
- 71.14.2.6 Create export-time commercial eligibility stamp indicating whether the selected asset can be used commercially under current policy
- 71.14.2.7 Implement customer-facing attribution bundle generation when model terms require notices, attribution, or documentation
- 71.14.2.8 Create periodic compliance audit reports for legal, operations, and enterprise customer review
71.14.3 Data, Style & Output Compliance#
- 71.14.3.1 Implement prompt and output policy checks for prohibited IP, trademark misuse, restricted branded products, and disallowed replicas
- 71.14.3.2 Create near-duplicate detection against protected internal asset libraries and customer-provided proprietary asset sets
- 71.14.3.3 Implement style-imitation risk flags for branded franchises, signature product lines, and protected marketplace catalogs
- 71.14.3.4 Create restricted dataset tagging for customer fine-tunes, adapters, and private style packs with contract-based usage rules
- 71.14.3.5 Implement retention and deletion policy controls for uploaded references, generated assets, review artifacts, and benchmark outputs
- 71.14.3.6 Create model sunset and forced migration workflow for cases where licenses change or upstream compliance issues are discovered
- 71.14.3.7 Implement exception approval path with expiry, approver identity, scope restriction, and audit trail
- 71.14.3.8 Create compliance test suite covering model policy permutations, export restrictions, and mixed-license routing scenarios
71.15 @isis/3d-marketplace-ops — Marketplace Trust, IP Protection & Seller Operations#
Upgrade the asset library into a production marketplace with identity, moderation, takedown, dispute resolution, listing governance, and seller operations needed for real commercial use.
71.15.1 Rights, Identity & Publisher Trust#
- 71.15.1.1 Create
libs/isis/3d-marketplace-ops/project with project.json (scope:isis, layer:domain) - 71.15.1.2 Implement creator and seller verification workflow for individual artists, studios, and enterprise publishers
- 71.15.1.3 Create ownership declaration and license grant capture during upload, remix publication, and bundle creation
- 71.15.1.4 Implement collaborator attribution and revenue-split model for multi-author asset packs and derived works
- 71.15.1.5 Create chain-of-custody tracking for remixed, retopologized, retextured, and part-replaced derivatives
- 71.15.1.6 Implement trademark and brand-usage disclosure fields with required warnings on publication
- 71.15.1.7 Create trust scoring based on account age, verification, prior disputes, listing quality, and customer support history
- 71.15.1.8 Implement tax, KYC, and payout-readiness integration hooks for sellers before monetized listings can go live
71.15.2 Trust, Safety, IP Enforcement & Moderation#
- 71.15.2.1 Implement DMCA and IP takedown intake workflow with evidence collection, claim tracking, and SLA-based routing
- 71.15.2.2 Create counterfeit, stolen asset, and unauthorized reupload reporting flow for creators, customers, and moderators
- 71.15.2.3 Implement duplicate and near-duplicate detection using geometry fingerprints, texture hashes, metadata similarity, and previews
- 71.15.2.4 Create quarantine and hold states for suspicious listings, disputed listings, and compliance-pending listings
- 71.15.2.5 Implement strike, warning, appeal, and reinstatement system for repeat infringers and chronic policy violators
- 71.15.2.6 Create moderation tools for previews, metadata, comments, collections, bundles, and external embed surfaces
- 71.15.2.7 Implement embargoed/private project asset controls for enterprise customers and unreleased productions
- 71.15.2.8 Create watermarking and fingerprinting for leak tracing, provenance verification, and private-sharing misuse detection
71.15.3 Listing Governance, Commerce & Customer Operations#
- 71.15.3.1 Implement listing states (draft, review, published, deprecated, hidden, suspended, takedown, enterprise-private)
- 71.15.3.2 Create pricing, subscription, per-seat, and project-scoped licensing options for marketplace listings
- 71.15.3.3 Implement seller payout ledger and reconciliation hooks for purchases, refunds, chargebacks, and split payouts
- 71.15.3.4 Create refund and dispute workflow for broken exports, missing deliverables, compliance problems, or infringement claims
- 71.15.3.5 Implement SLA-backed asset support and buyer-seller contact channels for enterprise and premium listings
- 71.15.3.6 Create sales analytics and conversion funnel reporting for views, downloads, purchases, refunds, and repeat customers
- 71.15.3.7 Implement versioned listing updates with customer migration notices, compatibility notes, and deprecation warnings
- 71.15.3.8 Create enterprise procurement checklist covering export formats, compliance docs, provenance, support terms, and licensing status
71.16 @isis/3d-product-parity — Prompting, Collaboration, Derived Media & Workflow UX#
Close remaining product-surface gaps by improving prompting, input quality, review/versioning, collaboration, and downstream derived-media workflows that commercial tools expose around core generation.
71.16.1 Prompting & Input Quality Assistance#
- 71.16.1.1 Create
libs/isis/3d-product-parity/project with project.json (scope:isis, layer:domain) - 71.16.1.2 Implement prompt helper tuned for 3D generation that suggests geometry, material, scale, style, and use-case details
- 71.16.1.3 Create structured prompt builder (subject, material family, style target, topology target, poly budget, export target, animation need)
- 71.16.1.4 Implement opt-out prompt enhancement for vague inputs with benchmark-backed templates rather than opaque rewriting
- 71.16.1.5 Create reference image quality enhancement and background cleanup before image-to-3D generation
- 71.16.1.6 Implement sketch cleanup and line extraction for doodle, scribble, and concept-art to 3D flows
- 71.16.1.7 Create guided capture checklist for image and video inputs with coverage scoring and specific recapture instructions
- 71.16.1.8 Implement multimodal prompt inspector showing the text, images, masks, and derived conditioning actually passed into each model
71.16.2 Versioning, Review, Collaboration & Handoff#
- 71.16.2.1 Implement asset generation session history with branch, fork, rollback, restore, and compare operations
- 71.16.2.2 Create side-by-side comparison across versions, providers, seeds, post-processing variants, and export variants
- 71.16.2.3 Implement threaded comments pinned to mesh regions, turntable timestamps, UV islands, and material slots
- 71.16.2.4 Create approval states for promoting assets into team libraries, marketplace listings, or production-ready catalogs
- 71.16.2.5 Implement shareable review links with expiring permissions, watermarking, and audit logs
- 71.16.2.6 Create team workspace collections, assignment queues, and handoff checklists for modelers, technical artists, and reviewers
- 71.16.2.7 Implement batch review actions for large asset sets (approve, reject, reroute, retexture, export, archive)
- 71.16.2.8 Create provenance-aware diff view covering prompts, seeds, models, adapters, textures, rigs, and export settings across versions
71.16.3 Derived Media, Sharing & Delivery Surfaces#
- 71.16.3.1 Implement 3D-to-image renderer for catalog stills, concept sheets, hero renders, and material breakdowns
- 71.16.3.2 Create 3D-to-video generator for turntables, exploded-view promos, assembly demonstrations, and social-ready clips
- 71.16.3.3 Implement exploded-view and part-callout render generation for marketplace listings, assembly docs, and review packets
- 71.16.3.4 Create AR share packages for iOS, Android, and web viewers with canonical scale, floor placement, and lighting presets
- 71.16.3.5 Implement embeddable web viewer snippets for external sites, docs portals, and commerce surfaces
- 71.16.3.6 Create API and webhook hooks for downstream DCC, game engine, render farm, and catalog automation
- 71.16.3.7 Implement customer-facing job templates for common flows (text-to-prop, image-to-avatar, scan-to-print, retexture, marketplace prep)
- 71.16.3.8 Create portable offline package export for enterprise and air-gapped deployments with model manifests and dependency bundles
71.17 @isis/3d-scene-assembly — Scene Graph, Kitbashing & Procedural Assembly#
Fill the scene-composition and assembly gap left by single-asset generation by introducing semantic scene graphs, connector-aware kitbashing, and procedural assembly workflows for modular content.
71.17.1 Semantic Scene Graph & Constraint System#
- 71.17.1.1 Create
libs/isis/3d-scene-assembly/project with project.json (scope:isis, layer:domain) - 71.17.1.2 Implement semantic scene graph schema covering assets, parts, materials, connectors, transforms, constraints, and provenance
- 71.17.1.3 Create placement rules engine using real-world scale, clearance zones, support surfaces, and category-aware defaults
- 71.17.1.4 Implement socket and connector taxonomy for modular props, architecture kits, weapons, mechs, furniture, and vehicle parts
- 71.17.1.5 Create collision and overlap validation during scene and assembly authoring before assets are merged or exported
- 71.17.1.6 Implement symmetry-aware and repetition-aware placement tools for mirrored assemblies, repeated windows, bolts, or seats
- 71.17.1.7 Create assembly intent language for structured kit instructions ("attach four legs to base", "mount turret on roof socket")
- 71.17.1.8 Implement scene-unit normalization and world-axis enforcement across imported assets from mixed generators and DCC tools
71.17.2 Kitbashing & Procedural Composition Workflows#
- 71.17.2.1 Create reusable kitbash library builder from marketplace assets, generated parts, and approved donor-part catalogs
- 71.17.2.2 Implement semantic search for compatible connectors and donor parts during assembly and localized editing flows
- 71.17.2.3 Create automatic donor-part suggestions when a localized edit request would be better solved by replacement than regeneration
- 71.17.2.4 Implement procedural assembly recipes for common classes (mechs, furniture, weapons, building facades, room kits, modular vehicles)
- 71.17.2.5 Create variation generator for assembled kits while preserving connector validity, style coherence, and scale constraints
- 71.17.2.6 Implement scene-level style harmonization across mixed-source assets using material adaptation and geometry profile constraints
- 71.17.2.7 Create geometry and material bake-down from assemblies into a single shipping asset or optimized scene package
- 71.17.2.8 Implement assembly provenance graph showing every source asset, donor part, generated replacement, and post-processing step used
71.17.3 Delivery, Validation & Documentation#
- 71.17.3.1 Implement game-engine readiness validation for assembled assets and scenes (pivot sanity, naming, scale, material, LOD, collision)
- 71.17.3.2 Create printability validation for multipart assemblies with support analysis, tolerances, wall thickness, and connector fit checks
- 71.17.3.3 Implement exploded assembly instruction export to PDF, HTML, and interactive 3D viewer formats
- 71.17.3.4 Create bill-of-materials and material-usage summaries for assembled kits and multi-part printed products
- 71.17.3.5 Implement automated scene preview renders from canonical camera angles for review, commerce, and documentation
- 71.17.3.6 Create regression tests for assembly rules, connector compatibility, and procedural recipe stability
- 71.17.3.7 Implement assembly repair suggestions when constraints fail, connectors mismatch, or geometry overlaps remain unresolved
- 71.17.3.8 Create batch scene generation from structured asset lists, project style profiles, and room/layout templates
71.18 @bellona/dcc-ecosystem-parity — Maya, 3ds Max & Cross-Editor Bridge Productization#
Close the remaining ecosystem-breadth and in-editor workflow gaps by adding Autodesk DCC support and fully productizing Bellona bridges across game engines so Oshun matches or exceeds the commercial plugin surface.
71.18.1 Maya Adapter, Bridge Runtime & Artist Workflow#
- 71.18.1.1 Create
libs/bellona/maya/project with project.json (scope:bellona, layer:domain) and establish version-support policy for supported Maya releases - 71.18.1.2 Implement Maya bridge runtime using Python/OpenMaya command execution plus resilient IPC/WebSocket transport for local and remote control
- 71.18.1.3 Create typed Maya action schema covering DAG nodes, meshes, joints, skin clusters, blend shapes, materials, namespaces, references, and animation clips
- 71.18.1.4 Implement FBX, USD, and GLTF import/export workflows with namespace-safe scene insertion, replace/update modes, and provenance metadata round-tripping
- 71.18.1.5 Create Maya rigging and animation utilities for HumanIK, skeleton validation, blend-shape binding, clip retargeting, and playblast preview
- 71.18.1.6 Implement one-click send/import workflows between Maya and Isis/Bellona for generate, fetch, update, publish, and replace operations
- 71.18.1.7 Create Maya shelf, dockable panels, and asset browser hooks for search, previews, auth state, diagnostics, and publish actions
- 71.18.1.8 Implement automated smoke tests for installation, startup, bridge connectivity, import/export, animation handoff, and version compatibility across supported Maya releases
71.18.2 3ds Max Adapter, Bridge Runtime & Artist Workflow#
- 71.18.2.1 Create
libs/bellona/3dsmax/project with project.json (scope:bellona, layer:domain) and establish supported 3ds Max version matrix - 71.18.2.2 Implement 3ds Max bridge runtime using MaxScript/Python execution plus resilient IPC/WebSocket transport for editor control and background jobs
- 71.18.2.3 Create typed 3ds Max action schema covering nodes, Editable Poly workflows, modifiers, materials, skeletons, CAT/Biped rigs, layers, and animation takes
- 71.18.2.4 Implement FBX, USD, OBJ, and GLTF import/export workflows with material conversion, unit normalization, replace/update modes, and provenance metadata
- 71.18.2.5 Create 3ds Max rigging and animation utilities for CAT, Biped, Skin, Morpher, retargeting, and viewport preview/playblast support
- 71.18.2.6 Implement one-click send/import workflows between 3ds Max and Isis/Bellona for generate, fetch, update, publish, and replace operations
- 71.18.2.7 Create 3ds Max toolbar, dockable UI, and asset browser hooks for search, previews, auth state, diagnostics, and publish actions
- 71.18.2.8 Implement automated smoke tests for installation, startup, bridge connectivity, import/export, rigged asset handoff, and version compatibility across supported 3ds Max releases
71.18.3 Unity, Unreal & Godot Bridge Productization#
- 71.18.3.1 Create distributable Unity, Unreal, and Godot plugin/addon packages with signed release artifacts, compatibility matrix, and rollback path
- 71.18.3.2 Implement first-run onboarding and connection wizard inside Unity, Unreal, and Godot with endpoint selection, auth/session setup, and environment checks
- 71.18.3.3 Create in-editor asset browser for personal, team, and marketplace assets with search, filters, previews, provenance badges, and target-platform handoff hints
- 71.18.3.4 Implement one-click send/import workflows for generate, fetch, update, publish, replace, and instantiate-in-scene operations from within each editor
- 71.18.3.5 Create resilient auth, reconnect, and offline-safe recovery UX for expired sessions, dropped bridge connections, and temporarily unavailable services
- 71.18.3.6 Implement update-channel management (stable, beta, nightly), in-editor upgrade notifications, and migration notes for each host
- 71.18.3.7 Create diagnostics panels and guided recovery flows covering bridge health, dependency checks, importer/exporter readiness, logs, and support bundle export
- 71.18.3.8 Implement editor-specific smoke tests for install, auth, fetch, import, publish, replace, and project/scene integration in Unity, Unreal, and Godot
71.18.4 Cross-DCC Consistency, Enterprise Delivery & Release Operations#
- 71.18.4.1 Create shared bridge UX specification and action contract so Blender, Maya, 3ds Max, Unity, Unreal, and Godot expose consistent core workflows and terminology
- 71.18.4.2 Implement cross-DCC handoff profiles preserving materials, skeletons, animation clips, LODs, collisions, metadata, and scene-unit semantics between supported hosts
- 71.18.4.3 Create central compatibility registry mapping exporter, importer, and bridge capabilities by host, version, and target runtime
- 71.18.4.4 Implement bridge telemetry, crash/error reporting, and release health dashboards segmented by host application, version, and workflow type
- 71.18.4.5 Create per-host documentation, quickstarts, troubleshooting playbooks, migration notes, and support-runbook coverage for all supported DCC/editor bridges
- 71.18.4.6 Implement enterprise/offline plugin distribution, internal mirror support, signed artifact verification, and air-gapped update workflows
- 71.18.4.7 Create shared secrets, token-storage, and permission handling policy across all bridge hosts with least-privilege defaults and auditability
- 71.18.4.8 Implement cross-host regression benchmark suite that runs the same representative asset workflows through Blender, Maya, 3ds Max, Unity, Unreal, and Godot before release
71.19 @bellona/unity-agent — Unity MCP Server & Full Editor Parity#
Architecture: C# MCP server running inside the Unity Editor. The server exposes Unity Editor APIs as MCP tools. Any MCP client (Claude Code, custom agents) can drive Unity directly — no custom RPC protocol or NL planner needed, as the LLM handles tool selection and sequencing natively via the MCP protocol. Domain-specific capabilities (Shader Graph, VFX Graph, Animator, Terrain, Physics, UI Toolkit, Addressables, build pipeline, etc.) are implemented as MCP tool groups within the server.
Dependency:
@bellona/unity(existing adapter),@bellona/bridge-core. C# Unity package (com.bellona.agent) implements the MCP server.
71.19.1 Unity MCP Server Runtime & Editor API Control Surface#
- 71.19.1.1 Create
libs/bellona/unity-agent/project with C# MCP server package as primary deliverable, plus optional thin TypeScript MCP client wrapper for programmatic orchestration - 71.19.1.2 Implement MCP tool input validation engine that checks each tool call against the current project's render pipeline (URP vs HDRP vs Built-in), Unity version, installed packages, and target platform before execution
- 71.19.1.3 Implement semantic selection resolver as an MCP tool
(
resolve_selection) that maps natural descriptions ("all enemies with NavMeshAgent", "the main camera's post-processing volume", "every door in Building_A") to concrete GameObject/Component references using tags, layers, names, component queries, and spatial relationships - 71.19.1.4 Create current-scene context capture as an MCP resource producing a structured snapshot of: hierarchy tree, component types per object, transform world positions, prefab instance status and overrides, LOD group configuration, lighting setup, active render pipeline features, loaded scenes, and assembly definitions
- 71.19.1.5 Implement reversible transaction layer wrapping Unity's
Undo.RecordObject,Undo.RegisterCreatedObjectUndo, andUndo.DestroyObjectImmediatewith named transaction groups, rollback support, undo-stack introspection, and multi-step compound operations - 71.19.1.6 Create headless batch execution controller using MCP stdio
transport for Unity
-batchmode -executeMethodinvocations with argument passing, stdout/stderr capture, exit code handling, timeout enforcement, and parallel job scheduling for CI and farm workloads - 71.19.1.7 Implement execution audit log capturing every Editor API call, its parameters, return values, execution duration, undo group membership, and any warnings or errors emitted by Unity's console log
- 71.19.1.8 Create permission policy engine classifying each MCP tool by risk level (read-only, reversible-modify, irreversible-modify, destructive) with configurable allow/deny/prompt rules per domain and per user role
- 71.19.1.9 Implement multi-scene editing support with additive scene loading, cross-scene references, scene isolation, and coordinated save/unload operations across all open scenes
- 71.19.1.10 Create project-wide asset search and query engine as an MCP tool supporting type filters, label queries, dependency graph traversal, AssetBundle membership, and Addressable group lookups
71.19.2 C# Unity Package — MCP Server (com.bellona.agent)#
- 71.19.2.1 Create Unity Package Manager (UPM) package layout at
libs/bellona/unity-agent/plugin/com.bellona.agent/with package.json, assembly definitions (Editor and Runtime), implementing a C# MCP server with streamable-HTTP transport (live Editor) and stdio transport (batchmode/CI) - 71.19.2.2 Implement MCP tool registration system using C# attributes or a fluent API so each Editor domain handler auto-registers its tools with typed input schemas and structured results at server startup
- 71.19.2.3 Create
EditorCoroutine-based async execution engine that runs long MCP tool operations (asset imports, builds, bakes) without blocking the Editor main thread, with progress reporting via MCP notifications - 71.19.2.4 Implement scene graph serialization producing the full hierarchy, components, serialized properties, prefab overrides, and cross-references in a compact JSON schema served as an MCP resource
- 71.19.2.5 Create asset operation MCP tools for import, reimport, move,
rename, delete, duplicate, label, and dependency query using
AssetDatabaseAPIs with proper refresh and import-settings control - 71.19.2.6 Implement
SerializedObject/SerializedPropertybridge as MCP tools allowing any MCP client to read and write any serialized field on any Unity object by property path, with type safety and undo integration - 71.19.2.7 Create build pipeline MCP tools exposing
BuildPipeline.BuildPlayer,BuildPipeline.BuildAssetBundles,AddressableAssetSettings.BuildPlayerContent, and SBP callbacks with full configuration, progress, and report output - 71.19.2.8 Implement Editor preferences and ProjectSettings MCP tools
allowing any client to read/write any entry in
EditorPrefs,PlayerSettings,QualitySettings,Physics2DSettings,PhysicsSettings,TagManager,TimeManager, andAudioManager - 71.19.2.9 Create diagnostic MCP tool returning Unity Editor version, installed packages, render pipeline, scripting backend, target platform, color space, .NET API compatibility level, and assembly list
- 71.19.2.10 Implement file-watcher integration emitting MCP notifications when assets change on disk (outside agent control), compilation starts/ends, play mode toggles, or scene dirty state changes
71.19.3 Shader Graph & Material Authoring Automation (MCP tool group)#
- 71.19.3.1 Implement Shader Graph document model representing nodes,
edges, properties, keywords, sub-graphs, master stack slots, and graph
precision in a serializable TypeScript schema that maps 1:1 to Unity's
.shadergraphJSON format - 71.19.3.2 Create Shader Graph node catalog covering all built-in URP and HDRP nodes: math, texture sampling, UV, channel, procedural, normal, lighting, matrix, boolean, and custom function nodes with full port type definitions
- 71.19.3.3 Implement Shader Graph authoring from structured intents (e.g., "PBR surface with detail map, vertex displacement, and emission mask") producing valid node graphs with correct connections and default values
- 71.19.3.4 Create Shader Graph template library with production-ready presets: Standard Lit, Toon/Cel, Holographic, Glass/Refraction, Foliage (wind + subsurface), Water (caustics + flow maps), Terrain Blend, Decal, Dissolve/VFX, Eye/Skin (subsurface scattering)
- 71.19.3.5 Implement material property binding that creates Material instances from Shader Graphs and maps PBR texture sets (albedo, normal, metallic, roughness, AO, height, emission) to the correct shader properties with proper import settings
- 71.19.3.6 Create Shader Graph diff and merge engine that detects added, removed, and modified nodes/edges between two graph versions and produces a three-way merge with conflict markers for manual resolution
- 71.19.3.7 Implement sub-graph extraction and refactoring — identify reusable patterns in a shader graph and extract them into shared sub-graph assets with exposed properties
- 71.19.3.8 Create material variant workflow automating Material Variant creation, property override management, and parent-child relationship maintenance across large asset libraries
- 71.19.3.9 Implement render-pipeline-aware validation ensuring shader graphs target the correct pipeline (URP/HDRP), use only available features for the minimum target platform, and report unsupported node/feature usage
- 71.19.3.10 Create batch material generation pipeline that processes an entire PBR texture library and generates optimized materials with LOD variants, mobile fallbacks, and platform-specific overrides
71.19.4 VFX Graph & Particle System Automation (MCP tool group)#
- 71.19.4.1 Implement VFX Graph document model representing contexts
(Initialize, Update, Output), blocks, operators, properties, sub-graphs,
and GPU event connections in a serializable TypeScript schema matching
Unity's
.vfxasset format - 71.19.4.2 Create VFX Graph node catalog covering all built-in blocks and operators: spawn, initialize (position, velocity, lifetime, size, color), update (forces, turbulence, collision, conforming), output (particle, mesh, decal, strip), and attribute nodes
- 71.19.4.3 Implement VFX Graph authoring from structured intents (e.g., "fire with ember particles, heat distortion, and rising smoke trail") producing valid graphs with correct context wiring and GPU event chains
- 71.19.4.4 Create VFX Graph template library with production-ready presets: fire/flame, smoke/fog, sparks/embers, rain/snow, blood/impact, magic/energy, portal/vortex, trail/ribbon, explosion/debris, dust/sand, water splash, lightning/electricity, confetti/celebration, shield/aura, and teleport/dissolve
- 71.19.4.5 Implement legacy Particle System (Shuriken) authoring for projects not using VFX Graph, covering all modules: emission, shape, velocity, color/size over lifetime, noise, collision, sub-emitters, triggers, renderer, and external forces
- 71.19.4.6 Create VFX Graph diff and merge engine for version control of generated VFX assets with node-level change detection
- 71.19.4.7 Implement VFX performance profiling integration querying particle counts, GPU time, overdraw, and memory from the VFX Graph debugger, with automatic LOD suggestions for performance budgets
- 71.19.4.8 Create VFX event binding automation connecting VFX Graph events to gameplay triggers (animation events, collision callbacks, timeline signals, and custom C# events)
71.19.5 Animator, Animation & Timeline Automation (MCP tool group)#
- 71.19.5.1 Implement Animator Controller document model representing layers, states, transitions, parameters, blend trees, state machine behaviours, and sub-state machines in a serializable TypeScript schema
- 71.19.5.2 Create Animator Controller authoring from structured intents (e.g., "humanoid locomotion with idle, walk, run, jump, and fall states with blend trees for directional movement") producing valid state machines with transition conditions and parameter bindings
- 71.19.5.3 Implement Blend Tree generation for 1D, 2D Simple Directional, 2D Freeform Directional, 2D Freeform Cartesian, and Direct blend types with automatic threshold/position calculation from clip names
- 71.19.5.4 Create Animation Clip manipulation: retargeting between rigs, curve editing (position, rotation, scale, blend shape), event insertion, loop configuration, root motion extraction, and clip splitting
- 71.19.5.5 Implement Avatar and Avatar Mask authoring: humanoid bone mapping validation, muscle range configuration, T-pose enforcement, and avatar mask creation for layered animation
- 71.19.5.6 Create Timeline authoring from structured intents producing Timeline assets with activation tracks, animation tracks, audio tracks, signal tracks, Cinemachine tracks, control tracks, and custom playable tracks with proper binding
- 71.19.5.7 Implement Timeline clip manipulation: trimming, splitting, blending, easing, extrapolation, loop configuration, and speed multiplier adjustment with frame-accurate precision
- 71.19.5.8 Create IK (Inverse Kinematics) setup automation for Animation Rigging package: multi-aim, two-bone IK, chain IK, twist correction, damped transform, and blend constraint configuration with rig builder integration
- 71.19.5.9 Implement Animator Override Controller generation for character variants that share the same state machine but use different animation clips (e.g., different weapon hold animations)
- 71.19.5.10 Create animation event and signal pipeline automation binding animation events to MonoBehaviour methods, Timeline signals to signal receivers, and Animator state callbacks to gameplay systems
71.19.6 Terrain, Foliage & Environment Automation (MCP tool group)#
- 71.19.6.1 Implement Terrain authoring from heightmap data, procedural noise, or natural-language descriptions ("rolling hills with a river valley and cliff face") with configurable resolution, size, and heightmap bit depth
- 71.19.6.2 Create Terrain Layer authoring with automatic splatmap painting based on slope, height, curvature, and noise masks — supporting up to 16 layers with proper atlas packing and distance blending configuration
- 71.19.6.3 Implement Terrain Detail (grass, flowers, ground cover) placement with density maps, wind settings, and LOD distance configuration per detail prototype
- 71.19.6.4 Create Terrain Tree placement with density control, height and width variation, color variation, species mixing rules, and exclusion zone masking around paths, buildings, and water
- 71.19.6.5 Implement Terrain hole punching and topology editing for caves, tunnels, and building foundations with proper collision and rendering mask updates
- 71.19.6.6 Create multi-terrain tile stitching for large worlds with seamless heightmap, splatmap, and detail transitions at tile boundaries and LOD group coordination
- 71.19.6.7 Implement SpeedTree integration for procedural tree/plant generation with wind zone configuration, billboard LOD distances, and material property binding
- 71.19.6.8 Create environment probe placement automation: Reflection Probes (box/sphere, resolution, refresh mode), Light Probes (tetrahedral placement, density by importance), Light Probe Proxy Volumes for large dynamic objects, and Probe Volume (APV) configuration for HDRP
- 71.19.6.9 Implement water system authoring: ocean/river/lake setup using Unity's Water System (HDRP) or third-party packages with flow maps, foam masks, caustics, and underwater post-processing
- 71.19.6.10 Create weather and atmosphere automation: volumetric fog, clouds (HDRP), sky settings (procedural/HDRI), wind zones, and time-of-day lighting with scriptable profiles
71.19.7 Physics, Collision & Navigation Automation (MCP tool group)#
- 71.19.7.1 Implement Rigidbody and Collider configuration automation: mass estimation from mesh bounds, compound collider generation from convex decomposition, collider fitting (box, sphere, capsule, mesh, wheel), physics material assignment, and layer-based collision matrix configuration
- 71.19.7.2 Create Joint setup automation for articulated objects: HingeJoint, SpringJoint, FixedJoint, CharacterJoint, ConfigurableJoint with proper anchor placement, axis alignment, limit/drive configuration, and break force thresholds
- 71.19.7.3 Implement Ragdoll generation from humanoid avatars with bone collider fitting (capsule per limb), joint limit calibration from anatomical constraints, mass distribution, and transition blending from animated to ragdoll state
- 71.19.7.4 Create Cloth simulation setup: vertex painting for constraint weights, collision sphere/capsule definition, tether constraints, self-collision, wind interaction, and virtual particle density for quality control
- 71.19.7.5 Implement NavMesh baking automation: surface selection, agent type configuration (radius, height, step, slope), NavMesh area assignment (walkable, not walkable, jump), off-mesh link generation for jumps/drops/ladders, and NavMesh modifier volume placement
- 71.19.7.6 Create NavMeshAgent and NavMeshObstacle configuration automation: speed, angular speed, acceleration, stopping distance, auto-braking, obstacle avoidance quality, path-finding priority, and area cost overrides for AI navigation profiles
- 71.19.7.7 Implement Physics2D automation for 2D projects: Rigidbody2D, all 2D collider types (Box, Circle, Capsule, Polygon, Edge, Composite), 2D joints (Distance, Fixed, Friction, Hinge, Relative, Slider, Spring, Target, Wheel), and 2D effectors (Area, Buoyancy, Point, Surface, Platform)
- 71.19.7.8 Create physics layer matrix automation that generates optimized collision layer assignments from gameplay role definitions (player, enemy, projectile, environment, trigger, UI) minimizing unnecessary collision checks
71.19.8 UI Toolkit & Canvas Automation (MCP tool group)#
- 71.19.8.1 Implement UI Toolkit (UIElements) document authoring from structured intents producing valid UXML documents with USS stylesheets, covering all built-in controls: Label, Button, Toggle, Slider, TextField, DropdownField, Foldout, ScrollView, ListView, TreeView, ProgressBar, RadioButton, MinMaxSlider, and custom VisualElements
- 71.19.8.2 Create USS stylesheet generation from design tokens (colors, typography, spacing, border, transitions, transforms) with theme variants (dark/light), responsive breakpoints, and USS variable inheritance
- 71.19.8.3 Implement UI Toolkit data binding automation connecting UI elements to SerializedProperties, runtime data sources, and INotifyValueChanged implementations with proper binding paths and update modes
- 71.19.8.4 Create legacy Canvas/uGUI authoring for projects using the traditional UI system: Canvas configuration (Screen Space Overlay/Camera/World Space), layout groups (Horizontal, Vertical, Grid), anchoring, content size fitters, scroll rects, and event system setup
- 71.19.8.5 Implement UI screen flow generation from wireframe descriptions producing complete UI panel hierarchies with navigation logic, transition animations, and focus management for gamepad/keyboard support
- 71.19.8.6 Create accessibility automation: TextMeshPro font sizing for readability, color-blind safe palette validation, screen reader annotation (ARIA-like labels), high-contrast mode variants, and input rebinding UI generation
- 71.19.8.7 Implement localization pipeline integration: string table creation, locale-aware layout adjustments for RTL and CJK, font fallback chain configuration, and smart localization placeholder generation
- 71.19.8.8 Create HUD and in-game overlay generation: health bars, minimaps, damage indicators, crosshairs, interaction prompts, notification toasts, and dialogue boxes from gameplay system descriptions
71.19.9 Addressables, AssetBundles & Content Delivery Automation (MCP tool group)#
- 71.19.9.1 Implement Addressable Asset System configuration automation: group creation with shared/packed bundle modes, label assignment, profile configuration for local/remote build paths, and catalog settings for CDN deployment
- 71.19.9.2 Create intelligent asset group optimization that analyzes asset dependencies, load patterns, and size targets to minimize bundle count while avoiding duplicated assets across groups
- 71.19.9.3 Implement Addressables build pipeline automation:
BuildPlayerContent, update-a-previous-build, content catalog generation, hash calculation, and CDN upload manifest creation - 71.19.9.4 Create Addressable asset loading code generation producing
type-safe C# wrappers with
AsyncOperationHandlemanagement, reference counting, memory budget awareness, and preload manifest support - 71.19.9.5 Implement legacy AssetBundle workflow automation: bundle assignment, dependency analysis, variant configuration, build pipeline, manifest parsing, and cache-aware loading utilities
- 71.19.9.6 Create asset redundancy analysis detecting duplicated textures, meshes, and materials across bundles with automated refactoring suggestions to shared groups
- 71.19.9.7 Implement content update and patching workflow: catalog versioning, delta bundle generation, download size estimation, and player-facing download progress UI integration
- 71.19.9.8 Create platform-specific asset variant management for quality tiers (mobile low/high, console, PC ultra) with automatic texture compression format selection (ASTC, ETC2, BC7, PVRTC) and mesh LOD tier assignment
71.19.10 Prefab, Component & Scene Graph Authoring (MCP tool group)#
- 71.19.10.1 Implement scene graph authoring from natural-language prompts or structured descriptions producing complete Unity scene hierarchies with GameObjects, components, nested prefab instances, transform layout, and layer/tag assignment
- 71.19.10.2 Create Prefab authoring automation: nested prefab construction from component specifications, Prefab Variant creation with property overrides, added/removed components, and added child GameObjects with proper override tracking
- 71.19.10.3 Implement component configuration macros for common gameplay patterns: character controller (movement, camera, input), interactable object (highlight, prompt, action), collectible (pickup, VFX, sound), door/gate (trigger, animation, lock), checkpoint/spawn point, and waypoint path
- 71.19.10.4 Create LOD Group automation: automatic LOD level assignment from mesh complexity analysis, screen percentage threshold calibration, cross-fade width configuration, and culling behavior settings
- 71.19.10.5 Implement object pooling setup automation generating pool managers, pre-warm lists, spawn/despawn wrappers, and capacity configuration from gameplay frequency analysis
- 71.19.10.6 Create scene template system with parameterized scene archetypes (main menu, loading screen, gameplay level, boss arena, cutscene stage) that auto-populate required managers, cameras, lighting, UI, and audio listener
- 71.19.10.7 Implement cross-scene reference management: scene loading additive with dependency ordering, cross-scene object linking with stable GUIDs, and scene unload safety checks
- 71.19.10.8 Create batch prefab update pipeline that propagates component changes, material swaps, or LOD adjustments across hundreds of prefab variants while preserving per-variant overrides
71.19.11 Lighting, Rendering & Post-Processing Automation (MCP tool group)#
- 71.19.11.1 Implement lighting setup automation from natural-language scene descriptions producing Directional, Point, Spot, and Area lights with proper intensity (lumen/lux for physical), color temperature, shadow resolution, culling masks, and baked/mixed/realtime mode selection
- 71.19.11.2 Create lightmap baking automation: lightmap resolution per object, UV2 padding, ambient occlusion settings, denoiser selection (OpenImageDenoise/Optix), bake target (GPU/CPU), and progressive lightmapper configuration with quality presets
- 71.19.11.3 Implement URP Renderer Feature configuration: SSAO, screen-space shadows, decal renderer, render objects pass, fullscreen pass, and custom renderer feature injection with proper ordering and event attachment
- 71.19.11.4 Create HDRP Volume Profile authoring with all overrides: exposure (fixed/auto), bloom, chromatic aberration, color adjustments, depth of field, film grain, lens distortion, motion blur, panini projection, tonemapping, vignette, fog (volumetric/exponential), SSGI, SSR, ray-traced effects, and custom post-process injection
- 71.19.11.5 Implement Cinemachine automation: Virtual Camera creation with Body (transposer, framing, orbital), Aim (composer, POV, hard look), Noise (Perlin profiles), Extensions (confiner, collider, impulse listener), and State-Driven Camera blending rules
- 71.19.11.6 Create render pipeline migration assistant that analyzes materials, shaders, lighting, and post-processing in a Built-in RP project and generates a migration plan to URP or HDRP with compatibility warnings and automated material conversion
- 71.19.11.7 Implement baked lighting probe automation: Light Probe placement optimization based on scene geometry (dense near walls/corners, sparse in open areas), automatic probe volume partitioning, and quality verification via lightmap preview
- 71.19.11.8 Create screenshot and turntable render automation for asset review: configurable camera orbits, consistent studio lighting, background options (checkerboard, gradient, HDRI), resolution presets, and contact sheet assembly
71.19.12 Audio & Cinemachine Automation (MCP tool group)#
- 71.19.12.1 Implement Audio Source configuration automation: 3D spatial blend, rolloff curves (logarithmic, linear, custom), priority, mixer group assignment, reverb zone awareness, and occlusion settings per Audio Source
- 71.19.12.2 Create Audio Mixer graph authoring: groups, effects chains (reverb, echo, distortion, chorus, flange, SFX reverb, lowpass, highpass, parametric EQ), snapshots, duck volumes, send/return routing, and exposed parameters for runtime control
- 71.19.12.3 Implement ambient audio zone setup: reverb zone placement (size, diffusion, decay), audio trigger volumes, crossfade regions, and background music layering with smooth transitions
- 71.19.12.4 Create Cinemachine dolly track and path authoring from waypoint descriptions with smooth spline interpolation, speed ramping, easing curves, and auto-waypoint generation from level geometry
71.19.13 C# Code Generation & Scripting Automation (MCP tool group)#
- 71.19.13.1 Implement MonoBehaviour code generation from behavioral descriptions producing Unity-idiomatic C# with proper lifecycle methods (Awake, Start, OnEnable, Update, FixedUpdate, LateUpdate, OnDestroy), serialized fields, RequireComponent attributes, and ExecuteInEditMode support
- 71.19.13.2 Create ScriptableObject code generation for data containers: custom asset types with CreateAssetMenu attributes, nested data structures, OnValidate hooks, and Editor-side property drawers
- 71.19.13.3 Implement Editor script generation: custom inspectors with SerializedProperty binding, property drawers, EditorWindow panels, SceneView overlays, and menu items with shortcut key binding
- 71.19.13.4 Create component-based gameplay system scaffolding: state machines (enum-based and ScriptableObject-based), event channels (ScriptableObject events), service locators, and command pattern implementations matching Unity best practices
- 71.19.13.5 Implement Assembly Definition management: creating and updating .asmdef files, reference resolution, platform filtering, define constraints, and dependency cycle detection with refactoring suggestions
- 71.19.13.6 Create unit and integration test scaffolding generating EditMode and PlayMode test assemblies with NUnit attributes, UnityTest coroutines, scene-based test fixtures, and mock/stub generation for Unity APIs
- 71.19.13.7 Implement Roslyn analyzer integration generating
.editorconfigrules and custom analyzers for project-specific coding standards (naming conventions, null safety, Unity API usage patterns) - 71.19.13.8 Create ECS (Entities) code generation for DOTS projects: IComponentData structs, ISystem implementations, EntityQuery definitions, jobs with Burst compilation attributes, and managed/ unmanaged component type registration
71.19.14 Build Pipeline, Platform & Deployment Automation (MCP tool group)#
- 71.19.14.1 Implement multi-platform build orchestration managing sequential or parallel builds for Windows, macOS, Linux, Android, iOS, WebGL, PS5, Xbox Series X, and Nintendo Switch with per-platform PlayerSettings, quality tiers, and scripting defines
- 71.19.14.2 Create Android build automation: keystore management, Gradle template customization, ProGuard rules, AAB/APK selection, target API level, ABI filtering (ARM64, x86_64), and Play Asset Delivery configuration
- 71.19.14.3 Implement iOS build automation: Xcode project generation, entitlements, provisioning profile selection, code signing identity management, capability configuration (Push Notifications, Game Center, In-App Purchase), and bitcode/framework embedding
- 71.19.14.4 Create WebGL build automation: compression selection (Brotli, Gzip), memory size configuration, WebAssembly streaming, template customization, progressive loading, and server configuration file generation
- 71.19.14.5 Implement build pre/post-processing hooks: pre-build validation (missing references, shader errors, script compilation), post-build steps (symbol upload, build report archival, notification), and build failure diagnostics
- 71.19.14.6 Create Scriptable Build Pipeline (SBP) integration for advanced build customization: custom build tasks, content pipeline callbacks, asset processing hooks, and deterministic build output
- 71.19.14.7 Implement build size analysis and optimization: per-asset size contribution report, texture format optimization suggestions, mesh compression recommendations, unused asset detection, and dependency-aware tree shaking guidance
- 71.19.14.8 Create CI/CD pipeline templates for GitHub Actions, GitLab CI, and Jenkins generating Unity license activation, build cache management, artifact publishing, and test result reporting
71.19.15 Scene Placement, Camera Staging & Render Review (MCP tool group)#
- 71.19.15.1 Implement asset placement workflow with Physics.CheckBox/ CheckSphere collision queries, surface snapping via raycasts, grid snapping, rotation randomization, scale variation, and terrain-height conforming for large-scale scene dressing
- 71.19.15.2 Create scatter and instancing placement engine distributing prefab instances across surfaces using density maps, slope/altitude masks, exclusion zones, and random seed control with preview-before- commit and undo support
- 71.19.15.3 Implement camera staging preset library with turntable orbit, front/back/left/right/top orthographic, 3/4 perspective hero shot, Cinemachine fly-through path, and character close-up positions — each with matching studio lighting rigs (three-point, HDRI, rim)
- 71.19.15.4 Create render preview and contact sheet generation from multiple camera angles producing a composite review image with clay render, wireframe, UV checker, material ID, AO-only, and final lit passes — output as PNG sheet or individual frames
- 71.19.15.5 Implement automated turntable video capture with frame count, rotation speed, resolution, output codec (H.264, ProRes, EXR sequence), and optional depth/normal/motion vector AOV export
- 71.19.15.6 Create asset review annotation pipeline allowing the agent to flag issues (non-manifold geometry, UV seams, LOD pop, shadow artifacts) on specific frames/angles with visual markup and severity
71.19.16 Export Validation, Packaging & Library Publishing (MCP tool group)#
- 71.19.16.1 Implement FBX export validation checking unit scale (1 unit = 1 meter), coordinate system (Y-up or Z-up as target requires), mesh topology (non-manifold, degenerate triangles, flipped normals), UV overlap, bone hierarchy, and animation curve integrity before export
- 71.19.16.2 Create glTF/GLB export validation checking PBR material compliance, texture resolution/format, accessor ranges, node transform decomposition, morph target limits, and extension usage against the target runtime's supported glTF extensions
- 71.19.16.3 Implement USD export validation checking layer composition, prim schema compliance, material binding (UsdPreviewSurface or MaterialX), skeletal animation, variant set completeness, and payload reference integrity
- 71.19.16.4 Create configurable export pipeline with pre-export transforms (apply modifiers, triangulate, tangent calculation, lightmap UV generation, mesh optimization), per-format settings, and post-export hash verification
- 71.19.16.5 Implement Unity Package (.unitypackage) export workflow packaging selected assets with all dependencies, GUIDs, meta files, import settings, and preview thumbnails into distributable archives
- 71.19.16.6 Create
.unitypackagedependency analysis ensuring all referenced materials, textures, shaders, scripts, prefab parents, and animation controllers are included — flagging external references that would break on import into a different project - 71.19.16.7 Implement publish-to-Isis-library flow exporting selected GameObjects/Prefabs/Materials with auto-generated screenshots from standard camera angles, extracted metadata (poly count, material count, bounds, LOD tiers), semantic tags, and full provenance chain
- 71.19.16.8 Create batch publish pipeline processing entire asset folders with consistent naming, thumbnail generation, metadata extraction, quality gate enforcement, and catalog registration in a single operation
71.19.17 Operator Recovery, Error Handling & Agent Resilience (MCP tool group)#
- 71.19.17.1 Create error taxonomy covering all Unity Editor failure modes: C# compilation errors, asset import failures, shader compilation errors, build failures, missing reference exceptions, serialization errors, and Editor crashes — with structured codes, severity levels, and categorization
- 71.19.17.2 Implement operator suggestion engine that, when a Unity operation fails, analyzes the error context (console log, stack trace, asset state) and suggests alternative approaches, parameter adjustments, or prerequisite steps the agent should have taken
- 71.19.17.3 Create recovery hint library mapping common Unity errors to known fixes: "Shader 'X' not found" → check render pipeline, "Missing reference" → scan and rebind, "Asset import failed" → check format/ settings, with confidence scores and automated fix attempts
- 71.19.17.4 Implement progressive retry strategy with escalating intervention: first retry with adjusted parameters, then try alternative API path, then capture diagnostic bundle and surface to user with specific guidance
- 71.19.17.5 Create user confirmation prompts for low-confidence operations showing the planned action, predicted outcome, risk assessment, and alternative options — requiring explicit approval before proceeding
- 71.19.17.6 Implement domain-reload resilience ensuring the agent recovers gracefully when Unity recompiles scripts (destroying all managed state), including WebSocket reconnection, session restore, and in-progress operation replay or clean abort
71.19.18 Asset Browser, Kitbash & In-Editor Marketplace Workflows (MCP tool group)#
- 71.19.18.1 Create in-editor asset browser EditorWindow with search, type/tag/label filters, thumbnail grid view, detail panel, sort by date/name/size, and pagination for browsing personal, team, and marketplace asset libraries
- 71.19.18.2 Implement kitbash assembly workflow: select multiple library assets, place into scene with snap points, auto-connect matching connectors, resolve material conflicts, and group under a named parent with proper naming conventions
- 71.19.18.3 Create drag-and-drop import-and-place pipeline: select asset from browser, auto-import with correct settings (model, texture format, material assignment), instantiate as prefab in scene at cursor position or specified transform
- 71.19.18.4 Implement asset versioning and update workflow: detect when library assets have newer versions, show diff summary (mesh changes, material updates, new LODs), and apply updates while preserving scene- level overrides on prefab instances
- 71.19.18.5 Create asset collection management: create/edit named collections (e.g., "Medieval Props", "Sci-Fi Walls"), bulk-tag, bulk- export, and share collections across team members with access controls
- 71.19.18.6 Implement smart asset suggestion engine that analyzes current scene context (genre, style, existing assets) and recommends complementary assets from the library for scene dressing and kitbashing
71.19.19 Scene Packaging, Delivery & Diff (MCP tool group)#
- 71.19.19.1 Create scene packaging workflow producing distributable scene archives containing the scene file, all referenced assets (prefabs, materials, textures, animations, audio), lighting data, NavMesh data, and reflection probe cubemaps with dependency verification
- 71.19.19.2 Implement scene comparison and diff engine capturing before/after snapshots of full hierarchy, component serialized values, transform deltas, added/removed objects, material property changes, lighting parameter changes, and producing both human-readable reports and machine-parseable JSON diffs
- 71.19.19.3 Create play-mode automated testing: enter Play Mode via
EditorApplication.EnterPlaymode(), execute scripted gameplay sequences through Unity's Input System simulation, capture screenshots at key moments, measure frame times via Profiler API, and validate expected game state assertions — enabling end-to-end QA from the agent - 71.19.19.4 Implement scene merge assistant for multi-user workflows: detect conflicting changes to the same GameObjects/components, present side-by-side comparison, and apply selected changes with undo support
71.19.20 Addon Productization, Onboarding & In-Editor UX (MCP tool group)#
- 71.19.20.1 Create distributable Unity Package Manager (UPM) packaging with signed release artifacts via npm/GitHub Packages, scoped registry configuration, compatibility matrix (Unity 2022 LTS through Unity 6+), and rollback path via package lock
- 71.19.20.2 Implement first-run onboarding flow as an EditorWindow wizard with endpoint configuration, authentication setup, environment validation (render pipeline detection, required packages check), and sample project import
- 71.19.20.3 Create in-editor agent panel as a dockable EditorWindow with natural-language input, action preview/approval, execution log, undo history, and permission controls — the primary artist-facing UI
- 71.19.20.4 Implement resilient auth and reconnect UX for expired tokens, dropped WebSocket connections, Unity domain reload after recompilation, and Editor restart recovery with session persistence
- 71.19.20.5 Create one-click send/import workflows between Unity and Isis/Bellona for generate, fetch, update, publish, and replace operations with progress indicators and conflict resolution
- 71.19.20.6 Implement update-channel management (stable, beta, nightly) with in-editor upgrade notifications, changelog display, migration notes, and automatic backup before upgrade
- 71.19.20.7 Create diagnostics panel covering bridge health, C# compilation status, render pipeline compatibility, installed package versions, asset database integrity, and support bundle export
- 71.19.20.8 Implement guided recovery and support bundle export for failed operations including Editor.log excerpts, scene snapshot, package manifest, bridge state, and reproduction steps
71.19.21 DOTS/ECS, Netcode & Multiplayer Automation (MCP tool group)#
- 71.19.21.1 Implement DOTS ECS world setup automation: World creation, system group ordering, entity archetype definition, component data layout, and Burst-compiled job scaffolding for high-performance gameplay systems
- 71.19.21.2 Create Netcode for GameObjects setup automation: NetworkManager configuration, NetworkObject/NetworkBehaviour scaffolding, NetworkVariable declarations, RPC method generation, spawn handler registration, and network scene management
- 71.19.21.3 Implement Netcode for Entities setup automation: GhostComponent authoring, ghost snapshot configuration, client/server world separation, prediction/interpolation settings, and network protocol configuration
- 71.19.21.4 Create multiplayer lobby and matchmaking scaffold generation: Unity Lobby Service integration, relay allocation, session management, player data synchronization, and ready-check state machines
- 71.19.21.5 Implement multiplayer testing automation: simulating multiple clients in-editor, network condition emulation (latency, jitter, packet loss), and deterministic replay for debugging desync issues
71.19.22 Validation, Quality Gates & Automation Testing (MCP tool group)#
- 71.19.22.1 Implement project-wide validation rules checking for: missing script references, broken prefab connections, unresolved material/shader references, empty GameObjects, unused assets, duplicate GUIDs, and assembly definition cycles
- 71.19.22.2 Create asset quality gates: texture power-of-two validation, mesh polygon budget enforcement per LOD tier, animation clip naming conventions, audio clip format/sample rate requirements, and Addressable group size limits
- 71.19.22.3 Implement scene quality gates: draw call budget, batch count, shadow caster count, real-time light limit, UI canvas rebuild frequency, physics body count, and NavMesh coverage verification
- 71.19.22.4 Create automated visual regression testing: capture screenshots from predefined camera angles, compare against golden images with perceptual diff (SSIM, FLIP), and flag regressions above configurable thresholds
- 71.19.22.5 Implement performance budget enforcement: frame time targets (16.6ms/33.3ms), GPU time allocation per category (rendering, shadows, post-processing, particles, UI), memory budgets per platform, and automated profiler snapshot capture
- 71.19.22.6 Create build validation pipeline: successful compilation check, missing reference scan, asset bundle dependency verification, player settings sanity check, and test suite execution as release gate
- 71.19.22.7 Implement code quality gates: C# static analysis via Roslyn analyzers, naming convention enforcement, MonoBehaviour anti- pattern detection (Find in Update, allocation in hot paths, coroutine leaks), and unsafe API usage warnings
- 71.19.22.8 Create agent self-test suite that exercises every RPC command, validates round-trip serialization, confirms undo/redo integrity, and stress-tests concurrent operation handling
71.19.23 ProBuilder Mesh Editing & UV Automation (MCP tool group)#
Unity's ProBuilder is the in-editor mesh modeling tool — the direct equivalent of Blender's edit-mode mesh operations and modifier-mesh-cleanup-macros. Without this section the agent cannot create or edit geometry inside Unity.
- 71.19.23.1 Implement ProBuilder mesh creation automation: create primitives (cube, cylinder, sphere, cone, torus, stair, arch, door, pipe, prism) with configurable dimensions, segments, and smoothing groups via ProBuilder API
- 71.19.23.2 Create ProBuilder face/edge/vertex selection and manipulation: select by normal direction, material, face area, or spatial query; extrude faces, bridge edges, collapse vertices, weld, connect, subdivide, and detach faces
- 71.19.23.3 Implement ProBuilder Boolean operations (union, subtract, intersect) between ProBuilder meshes with automatic cleanup of degenerate geometry and T-vertices
- 71.19.23.4 Create ProBuilder UV editing automation: auto-UV projection (box, planar, cylindrical, spherical), manual UV island manipulation (translate, rotate, scale, flip), UV seam control, and lightmap UV generation with configurable padding
- 71.19.23.5 Implement mesh optimization macros: remove degenerate triangles, merge coplanar faces, weld nearby vertices within tolerance, recalculate normals (smooth/hard by angle threshold), flip normals, center pivot, freeze transforms, and triangulate
- 71.19.23.6 Create ProBuilder mesh-to-prefab conversion pipeline: finalize ProBuilder mesh to standard Unity Mesh, assign materials, set up colliders, configure LODs via mesh simplification, and save as prefab asset
- 71.19.23.7 Implement architectural modeling macros: create walls from spline paths, punch window/door openings, generate floors/ceilings from room outlines, create stairs with configurable rise/run, and generate railings/trim from edge selections
- 71.19.23.8 Create Polybrush integration for mesh sculpting: push/pull vertices with brush, paint vertex colors, scatter prefabs on mesh surfaces, and blend textures with vertex color painting — all via scripted automation
71.19.24 Spline System & Path Authoring (MCP tool group)#
Unity's Spline package (2022.1+) is the standard for roads, rivers, railways, cable runs, roller coasters, camera paths, and any curve-based content.
- 71.19.24.1 Implement Spline creation from waypoint lists, mathematical curves, or natural-language descriptions ("winding mountain road from A to B with two switchbacks") producing SplineContainer components with Catmull-Rom, Bezier, or Linear knot types
- 71.19.24.2 Create SplineExtrude and SplineInstantiate automation: extrude meshes along splines (roads, pipes, cables, fences) with configurable cross-section profiles, segment count, UV tiling, and cap modes; instantiate prefabs along splines with spacing, rotation, and scale curves
- 71.19.24.3 Implement SplineAnimate setup for moving objects along paths: configure speed, easing, loop mode, alignment axis, and completion callbacks for gameplay mechanics (moving platforms, patrol paths, conveyor belts)
- 71.19.24.4 Create terrain-conforming spline automation: project spline knots onto terrain surface, smooth elevation transitions, and carve terrain to match road/river elevation with configurable shoulder width and falloff
- 71.19.24.5 Implement spline-based level design tools: create guard rail/fence runs, power line catenary curves, river/stream paths with flow direction and width variation, and railway tracks with switch junctions
- 71.19.24.6 Create spline editing operations: insert/remove knots, adjust tangent handles, split/join splines, reverse direction, resample at uniform distances, and smooth/simplify with tolerance control
71.19.25 Input System & Control Mapping Automation (MCP tool group)#
Every gameplay project needs input handling. Unity's Input System package is the modern replacement for the legacy Input Manager.
- 71.19.25.1 Implement Input Action Asset generation from gameplay descriptions: create action maps (Player, UI, Vehicle, Menu), actions (Move, Look, Jump, Fire, Interact), and bindings (keyboard, gamepad, touch) with correct control types (Vector2, Button, Axis) and interaction/processor configurations
- 71.19.25.2 Create control scheme generation for multi-platform projects: Keyboard+Mouse, Gamepad, Touch, and custom schemes with proper device requirements and binding group assignments
- 71.19.25.3 Implement input binding automation for common gameplay patterns: FPS controls, third-person controls, top-down controls, RTS controls, vehicle controls, and UI navigation — each producing production-ready Input Action Assets with standard bindings
- 71.19.25.4 Create PlayerInput component setup automation: configure input action asset reference, default map, behavior (Send Messages, Invoke Unity Events, Invoke C# Events), and auto-switch scheme settings
- 71.19.25.5 Implement on-screen control generation for mobile: create on-screen sticks, buttons, and touch zones with configurable visual appearance, dead zones, and input action bindings
- 71.19.25.6 Create input rebinding UI generation: scaffold runtime
rebinding panels using Input System's
InputActionRebindingExtensionswith conflict detection, default reset, and PlayerPrefs persistence
71.19.26 2D Workflow Suite (Tilemap, Sprites, 2D Animation, 2D Lighting) (MCP tool group)#
Full 2D game development parity — without this, the agent cannot build any 2D project (platformers, top-down, visual novels, mobile games).
- 71.19.26.1 Implement Tilemap creation and painting automation: create Tilemap GameObjects with Grid components, configure cell size/ layout (Rectangle, Hexagonal, Isometric, Isometric Z-as-Y), and paint tiles from Tile Palette assets with flood fill, box fill, line, and random brush patterns
- 71.19.26.2 Create Rule Tile generation from pattern descriptions: define tiling rules (center, edge, corner, inner corner) with neighbor matching, random rotation/flip variants, and animated tiles for water, lava, and conveyor surfaces
- 71.19.26.3 Implement Sprite Atlas generation: create SpriteAtlas assets with packing settings (tight/rectangle), padding, read/write flags, max texture size, and platform overrides; add folders/sprites, configure include-in-build and late-binding for Addressables
- 71.19.26.4 Create 2D Animation (Skeletal) setup automation: create bone hierarchies from sprite anatomy descriptions, bind sprite mesh to bones, configure weight painting, create IK constraints, and generate animation clips for idle/walk/attack cycles
- 71.19.26.5 Implement Sprite Shape authoring: create open/closed spline-based shapes with angle-range sprite assignment for terrain profiles (ground, cliffs, grass edges), configure fill texture, corner sprites, and collider generation
- 71.19.26.6 Create 2D Physics setup automation: Tilemap Collider 2D with Composite Collider 2D for efficient collision, configure geometry type (outlines/polygons), platform effectors for one-way platforms, and 2D trigger zones for gameplay events
- 71.19.26.7 Implement 2D Lighting automation (URP 2D Renderer): create Global Light 2D, Point Light 2D, Spot Light 2D, and Freeform Light 2D with sorting layer targets, blend style selection, shadow caster 2D setup, and normal map integration for 2D sprites
- 71.19.26.8 Create 2D Pixel Perfect Camera setup: configure reference resolution, upscale render texture, pixel snapping, crop/ letterbox mode, and sub-pixel movement smoothing for retro-style games
71.19.27 XR/AR/VR Automation (XR Interaction Toolkit, AR Foundation) (MCP tool group)#
Extended reality is a major Unity market segment. Without XR automation the agent cannot build VR/AR/MR experiences.
- 71.19.27.1 Implement XR Rig setup automation: create XR Origin with tracked pose drivers for head and hands, configure tracking origin mode (Floor/Device), input action bindings for XR controllers, and locomotion system (teleport, continuous move, climb)
- 71.19.27.2 Create XR Interaction Toolkit component configuration: XR Direct Interactor (grab), XR Ray Interactor (point), XR Socket Interactor (snap zone), XR Grab Interactable, XR Simple Interactable, and Teleportation Areas/Anchors with haptic feedback settings
- 71.19.27.3 Implement hand tracking setup: XR Hand Tracking Subsystem configuration, hand mesh visualization, pinch/grab gesture detection, poke interaction, and custom hand pose recognition for gestures
- 71.19.27.4 Create AR Foundation setup automation: AR Session and AR Session Origin configuration, plane detection (horizontal/vertical), image tracking with reference image library, face tracking, body tracking, point cloud, and environment probe setup
- 71.19.27.5 Implement AR content placement workflow: hit-test against detected planes, instantiate AR objects with anchors, configure occlusion (environment/human), and set up light estimation for realistic AR lighting
- 71.19.27.6 Create XR UI automation: world-space Canvas attached to hand or floating panel, XR UI Input Module configuration, gaze-based and controller-based UI interaction, and curved UI surfaces for immersive menus
- 71.19.27.7 Implement XR performance optimization: foveated rendering configuration, single-pass instanced/multi-view rendering, fixed foveated rendering levels, Application SpaceWarp (ASW) settings, and per-eye resolution scaling
- 71.19.27.8 Create multi-platform XR build profiles for Meta Quest, Apple Vision Pro, PlayStation VR2, SteamVR, and Windows Mixed Reality with per-platform interaction mapping and rendering settings
71.19.28 AI/ML Integration (Sentis, ML-Agents, Behavior Trees) (MCP tool group)#
Unity's AI/ML ecosystem for in-game inference and NPC behavior.
- 71.19.28.1 Implement Unity Sentis model loading and inference setup: import ONNX models, configure execution backend (CPU, GPU Compute, Pixel Shader), create inference workers, bind input/output tensors, and manage model lifecycle with memory budgets
- 71.19.28.2 Create Sentis inference pipeline for common use cases: image classification (item recognition), pose estimation (character animation), NLP (dialogue response selection), style transfer (runtime texture modification), and audio classification (voice commands)
- 71.19.28.3 Implement ML-Agents training environment setup: create Agent components with observation/action specifications, configure reward signals, set up training configuration YAML, and generate curriculum definitions for progressive difficulty
- 71.19.28.4 Create ML-Agents inference setup for trained policies: import trained ONNX models, configure Barracuda/Sentis inference, set up observation collection (ray perception, vector, visual), and create decision requester with interval configuration
- 71.19.28.5 Implement behavior tree scaffolding for AI characters: create behavior tree assets with composite nodes (selector, sequence, parallel), decorator nodes (inverter, cooldown, retry), and action nodes (move-to, attack, patrol, flee, idle) with blackboard variables
- 71.19.28.6 Create NavMesh-integrated AI patrol and combat behavior setup: waypoint routes, area awareness, sight/hearing detection zones, cover point evaluation, and group coordination tactics
71.19.29 USD & Industry Interchange (MCP tool group)#
Universal Scene Description support for professional DCC interchange.
- 71.19.29.1 Implement USD import pipeline automation: load .usd/.usda/ .usdc/.usdz files via Unity USD package, map USD prim hierarchy to Unity GameObject hierarchy, convert UsdPreviewSurface materials to URP/HDRP materials, import skeletal animation (UsdSkel), and resolve payload/reference composition
- 71.19.29.2 Create USD export pipeline: serialize Unity scene to USD with correct coordinate system and unit conversion, export meshes with subdivision scheme hints, materials as UsdPreviewSurface or MaterialX, animation clips as UsdSkel, and camera/light properties
- 71.19.29.3 Implement USD variant set management: enumerate and switch variant sets for LOD, material, and assembly variations within imported USD assets, and expose variant selection as agent-controllable parameters
- 71.19.29.4 Create USD round-trip validation: export a scene to USD, re-import, and diff against the original — checking mesh topology, material properties, transform precision, animation curves, and hierarchy structure to verify lossless interchange
- 71.19.29.5 Implement USDZ AR Quick Look export for Apple ecosystem: package scene with materials, textures, and animations into .usdz with Apple-compatible UsdPreviewSurface materials, object anchoring metadata, and AR interaction hints
- 71.19.29.6 Create cross-DCC USD workflow: receive USD stage from Blender/Maya/Houdini pipeline, apply Unity-specific overrides (render pipeline materials, LOD, colliders), and publish back as USD with Unity opinion layer — enabling non-destructive round-trip
71.20 @isis/3dgs-diffusion-editing — Text & Reference-Driven 3DGS Scene Editing#
While Phase 71.11 provides part-aware segmentation and local editing, and Phase 40.23.1.12 provides basic 4DGS editing tools, this section covers DIFFUSION-GUIDED 3DGS editing where text prompts or reference images drive multi-view-consistent changes to existing Gaussian splat scenes. GaussCtrl (CVPR 2024-2025) propagates text-guided edits across all views. InterGSEdit (ICCV 2025) provides interactive 3D editing with geometry-consistent attention. CTRL-D (CVPR 2025) extends to dynamic (time-varying) 3DGS scenes.
71.20.1 Multi-View Consistent Text-Driven Editing (8 tasks)#
- 71.20.1.1 Implement GaussCtrl model integration for multi-view consistent text-driven 3DGS editing — edits propagated across all views maintaining 3D consistency
- 71.20.1.2 Create GaussCtrl text-to-edit pipeline: existing 3DGS scene + text instruction ("make the building facade gothic", "add snow to the ground") → edited scene with multi-view coherence
- 71.20.1.3 Implement depth-conditioned ControlNet guidance for GaussCtrl: extract depth from 3DGS → use as structural constraint during per-view editing
- 71.20.1.4 Create InterGSEdit interactive 3D editing with 3D geometry-consistent attention: user-interactive interface where users select regions in 3D viewport → apply text-guided edits with geometric consistency enforcement
- 71.20.1.5 Implement CTRL-D controllable dynamic 3DGS scene editing: apply text or reference-image guided edits to time-varying 4DGS scenes (animated content editing)
- 71.20.1.6 Create CTRL-D personalized 2D diffusion priors: fine-tune diffusion model on the specific 3DGS scene's rendered views for scene-specific editing quality
- 71.20.1.7 Implement Morpheus text-driven shape and color stylization: global style transfer applied to 3DGS scene ("make everything low-poly", "apply watercolor style") with consistent 3D appearance
- 71.20.1.8 Create SyncNoise geometrically consistent noise prediction for text-based editing: shared noise across views ensures multi-view coherent edits
71.20.2 Reference-Based 3DGS Editing & Composition (6 tasks)#
- 71.20.2.1 Implement reference-image-guided 3DGS editing: provide reference image of desired style/appearance → transfer to specific region of 3DGS scene
- 71.20.2.2 Create 3DGS object insertion from reference: extract object from reference image → insert into 3DGS scene at specified position with correct scale, orientation, lighting
- 71.20.2.3 Implement 3DGS object removal with diffusion inpainting: remove specified Gaussians → inpaint gap using multi-view consistent diffusion
- 71.20.2.4 Create 3DGS material editing: change material properties of selected region (e.g., "make the wall brick" → regenerate texture for selected surface Gaussians)
- 71.20.2.5 Implement 3DGS editing quality validator: render novel views of edited scene → check multi-view consistency (epipolar error), depth consistency, and appearance coherence
- 71.20.2.6 Create 3DGS editing undo/history: track per-Gaussian edit operations → allow selective revert of specific edits while preserving others
71.21 @isis/3dgs-relighting-pbr — Material-Decomposed 3DGS Relighting#
While Phase 40.23.1.13 provides basic 4DGS relighting, this section covers MATERIAL-DECOMPOSED per-Gaussian relighting where each Gaussian stores precomputed material properties (diffuse, specular, transmission, roughness) enabling real-time relighting via dot-product with target lighting. LumiGauss (WACV 2025) achieves real-time outdoor relighting by precomputing per-Gaussian transfer coefficients. Relightable Gaussian models decompose Gaussians into physically-based components.
71.21.1 LumiGauss & Per-Gaussian Material Decomposition (8 tasks)#
- 71.21.1.1 Implement LumiGauss model loader for 2D/3D Gaussian Splatting with per-Gaussian material and transfer coefficient buffers
- 71.21.1.2 Create LumiGauss training pipeline: multi-view images with varying lighting → decompose each Gaussian into diffuse albedo, specular, roughness, and lighting transfer coefficients
- 71.21.1.3 Implement LumiGauss real-time relighting inference: precomputed material buffers × target environment map = relit image via dot products + splatting (no diffusion at render time)
- 71.21.1.4 Create environment map relighting interface: load HDRI environment map → apply to LumiGauss scene → real-time interactive preview
- 71.21.1.5 Implement per-Gaussian PBR material editing: select region of Gaussians → modify material properties (increase roughness, change albedo, add metallic) → instant re-render
- 71.21.1.6 Create Relightable Gaussian decomposition pipeline: separate each Gaussian into diffuse/specular/transmission components for physics-based rendering
- 71.21.1.7 Implement shadow and ambient occlusion estimation from decomposed Gaussians: compute screen-space or ray-traced shadows from estimated geometry
- 71.21.1.8 Create LumiGauss export to standard PBR formats: extract material maps (albedo, normal, roughness, metallic) from per-Gaussian properties for use in external renderers (Blender, Unreal, Unity)
71.21.2 Dynamic Relighting & Integration (6 tasks)#
- 71.21.2.1 Implement animated lighting on static 3DGS scenes: keyframe environment map rotation/intensity over time → render video of scene under changing lighting conditions
- 71.21.2.2 Create LumiGauss integration with 4DGS: apply material decomposition to time-varying Gaussian fields for relightable animated content
- 71.21.2.3 Implement multi-light-source decomposition: separate contributions of individual light sources for independent control (turn off/on specific lights, adjust individual intensities)
- 71.21.2.4 Create LumiGauss quality comparison against traditional rendering: compare relighting accuracy vs ground-truth renders from Blender/Unreal for same scene
- 71.21.2.5 Implement LumiGauss integration with @isis/3d-browser: browser-based real-time relighting of 3DGS scenes using WebGL with precomputed buffers
- 71.21.2.6 Create LumiGauss relighting quality metrics: angular error of estimated normals, albedo accuracy (sRGB error), relighting PSNR/SSIM against ground truth
71.22 @isis/mesh-transformers — Autoregressive Mesh Generation (MeshGPT/PolyGen)#
Reference: Siddiqui et al. "MeshGPT" (CVPR 2024), Nash et al. "PolyGen" (ICML 2020), Chen et al. "MeshXL" (2024). Mesh transformers generate topology directly as token sequences — producing clean artist-quality meshes with proper edge flow, quad-dominant topology, and meaningful face groupings. Unlike marching cubes extraction from implicit fields, these generate mesh structure that is immediately usable for animation and editing.
71.22.1 MeshGPT Integration & Training#
- 71.22.1.1 Create
libs/isis/mesh-transformers/library with MeshGPT model architecture — VQ-VAE mesh tokenizer (encodes face triplets as codebook tokens) + autoregressive transformer decoder generating token sequences - 71.22.1.2 Implement mesh vocabulary learning — train VQ-VAE on Objaverse dataset to learn codebook of ~8192 mesh face tokens encoding local geometry patterns
- 71.22.1.3 Create text-conditioned mesh generation — condition MeshGPT transformer on CLIP text embeddings for text-to-mesh generation with proper topology
- 71.22.1.4 Implement image-conditioned mesh generation — condition on DINOv2 image features for single-image-to-mesh with clean edge flow
- 71.22.1.5 Create mesh completion — given partial mesh (face token prefix), autoregressively generate remaining faces to complete the shape
- 71.22.1.6 Implement topology-aware sampling — nucleus/top-k sampling with manifoldness constraints ensuring generated meshes are watertight and non-self-intersecting
- 71.22.1.7 Create MeshGPT fine-tuning pipeline — fine-tune on domain- specific mesh datasets (architecture, vehicles, characters, furniture) for category-specialized generation
- 71.22.1.8 Implement MeshGPT quantitative evaluation — compare topology quality (quad ratio, edge flow smoothness, genus correctness) against FlexiCubes/marching cubes extraction on same shapes
71.22.2 PolyGen & MeshXL Variants#
- 71.22.2.1 Implement PolyGen vertex-then-face architecture — first generate vertex positions autoregressively, then generate face connectivity conditioned on vertices
- 71.22.2.2 Create MeshXL large-scale mesh generation — implement MeshXL's neural coordinate codec with 3-level residual tokenization for higher-resolution meshes (up to 8k faces)
- 71.22.2.3 Implement mesh token vocabulary expansion — extend codebook to encode UV coordinates and material IDs alongside geometry for texture-ready mesh generation
- 71.22.2.4 Create multi-resolution mesh generation — generate coarse mesh first → refine with subdivision-aware token prediction for progressive detail
- 71.22.2.5 Implement mesh transformer serving infrastructure — ONNX export of trained models, batched inference, KV-cache optimization for fast autoregressive generation
- 71.22.2.6 Create mesh transformer A/B comparison UI — side-by-side comparison of mesh transformer output vs FlexiCubes vs TRELLIS for same prompt, with topology quality metrics overlay
71.23 @isis/score-distillation — Text-to-3D via Score Distillation (SDS/VSD)#
Reference: Poole et al. "DreamFusion" (ICLR 2023), Wang et al. "ProlificDreamer" (NeurIPS 2023), Lin et al. "Magic3D" (CVPR 2023). Score distillation sampling lifts 2D diffusion model knowledge into 3D by optimizing a 3D representation so that rendered views score well under a pretrained 2D diffusion model. This enables text-to-3D without any paired 3D training data.
71.23.1 Score Distillation Sampling (SDS)#
- 71.23.1.1 Create
libs/isis/score-distillation/library implementing SDS loss — render 3D representation from random camera → add noise → denoise with pretrained diffusion model → compute SDS gradient to update 3D parameters - 71.23.1.2 Implement NeRF-based SDS optimization — optimize neural radiance field using SDS loss from Stable Diffusion, with coarse-to-fine training schedule (64→256 render resolution)
- 71.23.1.3 Create 3DGS-based SDS optimization — optimize Gaussian splatting representation via SDS for faster convergence and explicit geometry (DreamGaussian approach)
- 71.23.1.4 Implement FlexiCubes-based SDS — optimize differentiable mesh via SDS loss for direct mesh output (Magic3D stage 2 approach)
- 71.23.1.5 Create view-dependent prompt augmentation — augment text prompt with view direction ("front view of...", "side view of...") to reduce multi-face Janus artifacts
- 71.23.1.6 Implement negative SDS guidance — classifier-free guidance with negative prompts to steer away from common SDS failure modes (oversaturation, smoothness, Janus faces)
- 71.23.1.7 Create SDS camera scheduling — annealed camera distribution starting from elevation-biased views → uniform hemisphere for balanced geometry optimization
- 71.23.1.8 Implement SDS quality metrics — measure multi-view consistency (render from novel views → CLIP similarity to prompt), geometry quality (surface smoothness, genus), and texture quality (FID against text-matched images)
71.23.2 Variational Score Distillation (VSD) & Advanced Methods#
- 71.23.2.1 Implement VSD (ProlificDreamer) — train per-scene LoRA on diffusion model to approximate score of rendered image distribution, replacing SDS single-sample gradient with variational bound for sharper results
- 71.23.2.2 Create ISM (Interval Score Matching) — compute score difference between two noise levels instead of single-step SDS for reduced variance and better color fidelity
- 71.23.2.3 Implement two-stage coarse-to-fine pipeline — Stage 1: NeRF + SDS at 64px for coarse shape → Stage 2: mesh + VSD at 512px for high- detail texture (Magic3D approach)
- 71.23.2.4 Create SDS with 3D-aware diffusion prior — replace generic Stable Diffusion with multi-view diffusion model (Zero123++, SV3D) as score function for geometrically consistent distillation
- 71.23.2.5 Implement SDS + mesh transformer hybrid — use mesh transformer for initial topology → SDS for texture and geometric detail refinement
- 71.23.2.6 Create score distillation benchmark — compare SDS vs VSD vs ISM vs direct multi-view generation (TRELLIS) on Objaverse prompts for quality, speed, and failure rate
71.24 @isis/fast-generation — Flow Matching & Consistency Models for 3D#
Reference: Lipman et al. "Flow Matching" (ICLR 2023), Song et al. "Consistency Models" (ICML 2023), Esser et al. "Scaling Rectified Flow Transformers" (Stable Diffusion 3). Flow matching and consistency models achieve diffusion-quality generation in 1-4 steps instead of 20-50 — critical for interactive 3D generation workflows.
71.24.1 Flow Matching for 3D Generation#
- 71.24.1.1 Create
libs/isis/fast-generation/library with rectified flow training — learn straight ODE paths from noise to data distribution for single-step generation capability - 71.24.1.2 Implement flow matching for multi-view image generation — train rectified flow model to generate 6-view consistent images from text in 1-4 NFE (neural function evaluations)
- 71.24.1.3 Create flow matching for 3D latent generation — operate in TRELLIS-style 3D latent space with rectified flow for fast 3D shape generation
- 71.24.1.4 Implement flow matching texture generation — generate PBR texture maps (albedo, normal, roughness) via rectified flow conditioned on geometry for 4-step texture synthesis
- 71.24.1.5 Create velocity field visualization — render ODE velocity field for debugging and understanding generation dynamics
- 71.24.1.6 Implement adaptive step scheduling — use error estimator to dynamically choose NFE count per sample (simple shapes = 1 step, complex = 4 steps) for optimal speed/quality tradeoff
71.24.2 Consistency Models for Interactive 3D#
- 71.24.2.1 Implement consistency distillation — distill pretrained diffusion model into consistency model that maps any noise level directly to clean sample in single forward pass
- 71.24.2.2 Create latent consistency model (LCM) for multi-view — distill multi-view diffusion model to 1-2 step generation for real-time 3D preview during text editing
- 71.24.2.3 Implement consistency model fine-tuning — fine-tune consistency model on domain-specific 3D datasets while preserving single- step capability
- 71.24.2.4 Create interactive generation UI — user types text → see 3D preview update in <1 second via consistency model → confirm → run full-quality generation with flow matching or diffusion
- 71.24.2.5 Implement generation speed benchmark — compare wall-clock time for equivalent quality: consistency model (1 step) vs flow matching (4 steps) vs diffusion (20 steps) vs TRELLIS across GPU tiers
- 71.24.2.6 Create consistency model + mesh transformer pipeline — fast multi-view generation via consistency model → feed to mesh transformer for topology-aware reconstruction in <5 seconds total
71.25 @isis/3d-control — ControlNet & IP-Adapter for 3D Generation#
Reference: Zhang et al. "ControlNet" (ICCV 2023), Ye et al. "IP-Adapter" (2023). These provide fine-grained spatial and style control over generation — critical gaps in the current 3D pipeline which relies primarily on text prompting. ControlNet-style conditioning enables depth/edge/pose-guided 3D generation; IP-Adapter enables reference-image style injection.
71.25.1 Spatial Control for 3D Generation#
- 71.25.1.1 Create
libs/isis/3d-control/library with multi-view ControlNet — inject depth maps, edge maps, normal maps, or semantic segmentation as spatial conditions into multi-view diffusion model - 71.25.1.2 Implement depth-conditioned 3D generation — user provides rough depth sketch → ControlNet guides multi-view generation to match specified depth profile → reconstruct 3D mesh
- 71.25.1.3 Create edge/canny-conditioned 3D generation — from edge drawing of desired object silhouette, generate 3D model matching the outline from specified viewpoint
- 71.25.1.4 Implement pose-conditioned character generation — from OpenPose skeleton, generate 3D character matching the specified pose with correct anatomy
- 71.25.1.5 Create semantic-map-conditioned scene generation — from colored semantic segmentation (red=building, green=tree, blue=road), generate 3D scene matching the layout
- 71.25.1.6 Implement multi-condition fusion — combine depth + edge + semantic conditions simultaneously with per-condition strength weights for precise control
- 71.25.1.7 Create sketch-to-3D interactive workflow — user draws 2D sketch strokes → edge ControlNet generates views → reconstruct 3D in real-time preview loop
- 71.25.1.8 Implement 3D ControlNet training pipeline — fine-tune multi-view diffusion model with ControlNet adapters on paired (condition, multi-view) datasets rendered from Objaverse
71.25.2 Style & Reference Control (IP-Adapter for 3D)#
- 71.25.2.1 Implement IP-Adapter for 3D — inject reference image features (via CLIP/DINOv2 image encoder) into multi-view diffusion cross-attention layers for style-guided 3D generation
- 71.25.2.2 Create style transfer for 3D assets — given reference style image (art style, material appearance), generate 3D asset with matching visual style while following text prompt for shape
- 71.25.2.3 Implement multi-reference blending — combine features from multiple reference images with per-reference weights for hybrid style generation (e.g., 70% reference A shape + 30% reference B texture)
- 71.25.2.4 Create IP-Adapter + ControlNet composition — simultaneously control spatial layout (via depth/edge ControlNet) and visual style (via IP-Adapter reference image)
- 71.25.2.5 Implement face/identity preservation for characters — inject face identity features (via InsightFace/ArcFace) to generate 3D characters preserving specific face identity
- 71.25.2.6 Create IP-Adapter LoRA variants — train lightweight LoRA adapters for specific style domains (anime, photorealistic, low-poly, stylized) that compose with IP-Adapter
71.26 @isis/text-mesh-editing — Text-Guided Mesh Editing#
Reference: Sella et al. "Vox-E" (ICCV 2023), Kamata et al. "Instruct- NeRF2NeRF" (ICCV 2023), Haque et al. "Instruct 3D-to-3D" (2023). Text-guided mesh editing enables modifying existing 3D assets via natural language instructions — "make the building taller", "add damage to the left wall", "change the material to wood". Currently absent from the pipeline.
71.26.1 Text-Driven 3D Editing#
- 71.26.1.1 Create
libs/isis/text-mesh-editing/library with InstructPix2Pix- based 3D editing — render existing mesh from multiple views → apply InstructPix2Pix edit to each view → reconstruct edited mesh via multi-view consistency optimization - 71.26.1.2 Implement SDS-guided mesh deformation — optimize mesh vertex positions using SDS loss conditioned on edit instruction (e.g., "make taller" → SDS pushes vertices upward until diffusion model is satisfied)
- 71.26.1.3 Create localized mesh editing — user specifies edit region via 3D bounding box or semantic selection ("the roof") → edit applies only to selected region, preserving rest of mesh
- 71.26.1.4 Implement texture-only editing — text instruction modifies only texture/material ("make it rusty", "paint it red") without changing geometry, using SDS on texture map only
- 71.26.1.5 Create geometry-only editing — text instruction modifies shape without changing texture style ("add a chimney", "make it round"), using SDS on vertex positions with texture frozen
- 71.26.1.6 Implement sequential edit chain — apply multiple edits in sequence ("add a door" → "make it wooden" → "add wear marks") with each edit building on previous result
- 71.26.1.7 Create edit preview and undo — fast low-resolution preview of proposed edit → user confirms → full-resolution edit, with per-edit undo stack
- 71.26.1.8 Implement edit transfer — apply same edit instruction to batch of similar meshes (e.g., "add battle damage" to all building assets in a set) with per-mesh variation
71.27 @isis/3d-video-diffusion — 3D Generation from Video Diffusion Models#
Reference: Voleti et al. "SV3D" (2024), Shi et al. "Zero123++" (2023), Liu et al. "SyncDreamer" (ICLR 2024). Video diffusion models can generate multi-view consistent image sequences by treating orbital camera motion as video — enabling 3D reconstruction without explicit 3D training data.
71.27.1 Video-to-3D Pipeline#
- 71.27.1.1 Create
libs/isis/3d-video-diffusion/library with SV3D integration — condition Stable Video Diffusion on single image + camera orbit trajectory → generate multi-view consistent frames → reconstruct 3D via large reconstruction model - 71.27.1.2 Implement Zero123++ multi-view generation — generate 6 canonical views (front, back, left, right, top, 3/4) from single input image using fine-tuned diffusion model with camera-conditioned cross- attention
- 71.27.1.3 Create SyncDreamer synchronized multi-view — generate 16 views simultaneously with 3D-aware attention ensuring geometric consistency across all views
- 71.27.1.4 Implement video-to-3D reconstruction — from generated multi- view frames, run dense stereo matching + Poisson surface reconstruction or feed to large reconstruction model (LRM/InstantMesh)
- 71.27.1.5 Create camera trajectory customization — user specifies custom camera orbit (elevation, azimuth range, distance) for controlling which parts of the object are emphasized in generation
- 71.27.1.6 Implement video diffusion + mesh transformer hybrid — generate multi-views via video diffusion → extract topology via mesh transformer → refine texture via multi-view projection
- 71.27.1.7 Create 4D generation via video diffusion — generate temporal sequence of orbital views → reconstruct 4D dynamic content (animated 3D objects) from video diffusion output
- 71.27.1.8 Implement video diffusion quality comparison — benchmark SV3D vs Zero123++ vs SyncDreamer vs direct multi-view models (MVDream) for 3D quality, multi-view consistency, and generation diversity
71.28 @isis/hash-encoding — Multi-Resolution Hash Encoding for Neural Fields#
Reference: Müller et al. "Instant Neural Graphics Primitives" (SIGGRAPH 2022). Multi-resolution hash encoding enables real-time neural field queries by replacing MLP position encoding with learned hash table lookups at multiple resolutions. Critical for interactive neural 3D generation and editing.
71.28.1 Hash Encoding Implementation#
- 71.28.1.1 Create
libs/isis/hash-encoding/library with multi-resolution hash grid — L levels of hash tables (T entries each, F features per entry) with trilinear interpolation at each resolution - 71.28.1.2 Implement GPU-optimized hash table — fully fused CUDA/WebGPU kernel for hash lookup + trilinear interpolation + MLP forward pass in single kernel launch (<1ms for 1M query points)
- 71.28.1.3 Create Instant-NGP NeRF training — train NeRF with hash encoding in <5 minutes on single GPU (vs hours for original NeRF) with equivalent quality
- 71.28.1.4 Implement hash-encoded SDF — use hash encoding for neural signed distance field with real-time sphere tracing for interactive implicit surface editing
- 71.28.1.5 Create hash-encoded appearance field — separate hash grids for geometry (SDF/density) and appearance (color/material) enabling independent editing of shape and texture
- 71.28.1.6 Implement hash encoding for 3DGS initialization — use hash- encoded density field to initialize Gaussian positions and scales for faster 3DGS training convergence
- 71.28.1.7 Create hash encoding export — convert trained hash grid to standard mesh (via marching cubes on hash-encoded SDF) or standard texture (via sampling hash-encoded appearance) for DCC compatibility
- 71.28.1.8 Implement hash encoding streaming — load hash table levels on demand based on camera distance for LOD-aware neural field rendering
71.29 @isis/quad-mesh-gen — Native Quad Mesh Generation (QuadGPT/TopGen)#
Reference: QuadGPT (September 2025) — first autoregressive model generating native quad-dominant meshes end-to-end via unified tokenization for mixed tri/quad topologies with RL fine-tuning via tDPO. TopGen (March 2026) — predicts structural layouts and cross-fields for quad mesh generation mimicking professional "layout-first" workflow. These solve the industry's most critical mesh quality gap: no prior model could generate quad meshes natively.
71.29.1 Quad-Dominant Mesh Generation#
- 71.29.1.1 Create
libs/isis/quad-mesh-gen/library with QuadGPT integration — autoregressive transformer generating mixed tri/quad face sequences with unified tokenization that preserves quad dominance - 71.29.1.2 Implement QuadGPT RL fine-tuning pipeline — truncated Direct Preference Optimization (tDPO) with topological reward measuring edge flow quality, quad ratio, and manifoldness
- 71.29.1.3 Create TopGen layout-first generation — dual-query decoder predicting structural layout lines and cross-field orientation before face generation, producing meshes with rational edge flow matching professional artist workflow
- 71.29.1.4 Implement face-count conditioned generation — user specifies exact polygon budget → model generates mesh with that face count (MeshCraft approach: parallel diffusion DiT generating all faces simultaneously in 3.2 seconds for 800 faces, 35x faster than autoregressive)
- 71.29.1.5 Create quad mesh topology validation — measure quad ratio, valence distribution (ideal: all valence-4), edge flow smoothness, and manifold correctness of generated quad meshes
- 71.29.1.6 Implement quad mesh comparison benchmark — compare QuadGPT vs TopGen vs MeshAnything V2 vs FlexiCubes+instant-meshes remesh for topology quality, generation speed, and downstream animation deformation quality
- 71.29.1.7 Create game-ready pipeline integration — quad mesh generation → automatic UV seam prediction (Hunyuan3D Studio approach) → PBR texture synthesis → engine-compatible export (FBX/glTF)
- 71.29.1.8 Implement quad mesh LOD chain generation — from high-poly quad mesh, generate progressive LOD chain preserving edge flow topology at each level via edge collapse ordering
71.30 @isis/part-level-3d — Part-Level & Multi-Instance 3D Generation#
Reference: PartGen (CVPR 2025, Meta AI) — generates 3D objects composed of meaningful, independently manipulable parts via multi-view diffusion for part segmentation + occlusion completion. MIDI (CVPR 2025, Tripo AI) — multi- instance attention mechanism for generating multiple 3D objects with accurate spatial relationships simultaneously. SAMPart3D/SegviGen (2025-2026) — zero- shot multi-granularity 3D part segmentation.
71.30.1 Part-Aware & Scene-Level Generation#
- 71.30.1.1 Create
libs/isis/part-level-3d/library with PartGen integration — generate 3D objects decomposed into semantically meaningful parts (body, arms, legs for characters; doors, wheels, chassis for vehicles) each independently manipulable - 71.30.1.2 Implement per-part material assignment — each generated part gets independent PBR material, enabling mix-and-match part composition (swap character head between bodies, swap vehicle wheels)
- 71.30.1.3 Create MIDI multi-instance generation — generate complete 3D scenes with multiple objects simultaneously via multi-instance attention, preserving spatial relationships (table + chairs + plates arranged correctly)
- 71.30.1.4 Implement SAMPart3D zero-shot part segmentation — segment any existing 3D asset into parts at user-specified granularity without predefined labels, enabling "edit just the arm" or "recolor just the roof"
- 71.30.1.5 Create part-level editing pipeline — segment existing mesh → select part → apply text-guided edit (texture, shape, or replacement) to selected part only, preserving remainder
- 71.30.1.6 Implement part-aware kitbash assembly — recombine parts from different generated assets: take wheels from asset A, body from asset B, wings from asset C → generate coherent composite
- 71.30.1.7 Create scene composition from text — "a medieval tavern with tables, chairs, mugs, and a fireplace" → generate each object separately → arrange in scene with learned spatial priors (MIDI + Procedural Scene Programs approach)
- 71.30.1.8 Implement part generation quality metrics — measure part boundary cleanness, part semantic correctness (does "wheel" actually look like a wheel), inter-part spatial coherence, and part reusability across compositions
71.31 @isis/gigascale-3d — Gigascale & Next-Gen 3D Model Integration#
Reference: Direct3D-S2 (NeurIPS 2025) — sparse volumes with Spatial Sparse Attention at 1024^3 resolution on 8 GPUs (3.9x forward / 9.6x backward speedup). SAR3D (CVPR 2025) — multi-scale VQVAE autoregressive 3D in 0.82s. Hunyuan3D 2.5 (June 2025) — 1024 geometric resolution, 10x effective facet count. Rodin Gen-2 (October 2025) — 10B params, BANG architecture, recursive part-based generation. TRELLIS.2 (December 2025) — 4B params, O-Voxel representation, MIT license.
71.31.1 Latest Model Integration#
- 71.31.1.1 Integrate Hunyuan3D 2.5 — upgrade from 2.1 to 2.5: 1024 geometric resolution (2x over 2.1), 10x facet count increase, 25% latency reduction (8-20 seconds on A100), Apache 2.0 license
- 71.31.1.2 Integrate Rodin Gen-2 via API — 10B parameter model with BANG architecture: recursive part decomposition, 4x mesh quality over Gen-1, native seamless 3D textures, bounding box/voxel/point cloud control inputs
- 71.31.1.3 Integrate TRELLIS.2 — 4B parameter model with field-free O-Voxel sparse structure, flow-matching transformers, full PBR including transparency, MIT license — benchmark against Hunyuan3D 2.5 for quality and speed
- 71.31.1.4 Integrate Direct3D-S2 gigascale generation — sparse volumetric generation at 1024^3 resolution using Spatial Sparse Attention, unified sparse format across VAE stages, for highest-detail 3D generation
- 71.31.1.5 Integrate SAR3D autoregressive generation — multi-scale 3D VQVAE tokenization for 0.82-second generation as fast-preview alternative to diffusion-based models
- 71.31.1.6 Create model routing policy update — extend intelligent routing (§71.1) with Hunyuan3D 2.5, Rodin Gen-2, TRELLIS.2, Direct3D-S2, SAR3D based on quality requirements (resolution, part-awareness, speed), licensing constraints, and GPU tier
- 71.31.1.7 Create CHORD PBR decomposition pipeline — integrate Ubisoft's Chain of Rendering Decomposition (SIGGRAPH Asia 2025): from generated texture image, sequentially predict SVBRDF channels (albedo → normal → height → roughness → metalness) via single-step conditioned diffusion
- 71.31.1.8 Implement SV4D 2.0 dynamic generation — integrate Stability AI's updated video-to-4D pipeline: more robust to occlusions and large motion, 3D attention fusing spatial/temporal features, permissive license
71.32 @isis/3dgs-advanced — Advanced Gaussian Splatting (GI, Reflections, Editing)#
Reference: GI-GS (ICLR 2025) — global illumination via efficient path tracing on 3DGS with deferred shading. Reflective GS (ICLR 2025) — physically-based inter-reflections. SplatPainter (December 2025) — interactive-rate 3DGS editing. MILo (2025) — mesh-in-the-loop differentiable extraction. These address the gap between raw 3DGS capture and production-ready assets.
71.32.1 GS Relighting & Production Pipeline#
- 71.32.1.1 Implement GI-GS global illumination — integrate efficient path tracing with deferred shading on 3DGS for accurate material/lighting decomposition with real-time relighting under novel illumination
- 71.32.1.2 Create Reflective GS integration — physically-based deferred rendering with Gaussian-grounded inter-reflection, enabling correct mirror, glass, and metallic surface rendering in GS scenes
- 71.32.1.3 Implement MILo mesh-in-the-loop extraction — differentiable mesh extraction that encourages Gaussians to align with mesh surfaces during training, producing higher-quality meshes than post-hoc extraction
- 71.32.1.4 Create SplatPainter interactive editing — compact voxel representation + test-time training for interactive-rate GS editing: painting, graffiti, relighting, super-resolution on large-scale GS scenes
- 71.32.1.5 Implement confidence-based mesh extraction — self-supervised confidence values balancing photometric and geometric supervision for reliable GS-to-mesh conversion (March 2026 method)
- 71.32.1.6 Create GS editing benchmark — compare GaussianEditor vs SplatPainter vs InterGSEdit vs text-guided editing (§71.26) for edit quality, speed, multi-view consistency, and production usability
Phase 71 Summary Statistics#
| Section | Task Count |
|---|---|
| 71.1 Self-Hosted 3D Model Inference | 85 |
| 71.2 AI-Native PBR Texture Generation | 33 |
| 71.3 Universal Auto-Rigging | 47 |
| 71.4 Video-to-Mesh Reconstruction | 25 |
| 71.5 Browser-Based 3D Generation & Viewing | 29 |
| 71.6 3D Asset Library & Marketplace | 30 |
| 71.7 Quality Benchmarking Suite | 19 |
| 71.8 Unified Post-Processing Pipeline | 25 |
| 71.9 ComfyUI 3D Generation Nodes | 15 |
| 71.10 GPU Infrastructure for Self-Hosted Models | 14 |
| 71.11 Part-Aware Semantic Editing | 26 |
| 71.12 Blender MCP Server & DCC Parity | 39 |
| 71.13 Quality Gates & Active Learning | 26 |
| 71.14 Model Governance & Compliance | 24 |
| 71.15 Marketplace Trust & Operations | 24 |
| 71.16 Product Parity & Workflow UX | 24 |
| 71.17 Scene Graph & Kitbash Assembly | 24 |
| 71.18 DCC Ecosystem & Bridge Productization | 32 |
| 71.19 Unity MCP Server & Editor Parity | 219 |
| 71.20 Text & Reference-Driven 3DGS Editing | 14 |
| 71.21 Material-Decomposed 3DGS Relighting | 14 |
| 71.22 Autoregressive Mesh Generation | 14 |
| 71.23 Score Distillation (SDS/VSD) | 14 |
| 71.24 Flow Matching & Consistency Models | 12 |
| 71.25 ControlNet & IP-Adapter for 3D | 14 |
| 71.26 Text-Guided Mesh Editing | 8 |
| 71.27 3D from Video Diffusion | 8 |
| 71.28 Multi-Resolution Hash Encoding | 8 |
| 71.29 Native Quad Mesh Generation | 8 |
| 71.30 Part-Level & Multi-Instance 3D | 8 |
| 71.31 Gigascale & Next-Gen Models | 8 |
| 71.32 Advanced Gaussian Splatting | 6 |
| TOTAL Phase 71 | 923 |