AI-native PBR texture generation and refinement for Isis 3D pipelines.
71.2.1.2 is now implemented with a dedicated standalone Hunyuan3D-Paint
surface for applying PBR textures to arbitrary existing meshes, not just meshes
generated by Hunyuan shape pipelines.
Implemented now#
- standalone Hunyuan3D-Paint texture application on arbitrary
obj,ply, orglbmeshes image-only,text-only, andimage-and-textconditioning modes- dedicated text-guided PBR generation with prompt composition and required-map
validation for
albedo,normal,metallic, androughness - dedicated image-guided PBR generation that applies a reference material photo to a mesh and validates the same required PBR map set
- dedicated multi-view consistency workflow with profile-driven view counts, coverage analysis, and seam-risk reporting
- honest texture-resolution delivery from
512through8192, with native Hunyuan baking up to4096and an explicit post-scale + GLB rebuild path for8192 - automatic texture completeness validation over the final baked output, including required-map presence, UV island detection, painted-coverage scoring, and warnings for islands that still contain only background data
- dedicated re-texturing workflow for existing
objandglbassets that forces no-remesh operation, replaces the baked texture set, and blocks output if geometry or UV layout changed - depth-conditioned texture painting workflow that rasterizes real mesh depth views, passes them to an injected depth-ControlNet-style backend, and projects the generated view back into UV space as an albedo atlas with paint coverage reporting
- lighting-disentangled texture generation that estimates a smooth illumination field over the visible mesh surface, flattens baked shadows/highlights in the generated view, and bakes the neutralized albedo plus illumination map back into UV space
- iterative multi-view texture painting that reprojects already painted texels into each new canonical view, builds visible-surface-only inpaint masks for the remaining unseen regions, and skips redundant passes when a repeated view has nothing left to fill
- cross-attention view-consistency enforcement for iterative painting, with geometry-derived overlap planning that supplies reference views, target-attention masks, and pixel correspondences for later passes
- seam-aware UV-boundary blending that detects duplicated geometric edges mapped to split UV islands, smooths color discontinuities across the seam, and emits a seam mask plus blend telemetry
- local tangent-space normal-map synthesis that combines projected geometry depth with albedo high-frequency detail and supports both OpenGL and DirectX green-channel conventions
- local material-property estimation that infers
roughness,metallic, and packedmetallicRoughnessmaps from the synthesized albedo and normal detail using canonical metal-color priors plus texture-statistics inference - local ambient-occlusion baking that ray-samples the real mesh geometry in UV
space and emits an
aomap aligned to the baked atlas - local displacement-map synthesis that combines projected depth structure with
diffuse high-frequency detail to emit a normalized
displacementatlas - optional UV-island atlas optimization that repacks the existing layout before painting so the same texture resolution yields higher texel utilization
- text-to-material-graph generation that converts a material description into a typed Substance-style procedural node graph with exposed parameters, layered coating/weathering support, and explicit output-channel bindings
- material-graph parameter extraction that turns exposed graph inputs into a high-level tweakable control catalog with semantic labels, suggested ranges, UI groups, and output-channel impact
- material-graph tiling control that rewrites procedural graph sources with managed periodic transforms, exposed repeat/offset/mirror controls, and an honest seamlessness report
- deterministic material-graph variation generation that can emit curated sets
such as 10
weathered concretevariants with different stain distributions, crack densities, and parameter overrides - material blending recipes that merge generated procedural materials such as
mossy stonevia an explicit height-derived cavity mask and rebaked normal/AO outputs - environment-oriented material library builder that batch-generates curated urban, forest, desert, sci-fi, and medieval material sets using a mix of base graphs, variations, and recipe entries
- deterministic Substance Designer
.sbsexport that converts generated material graphs into XML package documents with compositing nodes, exposed parameter inputs, graph outputs, and stable node/output UIDs - OpenUSD-ready MaterialX export that emits both a
.mtlxprocedural material document and a companion.usdawrapper referencing it throughusdMtlx - PBR-aware texture super-resolution with channel-specific processing: gamma-aware albedo detail reconstruction, tangent-space normal renormalization, scalar-preserving roughness upscaling, metallic coverage stabilization, and glTF-packed metallic-roughness channel splitting/repacking
- PBR-aware texture denoising with artifact-adaptive block suppression, microdetail residual reinjection, vector-safe normal cleanup, histogram-preserving roughness filtering, metallic coverage preservation, and glTF-packed metallic-roughness channel splitting/repacking
- PBR-aware texture detail synthesis with seeded material-specific microdetail generation for pores, grain, scratches, and stone breakup while preserving UV-authored structure, normal safety, and metallic coverage
- PBR-aware texture compression optimization that preconditions albedo, normal, roughness, metallic, and packed metallic-roughness maps for BC7 / ASTC / ETC2 block encoders, emits codec-ready KTX2 preparation manifests, and reports surrogate compressibility gain plus GPU-memory estimates
- PBR-aware texture style transfer with reference-driven albedo statistics transfer, structure-aware detail preservation, estimated roughness/metallic fallback from the material photo, and glTF-packed channel support
- UDIM-aware texture processing that resolves
<UDIM>and embedded numeric tile patterns, runs super-resolution / denoising / detail synthesis / style transfer per tile, supports heterogeneous tile resolutions, and harmonizes cross-tile seams for high-detail character and environment layouts - AI-based texture baking that distills high-poly surface normals into a low-poly tangent-space normal map through shared-frustum soft rasterization, multi-view confidence fusion, UV-space refinement, and explicit non-raycast coverage / confidence outputs
- textured
objandglboutputs with albedo / metallic / roughness / metallic-roughness / normal / ambient-occlusion / displacement map passthrough - runtime preparation hooks and package-level capability manifest
Usage#
import { Local3DModelRegistry } from '@isis/3d-inference-local';
import { createLocal3DHunyuan3DPaintStandaloneTexturingPipeline } from '@isis/ai-texturing';
const registry = new Local3DModelRegistry({
rootDir: '/models/registry',
hub: {},
cachePolicy: {
maxDiskUsageGb: 500,
minFreeDiskGb: 25,
},
});
const pipeline = createLocal3DHunyuan3DPaintStandaloneTexturingPipeline({
registry,
runtimeRoot: '/opt/hunyuan3d-2.1',
});
const result = await pipeline.run({
mesh: await fs.readFile('/assets/chair.glb'),
meshFormat: 'glb',
referenceImage: await fs.readFile('/refs/worn-leather.png'),
prompt: 'aged brown leather with subtle stitching',
outputFormat: 'glb',
textureResolution: 8192,
});
result includes the textured output mesh path, preview image path, texture-map
paths, provider-style completion assets, runtime asset locations, warnings, and
standalone workflow metadata. When textureResolution is 8192, metadata also
records that delivery used a native 4096 Hunyuan bake followed by a 2x
post-scale step. The same metadata now also includes
textureCompletenessReport, textureCompletenessValid,
texturePaintedCoverage, and untexturedIslandCount so downstream workflows
can reject or review partially baked outputs instead of silently shipping them.
PBR Texture Denoising#
import { denoiseLocal3DAiTexturingPbrTextures } from '@isis/ai-texturing';
const result = await denoiseLocal3DAiTexturingPbrTextures({
mapPaths: {
albedo: '/assets/albedo-blocky.png',
normal: '/assets/normal-blocky.png',
metallicRoughness: '/assets/metallic-roughness-blocky.png',
},
artifactProfile: 'balanced',
strength: 0.85,
});
This workflow preserves each source map at its original resolution, rewrites the output as lossless PNG, and returns per-map reports for blockiness reduction, retained detail energy, and mean absolute texture delta so you can review cleanup quality rather than assuming the denoiser was safe.
PBR Texture Detail Synthesis#
import { synthesizeLocal3DAiTexturingPbrTextureDetails } from '@isis/ai-texturing';
const result = await synthesizeLocal3DAiTexturingPbrTextureDetails({
mapPaths: {
albedo: '/assets/low-detail-wood-albedo.png',
normal: '/assets/low-detail-wood-normal.png',
roughness: '/assets/low-detail-wood-roughness.png',
},
detailProfile: 'wood',
strength: 0.74,
seed: 42,
});
This workflow preserves each source map at its original resolution, adds deterministic UV-space microdetail using profile-specific procedural synthesis, and returns per-map reports for detail gain, UV-detail preservation, mean absolute delta, and metallic-coverage drift where applicable so you can review whether the added grain, pores, or scratches stayed physically plausible.
PBR Texture Style Transfer#
import { transferLocal3DAiTexturingPbrTextureStyle } from '@isis/ai-texturing';
const result = await transferLocal3DAiTexturingPbrTextureStyle({
mapPaths: {
albedo: '/assets/cool-steel-albedo.png',
normal: '/assets/cool-steel-normal.png',
metallicRoughness: '/assets/cool-steel-metallic-roughness.png',
},
styleReferenceImagePath: '/refs/brushed-copper-material.jpg',
transferProfile: 'balanced',
strength: 0.74,
});
This workflow preserves each source map at its original resolution, applies reference-material style using AdaIN-like low-frequency transfer plus content-residual preservation, and derives fallback roughness/metallic style targets from the reference photo when explicit style PBR maps are not available. The result returns per-map reports for style-match improvement, UV-detail preservation, mean absolute delta, and metallic coverage drift so you can verify that the style landed without flattening authored UV detail.
Retexturing#
import { createLocal3DHunyuan3DPaintRetexturingPipeline } from '@isis/ai-texturing';
const pipeline = createLocal3DHunyuan3DPaintRetexturingPipeline({
registry,
runtimeRoot: '/opt/hunyuan3d-2.1',
});
const result = await pipeline.run({
mesh: await readFile('/assets/robot.glb'),
meshFormat: 'glb',
referenceImage: await readFile('/refs/anodized-blue-metal.jpg'),
materialPrompt: 'anodized cobalt blue aluminum with subtle machining marks',
textureResolution: 4096,
});
This workflow is stricter than generic standalone texturing. It always runs with
useRemesh: false, validates the output against the source mesh, and fails if
vertex positions, face topology, or UV layout changed. The result metadata
includes retextureMode, preservationReport, geometryPreserved,
uvLayoutPreserved, replacedTextureMaps, and the same texture completeness
report already exposed by the standalone workflow.
Depth-Conditioned Texture Painting#
import { createLocal3DHunyuan3DPaintDepthConditionedTexturePaintingPipeline } from '@isis/ai-texturing';
const pipeline =
createLocal3DHunyuan3DPaintDepthConditionedTexturePaintingPipeline({
backend: {
async generateView({ depthMap, prompt }) {
return {
image: await someDepthControlBackend(prompt, depthMap),
model: 'sd15-depth-controlnet',
};
},
},
});
const result = await pipeline.run({
mesh: await readFile('/assets/helmet.obj'),
meshFormat: 'obj',
prompt: 'worn painted steel with chipped edges',
viewpoint: 'front',
renderWidth: 512,
renderHeight: 512,
textureResolution: 1024,
});
This workflow is intentionally honest about its current role in the stack. It
performs the real mesh-to-depth rendering and UV reprojection locally, but it
expects an injected depth-conditioned diffusion backend for the image synthesis
step. The output includes a rendered depth map, the backend-generated view, the
projected albedo texture, a paint mask, an occupied-texel mask, and coverage
metadata so later multi-view passes can target the remaining unseen texels.
After UV projection, the local pipeline also runs seam-aware blending over
duplicated UV boundaries and emits a seam-mask texture plus quantitative
seam-blending metadata. It also synthesizes a tangent-space normal map from
the projected depth atlas and the final albedo detail signal so downstream PBR
consumers are not left with a flat material response. The same local stack now
also estimates roughness, metallic, and packed metallicRoughness maps from
the generated albedo and normal detail so the workflow emits a usable PBR set
instead of albedo alone. The same baked atlas workspace now also ray-bakes an
ao map from the mesh geometry, so cavity shading tracks real self-occlusion
instead of image-space darkening. That same atlas workspace also derives a
normalized displacement map by combining the projected depth field with the
albedo's high-frequency relief signal so downstream renderers can recover macro
and micro surface variation from the generated texture set. When
optimizeTextureAtlas is enabled, the pipeline first repacks the mesh's
existing UV islands before any rendering or baking work begins, then writes an
optimized-mesh artifact plus atlas-usage telemetry alongside the painted maps
so the higher-utilization texture set has a matching mesh.
Lighting-Disentangled Texture Generation#
import { createLocal3DHunyuan3DPaintLightingDisentangledTextureGenerationPipeline } from '@isis/ai-texturing';
const pipeline =
createLocal3DHunyuan3DPaintLightingDisentangledTextureGenerationPipeline({
backend: {
async generateView({ prompt, depthMap }) {
return {
image: await someDepthControlBackend(prompt, depthMap),
model: 'sd15-depth-controlnet',
};
},
},
});
const result = await pipeline.run({
mesh: await readFile('/assets/helmet.obj'),
meshFormat: 'obj',
prompt: 'worn painted steel with chipped edges',
viewpoint: 'front',
renderWidth: 512,
renderHeight: 512,
textureResolution: 1024,
strength: 0.95,
});
This workflow starts from the same real depth-conditioned rendering path, but it
runs a local illumination-estimation pass over the backend-generated view before
UV projection. The visible-surface luminance is box-blurred into a smooth
illumination field, then shadow and highlight deltas are compressed so the baked
result trends toward flat intrinsic albedo instead of carrying view lighting.
The output includes the original generated view, a lighting-neutral view, an
illumination-map, the baked albedo texture, and metadata describing the
measured shading range plus the applied flattening strength. The final baked
albedo then passes through the same seam-aware UV blending stage, producing a
seam-mask and seam-refinement telemetry alongside the lighting outputs. A
derived tangent-space normal map is also emitted by combining the projected
mesh depth field with the lighting-neutral albedo detail. That normal map then
feeds local material-property inference, which estimates roughness,
metallic, and packed metallicRoughness outputs from the final albedo/normal
pair. The same UV-space geometry pass now also emits an ao texture derived
from actual mesh occlusion before export. That lighting-neutral atlas then also
feeds local displacement synthesis, which blends the projected depth structure
with detail extracted from the flattened albedo into a normalized displacement
map. The same optional optimizeTextureAtlas prepass is available here too, so
lighting-neutral bakes can target a repacked UV layout before any ControlNet
rendering or UV projection occurs.
Iterative Multi-View Painting#
import { createLocal3DHunyuan3DPaintIterativeMultiViewTexturePaintingPipeline } from '@isis/ai-texturing';
const pipeline =
createLocal3DHunyuan3DPaintIterativeMultiViewTexturePaintingPipeline({
backend: {
async generateView({ prompt, depthMap, baseView, inpaintMask }) {
return {
image: await someDepthAwareInpaintBackend({
prompt,
depthMap,
baseView,
inpaintMask,
}),
model: 'sdxl-depth-inpaint',
};
},
},
});
const result = await pipeline.run({
mesh: await readFile('/assets/helmet.obj'),
meshFormat: 'obj',
prompt: 'worn painted steel with chipped edges',
viewSequence: ['front', 'right', 'back', 'left', 'top', 'bottom'],
renderWidth: 512,
renderHeight: 512,
textureResolution: 1024,
});
This workflow extends the single-view depth-conditioned path into a real
front-to-back accumulation loop. Each pass reprojects the current atlas into the
next camera view, computes an inpaint mask only over visible mesh pixels that
still have no painted texel assignment, calls the injected backend with
depthMap + baseView + inpaintMask, and projects the generated result back into
UV space. The result metadata exposes per-pass missingPixelCount,
newlyPaintedTexelCount, cumulativeCoverageRatio, and skippedGeneration
flags so downstream orchestration can see exactly where coverage increased and
which repeated views became redundant.
By default, the iterative workflow now also enables
consistencyMode: 'cross-attention'. Before each later pass, it computes real
overlap between the current visible texels and the texels owned by previous
passes, selects the strongest prior views, and passes crossAttentionReferences
into the backend. Each reference includes the prior generated image, its
coverage mask, a target-view attention mask for overlapping pixels, and a dense
referencePixelIndexByTargetPixel mapping so a diffusion backend can enforce
view consistency using actual UV-mediated correspondences instead of a prompt
hint. Once the atlas is complete, the pipeline also performs seam-aware UV
boundary blending across duplicated geometric edges and records the resulting
seam-mask, seam texel counts, and average blend delta in the final metadata.
The same final atlas then feeds local normal-map synthesis so the completed
multi-view result ships with a generated tangent-space normal texture instead
of albedo alone. The finalized albedo + normal pair also feeds local
material-property inference so iterative output now includes generated
roughness, metallic, and packed metallicRoughness textures. That shared UV
workspace also now bakes an ao texture from geometry, so the final atlas set
carries per-texel ambient occlusion without requiring a separate DCC bake. The
completed diffuse atlas and accumulated projected depth coverage also feed a
local displacement synthesis pass, so the iterative workflow now exports a
normalized height field alongside the rest of the PBR atlas set. Enabling
optimizeTextureAtlas applies the same pre-pass UV-island repack before the
iterative view loop begins, and the returned metadata includes before/after
utilization plus texel-density deltas for the optimized layout.
Text To Material Graph#
import {
applyLocal3DAiTexturingMaterialGraphTilingControl,
extractLocal3DAiTexturingMaterialGraphParameters,
generateLocal3DAiTexturingTextToMaterialGraph,
} from '@isis/ai-texturing';
const graphResult = generateLocal3DAiTexturingTextToMaterialGraph({
prompt: 'weathered painted steel with chipped red paint and rust streaks',
});
const tiledGraphResult = applyLocal3DAiTexturingMaterialGraphTilingControl({
graph: graphResult.graph,
targetScale: 3.5,
repeatX: 4,
repeatY: 2,
});
const parameterResult = extractLocal3DAiTexturingMaterialGraphParameters({
graph: tiledGraphResult.graph,
});
This workflow emits a procedural graph description instead of baked textures.
The result includes prompt analysis, detected material family, a typed
Substance-style node/edge graph, layered material-stack metadata, exposed tweak
parameters such as tile_scale, base_color_primary, roughness_amount,
wear_amount, and optional coating/contaminant controls, plus explicit output
bindings for channels like baseColor, roughness, metallic, normal,
height, ao, opacity, transmission, and emissive when the prompt
implies them. The internal graph model follows the same node-graph concepts used
by MaterialX and Substance Designer so later .sbs and MaterialX export tasks
can target a real structured graph rather than a lossy prompt summary. The
parameter extractor then walks the graph topology to identify the practical
artist-facing controls, groups them into buckets like Surface/Color,
Pattern/Scale, and Weathering/Primary, and records which output channels
each control actually influences. The tiling-control pass can then rewrite the
graph into a seamless periodic variant by inserting managed transform-2d nodes
after every procedural pattern source, exposing repeat/offset/rotation and
mirror controls, and returning a report with per-axis coverage plus an estimated
seamlessness score for the tiled outputs.
import { generateLocal3DAiTexturingMaterialGraphVariations } from '@isis/ai-texturing';
const variationSet = generateLocal3DAiTexturingMaterialGraphVariations({
prompt: 'weathered concrete',
count: 10,
});
The variation generator builds deterministic prompt-backed variants on top of the same graph workflow, then injects real stain overlays and crack tuning into each graph so the output set spans different runoff/blotchy/speckled stain patterns, crack profiles, roughness levels, and tile scales instead of merely relabeling one underlying graph.
import { buildLocal3DAiTexturingMaterialBlendRecipe } from '@isis/ai-texturing';
const recipe = buildLocal3DAiTexturingMaterialBlendRecipe({
recipeId: 'mossy-stone',
});
The recipe builder composes separately generated base and overlay graphs,
imports the overlay graph with isolated parameter namespaces, derives a real
blend mask from the base height field plus overlay growth structure, blends the
main PBR channels, and regenerates normal plus ao from the merged height so
the resulting material graph stays internally coherent.
import { buildLocal3DAiTexturingMaterialLibrary } from '@isis/ai-texturing';
const library = buildLocal3DAiTexturingMaterialLibrary({
environments: ['urban', 'medieval'],
variationsPerMaterial: 2,
});
The library builder turns those lower-level graph tools into environment packs: it batches base materials, variation sets, and recipe-driven blends into tagged catalogs that can seed urban streets, forest floors, desert ruins, sci-fi hard-surface scenes, or medieval environments with deterministic graph assets rather than ad hoc prompt lists.
Text-Guided PBR#
import { createLocal3DHunyuan3DPaintTextGuidedPbrPipeline } from '@isis/ai-texturing';
const pipeline = createLocal3DHunyuan3DPaintTextGuidedPbrPipeline({
registry,
runtimeRoot: '/opt/hunyuan3d-2.1',
});
const result = await pipeline.run({
mesh: await readFile('/assets/statue.glb'),
meshFormat: 'glb',
materialDescriptor: {
material: 'weathered bronze',
finish: 'satin',
wear: 'edge patina and subtle scratches',
microDetail: 'fine cast metal grain',
},
negativeMaterialPrompt: 'plastic sheen',
textureResolution: 2048,
});
This workflow is stricter than generic standalone texturing: it composes a material-focused prompt and fails fast if any of the required PBR maps are missing from the result.
Image-Guided PBR#
import { createLocal3DHunyuan3DPaintImageGuidedPbrPipeline } from '@isis/ai-texturing';
const pipeline = createLocal3DHunyuan3DPaintImageGuidedPbrPipeline({
registry,
runtimeRoot: '/opt/hunyuan3d-2.1',
});
const result = await pipeline.run({
mesh: await readFile('/assets/chair.glb'),
meshFormat: 'glb',
referenceImage: await readFile('/refs/distressed-leather.jpg'),
materialHint: 'retain the stitched leather look and dark creases',
textureResolution: 2048,
});
This workflow requires a real reference image and fails fast if the returned map
set is missing albedo, normal, metallic, or roughness.
Multi-View Consistency#
import { createLocal3DHunyuan3DPaintMultiViewConsistencyPipeline } from '@isis/ai-texturing';
const pipeline = createLocal3DHunyuan3DPaintMultiViewConsistencyPipeline({
registry,
runtimeRoot: '/opt/hunyuan3d-2.1',
});
const result = await pipeline.run({
mesh: await readFile('/assets/vase.glb'),
meshFormat: 'glb',
prompt: 'hand-painted ceramic glaze with subtle edge wear',
consistencyProfile: 'maximum',
textureResolution: 2048,
});
This workflow still uses the real Hunyuan multiview bake path, but it adds a coverage report over the selected front/back/side/top/bottom views and reports a computed seam-risk score instead of silently assuming the bake was consistent.
Notes#
- The upstream Hunyuan3D-Paint release is fundamentally a mesh texturing stage, so this package wraps the existing local runtime instead of reimplementing it.
- The official quick-start centers on
mesh + reference image -> textured mesh; this package also supportstext-onlyby delegating to the existing local wrapper, which uses the same neutral anchor fallback already implemented in@isis/3d-inference-local.