# @isis/ai-texturing

AI-native PBR texture generation and refinement for Isis 3D pipelines.

`71.2.1.2` is now implemented with a dedicated standalone Hunyuan3D-Paint
surface for applying PBR textures to arbitrary existing meshes, not just meshes
generated by Hunyuan shape pipelines.

## Implemented now

- standalone Hunyuan3D-Paint texture application on arbitrary `obj`, `ply`, or
  `glb` meshes
- `image-only`, `text-only`, and `image-and-text` conditioning modes
- dedicated text-guided PBR generation with prompt composition and required-map
  validation for `albedo`, `normal`, `metallic`, and `roughness`
- dedicated image-guided PBR generation that applies a reference material photo
  to a mesh and validates the same required PBR map set
- dedicated multi-view consistency workflow with profile-driven view counts,
  coverage analysis, and seam-risk reporting
- honest texture-resolution delivery from `512` through `8192`, with native
  Hunyuan baking up to `4096` and an explicit post-scale + GLB rebuild path for
  `8192`
- automatic texture completeness validation over the final baked output,
  including required-map presence, UV island detection, painted-coverage
  scoring, and warnings for islands that still contain only background data
- dedicated re-texturing workflow for existing `obj` and `glb` assets that
  forces no-remesh operation, replaces the baked texture set, and blocks output
  if geometry or UV layout changed
- depth-conditioned texture painting workflow that rasterizes real mesh depth
  views, passes them to an injected depth-ControlNet-style backend, and projects
  the generated view back into UV space as an albedo atlas with paint coverage
  reporting
- lighting-disentangled texture generation that estimates a smooth illumination
  field over the visible mesh surface, flattens baked shadows/highlights in the
  generated view, and bakes the neutralized albedo plus illumination map back
  into UV space
- iterative multi-view texture painting that reprojects already painted texels
  into each new canonical view, builds visible-surface-only inpaint masks for
  the remaining unseen regions, and skips redundant passes when a repeated view
  has nothing left to fill
- cross-attention view-consistency enforcement for iterative painting, with
  geometry-derived overlap planning that supplies reference views,
  target-attention masks, and pixel correspondences for later passes
- seam-aware UV-boundary blending that detects duplicated geometric edges mapped
  to split UV islands, smooths color discontinuities across the seam, and emits
  a seam mask plus blend telemetry
- local tangent-space normal-map synthesis that combines projected geometry
  depth with albedo high-frequency detail and supports both OpenGL and DirectX
  green-channel conventions
- local material-property estimation that infers `roughness`, `metallic`, and
  packed `metallicRoughness` maps from the synthesized albedo and normal detail
  using canonical metal-color priors plus texture-statistics inference
- local ambient-occlusion baking that ray-samples the real mesh geometry in UV
  space and emits an `ao` map aligned to the baked atlas
- local displacement-map synthesis that combines projected depth structure with
  diffuse high-frequency detail to emit a normalized `displacement` atlas
- optional UV-island atlas optimization that repacks the existing layout before
  painting so the same texture resolution yields higher texel utilization
- text-to-material-graph generation that converts a material description into a
  typed Substance-style procedural node graph with exposed parameters, layered
  coating/weathering support, and explicit output-channel bindings
- material-graph parameter extraction that turns exposed graph inputs into a
  high-level tweakable control catalog with semantic labels, suggested ranges,
  UI groups, and output-channel impact
- material-graph tiling control that rewrites procedural graph sources with
  managed periodic transforms, exposed repeat/offset/mirror controls, and an
  honest seamlessness report
- deterministic material-graph variation generation that can emit curated sets
  such as 10 `weathered concrete` variants with different stain distributions,
  crack densities, and parameter overrides
- material blending recipes that merge generated procedural materials such as
  `mossy stone` via an explicit height-derived cavity mask and rebaked normal/AO
  outputs
- environment-oriented material library builder that batch-generates curated
  urban, forest, desert, sci-fi, and medieval material sets using a mix of base
  graphs, variations, and recipe entries
- deterministic Substance Designer `.sbs` export that converts generated
  material graphs into XML package documents with compositing nodes, exposed
  parameter inputs, graph outputs, and stable node/output UIDs
- OpenUSD-ready MaterialX export that emits both a `.mtlx` procedural material
  document and a companion `.usda` wrapper referencing it through `usdMtlx`
- PBR-aware texture super-resolution with channel-specific processing:
  gamma-aware albedo detail reconstruction, tangent-space normal
  renormalization, scalar-preserving roughness upscaling, metallic coverage
  stabilization, and glTF-packed metallic-roughness channel splitting/repacking
- PBR-aware texture denoising with artifact-adaptive block suppression,
  microdetail residual reinjection, vector-safe normal cleanup,
  histogram-preserving roughness filtering, metallic coverage preservation, and
  glTF-packed metallic-roughness channel splitting/repacking
- PBR-aware texture detail synthesis with seeded material-specific microdetail
  generation for pores, grain, scratches, and stone breakup while preserving
  UV-authored structure, normal safety, and metallic coverage
- PBR-aware texture compression optimization that preconditions albedo, normal,
  roughness, metallic, and packed metallic-roughness maps for BC7 / ASTC / ETC2
  block encoders, emits codec-ready KTX2 preparation manifests, and reports
  surrogate compressibility gain plus GPU-memory estimates
- PBR-aware texture style transfer with reference-driven albedo statistics
  transfer, structure-aware detail preservation, estimated roughness/metallic
  fallback from the material photo, and glTF-packed channel support
- UDIM-aware texture processing that resolves `<UDIM>` and embedded numeric tile
  patterns, runs super-resolution / denoising / detail synthesis / style
  transfer per tile, supports heterogeneous tile resolutions, and harmonizes
  cross-tile seams for high-detail character and environment layouts
- AI-based texture baking that distills high-poly surface normals into a
  low-poly tangent-space normal map through shared-frustum soft rasterization,
  multi-view confidence fusion, UV-space refinement, and explicit non-raycast
  coverage / confidence outputs
- textured `obj` and `glb` outputs with albedo / metallic / roughness /
  metallic-roughness / normal / ambient-occlusion / displacement map passthrough
- runtime preparation hooks and package-level capability manifest

## Usage

```ts
import { Local3DModelRegistry } from '@isis/3d-inference-local';
import { createLocal3DHunyuan3DPaintStandaloneTexturingPipeline } from '@isis/ai-texturing';

const registry = new Local3DModelRegistry({
  rootDir: '/models/registry',
  hub: {},
  cachePolicy: {
    maxDiskUsageGb: 500,
    minFreeDiskGb: 25,
  },
});

const pipeline = createLocal3DHunyuan3DPaintStandaloneTexturingPipeline({
  registry,
  runtimeRoot: '/opt/hunyuan3d-2.1',
});

const result = await pipeline.run({
  mesh: await fs.readFile('/assets/chair.glb'),
  meshFormat: 'glb',
  referenceImage: await fs.readFile('/refs/worn-leather.png'),
  prompt: 'aged brown leather with subtle stitching',
  outputFormat: 'glb',
  textureResolution: 8192,
});
```

`result` includes the textured output mesh path, preview image path, texture-map
paths, provider-style completion assets, runtime asset locations, warnings, and
standalone workflow metadata. When `textureResolution` is `8192`, metadata also
records that delivery used a native `4096` Hunyuan bake followed by a `2x`
post-scale step. The same metadata now also includes
`textureCompletenessReport`, `textureCompletenessValid`,
`texturePaintedCoverage`, and `untexturedIslandCount` so downstream workflows
can reject or review partially baked outputs instead of silently shipping them.

## PBR Texture Denoising

```ts
import { denoiseLocal3DAiTexturingPbrTextures } from '@isis/ai-texturing';

const result = await denoiseLocal3DAiTexturingPbrTextures({
  mapPaths: {
    albedo: '/assets/albedo-blocky.png',
    normal: '/assets/normal-blocky.png',
    metallicRoughness: '/assets/metallic-roughness-blocky.png',
  },
  artifactProfile: 'balanced',
  strength: 0.85,
});
```

This workflow preserves each source map at its original resolution, rewrites the
output as lossless PNG, and returns per-map reports for blockiness reduction,
retained detail energy, and mean absolute texture delta so you can review
cleanup quality rather than assuming the denoiser was safe.

## PBR Texture Detail Synthesis

```ts
import { synthesizeLocal3DAiTexturingPbrTextureDetails } from '@isis/ai-texturing';

const result = await synthesizeLocal3DAiTexturingPbrTextureDetails({
  mapPaths: {
    albedo: '/assets/low-detail-wood-albedo.png',
    normal: '/assets/low-detail-wood-normal.png',
    roughness: '/assets/low-detail-wood-roughness.png',
  },
  detailProfile: 'wood',
  strength: 0.74,
  seed: 42,
});
```

This workflow preserves each source map at its original resolution, adds
deterministic UV-space microdetail using profile-specific procedural synthesis,
and returns per-map reports for detail gain, UV-detail preservation, mean
absolute delta, and metallic-coverage drift where applicable so you can review
whether the added grain, pores, or scratches stayed physically plausible.

## PBR Texture Style Transfer

```ts
import { transferLocal3DAiTexturingPbrTextureStyle } from '@isis/ai-texturing';

const result = await transferLocal3DAiTexturingPbrTextureStyle({
  mapPaths: {
    albedo: '/assets/cool-steel-albedo.png',
    normal: '/assets/cool-steel-normal.png',
    metallicRoughness: '/assets/cool-steel-metallic-roughness.png',
  },
  styleReferenceImagePath: '/refs/brushed-copper-material.jpg',
  transferProfile: 'balanced',
  strength: 0.74,
});
```

This workflow preserves each source map at its original resolution, applies
reference-material style using AdaIN-like low-frequency transfer plus
content-residual preservation, and derives fallback roughness/metallic style
targets from the reference photo when explicit style PBR maps are not available.
The result returns per-map reports for style-match improvement, UV-detail
preservation, mean absolute delta, and metallic coverage drift so you can verify
that the style landed without flattening authored UV detail.

## Retexturing

```ts
import { createLocal3DHunyuan3DPaintRetexturingPipeline } from '@isis/ai-texturing';

const pipeline = createLocal3DHunyuan3DPaintRetexturingPipeline({
  registry,
  runtimeRoot: '/opt/hunyuan3d-2.1',
});

const result = await pipeline.run({
  mesh: await readFile('/assets/robot.glb'),
  meshFormat: 'glb',
  referenceImage: await readFile('/refs/anodized-blue-metal.jpg'),
  materialPrompt: 'anodized cobalt blue aluminum with subtle machining marks',
  textureResolution: 4096,
});
```

This workflow is stricter than generic standalone texturing. It always runs with
`useRemesh: false`, validates the output against the source mesh, and fails if
vertex positions, face topology, or UV layout changed. The result metadata
includes `retextureMode`, `preservationReport`, `geometryPreserved`,
`uvLayoutPreserved`, `replacedTextureMaps`, and the same texture completeness
report already exposed by the standalone workflow.

## Depth-Conditioned Texture Painting

```ts
import { createLocal3DHunyuan3DPaintDepthConditionedTexturePaintingPipeline } from '@isis/ai-texturing';

const pipeline =
  createLocal3DHunyuan3DPaintDepthConditionedTexturePaintingPipeline({
    backend: {
      async generateView({ depthMap, prompt }) {
        return {
          image: await someDepthControlBackend(prompt, depthMap),
          model: 'sd15-depth-controlnet',
        };
      },
    },
  });

const result = await pipeline.run({
  mesh: await readFile('/assets/helmet.obj'),
  meshFormat: 'obj',
  prompt: 'worn painted steel with chipped edges',
  viewpoint: 'front',
  renderWidth: 512,
  renderHeight: 512,
  textureResolution: 1024,
});
```

This workflow is intentionally honest about its current role in the stack. It
performs the real mesh-to-depth rendering and UV reprojection locally, but it
expects an injected depth-conditioned diffusion backend for the image synthesis
step. The output includes a rendered depth map, the backend-generated view, the
projected `albedo` texture, a paint mask, an occupied-texel mask, and coverage
metadata so later multi-view passes can target the remaining unseen texels.
After UV projection, the local pipeline also runs seam-aware blending over
duplicated UV boundaries and emits a `seam-mask` texture plus quantitative
seam-blending metadata. It also synthesizes a tangent-space `normal` map from
the projected depth atlas and the final albedo detail signal so downstream PBR
consumers are not left with a flat material response. The same local stack now
also estimates `roughness`, `metallic`, and packed `metallicRoughness` maps from
the generated albedo and normal detail so the workflow emits a usable PBR set
instead of albedo alone. The same baked atlas workspace now also ray-bakes an
`ao` map from the mesh geometry, so cavity shading tracks real self-occlusion
instead of image-space darkening. That same atlas workspace also derives a
normalized `displacement` map by combining the projected depth field with the
albedo's high-frequency relief signal so downstream renderers can recover macro
and micro surface variation from the generated texture set. When
`optimizeTextureAtlas` is enabled, the pipeline first repacks the mesh's
existing UV islands before any rendering or baking work begins, then writes an
`optimized-mesh` artifact plus atlas-usage telemetry alongside the painted maps
so the higher-utilization texture set has a matching mesh.

## Lighting-Disentangled Texture Generation

```ts
import { createLocal3DHunyuan3DPaintLightingDisentangledTextureGenerationPipeline } from '@isis/ai-texturing';

const pipeline =
  createLocal3DHunyuan3DPaintLightingDisentangledTextureGenerationPipeline({
    backend: {
      async generateView({ prompt, depthMap }) {
        return {
          image: await someDepthControlBackend(prompt, depthMap),
          model: 'sd15-depth-controlnet',
        };
      },
    },
  });

const result = await pipeline.run({
  mesh: await readFile('/assets/helmet.obj'),
  meshFormat: 'obj',
  prompt: 'worn painted steel with chipped edges',
  viewpoint: 'front',
  renderWidth: 512,
  renderHeight: 512,
  textureResolution: 1024,
  strength: 0.95,
});
```

This workflow starts from the same real depth-conditioned rendering path, but it
runs a local illumination-estimation pass over the backend-generated view before
UV projection. The visible-surface luminance is box-blurred into a smooth
illumination field, then shadow and highlight deltas are compressed so the baked
result trends toward flat intrinsic albedo instead of carrying view lighting.
The output includes the original generated view, a `lighting-neutral` view, an
`illumination-map`, the baked `albedo` texture, and metadata describing the
measured shading range plus the applied flattening strength. The final baked
albedo then passes through the same seam-aware UV blending stage, producing a
`seam-mask` and seam-refinement telemetry alongside the lighting outputs. A
derived tangent-space `normal` map is also emitted by combining the projected
mesh depth field with the lighting-neutral albedo detail. That normal map then
feeds local material-property inference, which estimates `roughness`,
`metallic`, and packed `metallicRoughness` outputs from the final albedo/normal
pair. The same UV-space geometry pass now also emits an `ao` texture derived
from actual mesh occlusion before export. That lighting-neutral atlas then also
feeds local displacement synthesis, which blends the projected depth structure
with detail extracted from the flattened albedo into a normalized `displacement`
map. The same optional `optimizeTextureAtlas` prepass is available here too, so
lighting-neutral bakes can target a repacked UV layout before any ControlNet
rendering or UV projection occurs.

## Iterative Multi-View Painting

```ts
import { createLocal3DHunyuan3DPaintIterativeMultiViewTexturePaintingPipeline } from '@isis/ai-texturing';

const pipeline =
  createLocal3DHunyuan3DPaintIterativeMultiViewTexturePaintingPipeline({
    backend: {
      async generateView({ prompt, depthMap, baseView, inpaintMask }) {
        return {
          image: await someDepthAwareInpaintBackend({
            prompt,
            depthMap,
            baseView,
            inpaintMask,
          }),
          model: 'sdxl-depth-inpaint',
        };
      },
    },
  });

const result = await pipeline.run({
  mesh: await readFile('/assets/helmet.obj'),
  meshFormat: 'obj',
  prompt: 'worn painted steel with chipped edges',
  viewSequence: ['front', 'right', 'back', 'left', 'top', 'bottom'],
  renderWidth: 512,
  renderHeight: 512,
  textureResolution: 1024,
});
```

This workflow extends the single-view depth-conditioned path into a real
front-to-back accumulation loop. Each pass reprojects the current atlas into the
next camera view, computes an inpaint mask only over visible mesh pixels that
still have no painted texel assignment, calls the injected backend with
`depthMap + baseView + inpaintMask`, and projects the generated result back into
UV space. The result metadata exposes per-pass `missingPixelCount`,
`newlyPaintedTexelCount`, `cumulativeCoverageRatio`, and `skippedGeneration`
flags so downstream orchestration can see exactly where coverage increased and
which repeated views became redundant.

By default, the iterative workflow now also enables
`consistencyMode: 'cross-attention'`. Before each later pass, it computes real
overlap between the current visible texels and the texels owned by previous
passes, selects the strongest prior views, and passes `crossAttentionReferences`
into the backend. Each reference includes the prior generated image, its
coverage mask, a target-view attention mask for overlapping pixels, and a dense
`referencePixelIndexByTargetPixel` mapping so a diffusion backend can enforce
view consistency using actual UV-mediated correspondences instead of a prompt
hint. Once the atlas is complete, the pipeline also performs seam-aware UV
boundary blending across duplicated geometric edges and records the resulting
`seam-mask`, seam texel counts, and average blend delta in the final metadata.
The same final atlas then feeds local normal-map synthesis so the completed
multi-view result ships with a generated tangent-space `normal` texture instead
of albedo alone. The finalized albedo + normal pair also feeds local
material-property inference so iterative output now includes generated
`roughness`, `metallic`, and packed `metallicRoughness` textures. That shared UV
workspace also now bakes an `ao` texture from geometry, so the final atlas set
carries per-texel ambient occlusion without requiring a separate DCC bake. The
completed diffuse atlas and accumulated projected depth coverage also feed a
local `displacement` synthesis pass, so the iterative workflow now exports a
normalized height field alongside the rest of the PBR atlas set. Enabling
`optimizeTextureAtlas` applies the same pre-pass UV-island repack before the
iterative view loop begins, and the returned metadata includes before/after
utilization plus texel-density deltas for the optimized layout.

## Text To Material Graph

```ts
import {
  applyLocal3DAiTexturingMaterialGraphTilingControl,
  extractLocal3DAiTexturingMaterialGraphParameters,
  generateLocal3DAiTexturingTextToMaterialGraph,
} from '@isis/ai-texturing';

const graphResult = generateLocal3DAiTexturingTextToMaterialGraph({
  prompt: 'weathered painted steel with chipped red paint and rust streaks',
});
const tiledGraphResult = applyLocal3DAiTexturingMaterialGraphTilingControl({
  graph: graphResult.graph,
  targetScale: 3.5,
  repeatX: 4,
  repeatY: 2,
});
const parameterResult = extractLocal3DAiTexturingMaterialGraphParameters({
  graph: tiledGraphResult.graph,
});
```

This workflow emits a procedural graph description instead of baked textures.
The result includes prompt analysis, detected material family, a typed
Substance-style node/edge graph, layered material-stack metadata, exposed tweak
parameters such as `tile_scale`, `base_color_primary`, `roughness_amount`,
`wear_amount`, and optional coating/contaminant controls, plus explicit output
bindings for channels like `baseColor`, `roughness`, `metallic`, `normal`,
`height`, `ao`, `opacity`, `transmission`, and `emissive` when the prompt
implies them. The internal graph model follows the same node-graph concepts used
by MaterialX and Substance Designer so later `.sbs` and MaterialX export tasks
can target a real structured graph rather than a lossy prompt summary. The
parameter extractor then walks the graph topology to identify the practical
artist-facing controls, groups them into buckets like `Surface/Color`,
`Pattern/Scale`, and `Weathering/Primary`, and records which output channels
each control actually influences. The tiling-control pass can then rewrite the
graph into a seamless periodic variant by inserting managed `transform-2d` nodes
after every procedural pattern source, exposing repeat/offset/rotation and
mirror controls, and returning a report with per-axis coverage plus an estimated
seamlessness score for the tiled outputs.

```ts
import { generateLocal3DAiTexturingMaterialGraphVariations } from '@isis/ai-texturing';

const variationSet = generateLocal3DAiTexturingMaterialGraphVariations({
  prompt: 'weathered concrete',
  count: 10,
});
```

The variation generator builds deterministic prompt-backed variants on top of
the same graph workflow, then injects real stain overlays and crack tuning into
each graph so the output set spans different runoff/blotchy/speckled stain
patterns, crack profiles, roughness levels, and tile scales instead of merely
relabeling one underlying graph.

```ts
import { buildLocal3DAiTexturingMaterialBlendRecipe } from '@isis/ai-texturing';

const recipe = buildLocal3DAiTexturingMaterialBlendRecipe({
  recipeId: 'mossy-stone',
});
```

The recipe builder composes separately generated base and overlay graphs,
imports the overlay graph with isolated parameter namespaces, derives a real
blend mask from the base height field plus overlay growth structure, blends the
main PBR channels, and regenerates `normal` plus `ao` from the merged height so
the resulting material graph stays internally coherent.

```ts
import { buildLocal3DAiTexturingMaterialLibrary } from '@isis/ai-texturing';

const library = buildLocal3DAiTexturingMaterialLibrary({
  environments: ['urban', 'medieval'],
  variationsPerMaterial: 2,
});
```

The library builder turns those lower-level graph tools into environment packs:
it batches base materials, variation sets, and recipe-driven blends into tagged
catalogs that can seed urban streets, forest floors, desert ruins, sci-fi
hard-surface scenes, or medieval environments with deterministic graph assets
rather than ad hoc prompt lists.

## Text-Guided PBR

```ts
import { createLocal3DHunyuan3DPaintTextGuidedPbrPipeline } from '@isis/ai-texturing';

const pipeline = createLocal3DHunyuan3DPaintTextGuidedPbrPipeline({
  registry,
  runtimeRoot: '/opt/hunyuan3d-2.1',
});

const result = await pipeline.run({
  mesh: await readFile('/assets/statue.glb'),
  meshFormat: 'glb',
  materialDescriptor: {
    material: 'weathered bronze',
    finish: 'satin',
    wear: 'edge patina and subtle scratches',
    microDetail: 'fine cast metal grain',
  },
  negativeMaterialPrompt: 'plastic sheen',
  textureResolution: 2048,
});
```

This workflow is stricter than generic standalone texturing: it composes a
material-focused prompt and fails fast if any of the required PBR maps are
missing from the result.

## Image-Guided PBR

```ts
import { createLocal3DHunyuan3DPaintImageGuidedPbrPipeline } from '@isis/ai-texturing';

const pipeline = createLocal3DHunyuan3DPaintImageGuidedPbrPipeline({
  registry,
  runtimeRoot: '/opt/hunyuan3d-2.1',
});

const result = await pipeline.run({
  mesh: await readFile('/assets/chair.glb'),
  meshFormat: 'glb',
  referenceImage: await readFile('/refs/distressed-leather.jpg'),
  materialHint: 'retain the stitched leather look and dark creases',
  textureResolution: 2048,
});
```

This workflow requires a real reference image and fails fast if the returned map
set is missing `albedo`, `normal`, `metallic`, or `roughness`.

## Multi-View Consistency

```ts
import { createLocal3DHunyuan3DPaintMultiViewConsistencyPipeline } from '@isis/ai-texturing';

const pipeline = createLocal3DHunyuan3DPaintMultiViewConsistencyPipeline({
  registry,
  runtimeRoot: '/opt/hunyuan3d-2.1',
});

const result = await pipeline.run({
  mesh: await readFile('/assets/vase.glb'),
  meshFormat: 'glb',
  prompt: 'hand-painted ceramic glaze with subtle edge wear',
  consistencyProfile: 'maximum',
  textureResolution: 2048,
});
```

This workflow still uses the real Hunyuan multiview bake path, but it adds a
coverage report over the selected front/back/side/top/bottom views and reports a
computed seam-risk score instead of silently assuming the bake was consistent.

## Notes

- The upstream Hunyuan3D-Paint release is fundamentally a mesh texturing stage,
  so this package wraps the existing local runtime instead of reimplementing it.
- The official quick-start centers on `mesh + reference image -> textured mesh`;
  this package also supports `text-only` by delegating to the existing local
  wrapper, which uses the same neutral anchor fallback already implemented in
  `@isis/3d-inference-local`.