Fighting Game · Guides & deep dives

Maya Shader Pipeline

The implementation lives in libs/maya/engine-core/crates/maya-renderer/src.

9sections5 minread1table
Markdown source
On this page

This document is the engineering source of truth for the Maya renderer shader pipeline in V2. It connects renderer-owned WGSL sources, permutation planning, cross-compilation, cache generation, diagnostics, complexity analysis, usage tracking, parallel compilation, and binary size regression gates.

Source Of Truth#

The implementation lives in libs/maya/engine-core/crates/maya-renderer/src. The pipeline uses WGSL-primary shader sources and stable metadata from these contracts:

  • V2/ue/Build/Shaders/v2-maya-shader-source-management.json
  • V2/ue/Build/Shaders/v2-maya-shader-permutation-system.json
  • V2/ue/Build/Shaders/v2-maya-shader-permutation-explosion-management.json
  • V2/ue/Build/Shaders/v2-maya-shader-cross-compilation-pipeline.json
  • V2/ue/Build/Shaders/v2-maya-shader-pso-caching.json
  • V2/ue/Build/Shaders/v2-maya-shader-precompilation-cook.json
  • V2/ue/Build/Shaders/v2-maya-shader-hot-reload.json
  • V2/ue/Build/Shaders/v2-maya-shader-include-module-system.json
  • V2/ue/Build/Shaders/v2-maya-shader-debugging-symbols.json
  • V2/ue/Build/Shaders/v2-maya-shader-complexity-analysis.json
  • V2/ue/Build/Shaders/v2-maya-shader-variant-usage-tracking.json
  • V2/ue/Build/Shaders/v2-maya-shader-compilation-error-reporting.json
  • V2/ue/Build/Shaders/v2-maya-shader-compilation-parallelization.json
  • V2/ue/Build/Shaders/v2-maya-shader-binary-size-tracking.json
  • V2/ue/Build/Shaders/v2-shader-distribution.json

The required CI validator for this document is:

bash
python3 V2/ue/Tools/check-v2-maya-shader-pipeline-documentation.py

Architecture#

The pipeline has one ordering rule: source identity is decided before permutation identity, permutation identity is decided before target compilation, and target compilation is decided before cache, size, and release gates.

Stage Owner Primary artifact
Source catalog shader_source_management WGSL descriptors under shaders/maya
Include composition shader_include_module_system composed WGSL with line spans
Permutations shader_permutation_system ShaderPermutationManifest
Pruning shader_permutation_explosion_management management plan from usage profile
Cross-compile jobs shader_cross_compilation_pipeline ShaderCrossCompileJob rows
Local and farm scheduling shader_compilation_parallelization local workers and Horde shards
Diagnostics shader_compilation_error_reporting source-mapped reports
Debug symbols shader_debugging_symbols source maps and symbol packages
PSO and shader caches shader_cache, pipeline_state_object_cache content-addressed blobs
Cook and distribution shader_precompilation_cook, shader distribution plan DDC fill outputs
Quality gates shader_complexity_analysis, shader_binary_size_tracking complexity and size reports

The core Maya shader library currently expands to 32 variants and 96 default cross-compile jobs for Vulkan SPIR-V, DX12 DXIL, and Metal MSL. The release target shorthand is Vulkan SPIR-V DX12 DXIL Metal MSL.

Source And Include Authoring#

All new renderer-owned shader source starts as WGSL. Add a ShaderSourceDescriptor with a stable maya.<feature>.<name> id, a relative path under shaders/maya/<feature>/, a concrete ShaderStage, and an entry point that exists in the WGSL source.

Reusable code belongs in include modules when it is shared across features or would otherwise duplicate math, binding, or lighting logic. Include modules must keep source spans intact so include source mapping can let ShaderComposedSource map generated compiler diagnostics back to original shader files and include modules.

Do not add backend-specific source as the canonical file. HLSL, MSL, DXIL, and SPIR-V are generated artifacts produced by the cross-compilation pipeline and validated by the target contracts.

Permutations And Usage#

Permutation dimensions must describe real runtime choices such as skin, damage, sweat, cloth, hair, lighting, shadow quality, reflection, volumetric, or post-process features. A new dimension must have:

  • a stable toggle name and bit assignment,
  • a default state,
  • a content or runtime owner,
  • a pruning rule in the explosion manager when the dimension can multiply many variants,
  • usage telemetry through shader_variant_usage_tracking when runtime evidence can prove which variants are used.

The pruning loop is evidence-driven. Runtime profiles feed ShaderPermutationUsageProfile; the explosion manager keeps used variants, prunes unused variants, and protects required platform or content variants.

Compilation And Distribution#

shader_cross_compilation_pipeline converts each permutation into deterministic jobs for ShaderCrossCompileTarget::VulkanSpirV, ShaderCrossCompileTarget::Dx12Dxil, and ShaderCrossCompileTarget::MetalMsl. Each job carries source metadata, defines, target key, compiler version, stage list, debug-symbol setting, and line-map preservation setting.

shader_compilation_parallelization schedules those jobs across local CPU workers and emits build-farm shards grouped by target and Horde pool. Local execution uses scoped worker threads and restores deterministic output ordering. Farm execution uses the same target/job metadata so local and remote compilers publish compatible cache artifacts.

The distribution plan in V2/ue/Build/Shaders/v2-shader-distribution.json fills shared DDC through ShaderCompileWorker-backed commandlets, bounded shard sizes, and platform-specific Horde pools.

Diagnostics And Debugging#

Compiler failures must be routed through shader_compilation_error_reporting. The reporter normalizes backend-specific diagnostics, classifies syntax, type, symbol, entry point, binding, unsupported-feature, and include-module failures, and maps generated lines back through include composition spans.

Debug symbol packages must preserve source path, entry point, target, stage, and line-map metadata. Do not strip line maps from development or CI builds that are used for shader triage.

When a shader fails CI, the first triage artifact should be the human-readable error report, followed by the composed source excerpt, debug-symbol package, and the exact ShaderCrossCompileJob metadata.

Cache Size And Regression Gates#

Shader cache quality has two separate gates:

  • shader_complexity_analysis estimates ALU operations, texture fetches, branch counts, and severity for expensive shader variants.
  • shader_binary_size_tracking records compiled artifact bytes, aggregates per-platform shader cache totals, exports baselines, and alerts on total, platform, or individual artifact growth.

The default binary-size gates use a 10 percent total cache regression threshold, a 10 percent per-platform cache regression threshold, a 65536 byte artifact growth warning threshold, and a 67108864 byte default platform cache budget.

PSO cache keys and shader cache keys are content-addressed. Changing source code, defines, entry point, backend, or compiler version must produce a new key and new size telemetry.

CI And Release Evidence#

The V2 CI workflow must validate every shader contract listed in Source Of Truth. Horde must publish required gates for source management, permutation generation, permutation pruning, cross-compilation, PSO caching, precompilation, hot reload, include modules, debugging symbols, complexity, variant usage, error reporting, parallelization, binary size tracking, and shader distribution.

Release evidence for shader pipeline changes must include:

  • the changed contract and checker,
  • the relevant Rust test filter when code changes,
  • the generated or validated JSON contract,
  • CI workflow and Horde gate validation,
  • shader distribution evidence when DDC or build-farm behavior changes,
  • binary-size baseline comparison when compiled output size can change,
  • a TODO evidence entry with the verification commands that passed.

Verification Commands#

Use targeted checks while iterating:

bash
python3 V2/ue/Tools/check-v2-maya-shader-pipeline-documentation.py
python3 V2/ue/Tools/check-v2-ci-workflow.py
python3 V2/ue/Tools/check-v2-buildgraph.py
python3 V2/tools/validate-v2-docs.py

Use Rust filters when implementation modules change:

bash
CARGO_BUILD_JOBS=1 cargo test --locked --manifest-path libs/maya/engine-core/Cargo.toml -p maya-renderer shader_cross_compilation_pipeline --lib
CARGO_BUILD_JOBS=1 cargo test --locked --manifest-path libs/maya/engine-core/Cargo.toml -p maya-renderer shader_compilation_parallelization --lib
CARGO_BUILD_JOBS=1 cargo test --locked --manifest-path libs/maya/engine-core/Cargo.toml -p maya-renderer shader_binary_size_tracking --lib