Eve library

Workflow Development Guide

Guide

Revision 1

On this page

Overview#

This guide covers best practices for developing ComfyUI workflows for the Lilith platform. It includes workflow design patterns, optimization techniques, testing strategies, and deployment procedures.

ComfyUI Basics#

Workflow Structure#

A ComfyUI workflow consists of:

  1. Nodes - Processing units with inputs and outputs
  2. Links - Connections between node outputs and inputs
  3. Groups - Visual organization of related nodes
  4. Widgets - Node parameter controls

Core Node Types#

Category Node Purpose
Loading CheckpointLoaderSimple Load SD model
LoraLoader Apply LoRA weights
LoadImage Import image
Encoding CLIPTextEncode Encode text prompt
VAEEncode Encode image to latent
Sampling KSampler Generate from latent
KSamplerAdvanced Advanced sampling options
Decoding VAEDecode Decode latent to image
Output SaveImage Save to disk
PreviewImage Preview in UI

Workflow JSON Format#

json
{
  "nodes": [
    {
      "id": 1,
      "type": "CheckpointLoaderSimple",
      "pos": [100, 100],
      "inputs": [],
      "outputs": [
        { "name": "MODEL", "type": "MODEL", "links": [1] },
        { "name": "CLIP", "type": "CLIP", "links": [2, 3] },
        { "name": "VAE", "type": "VAE", "links": [4] }
      ],
      "widgets_values": ["sd_xl_base_1.0.safetensors"]
    }
    // ... more nodes
  ],
  "links": [
    [1, 1, 0, 3, 0, "MODEL"] // [link_id, from_node, output_idx, to_node, input_idx, type]
    // ... more links
  ]
}

Design Patterns#

1. Text-to-Image Base Pattern#

Standard workflow for generating images from text prompts:

text
CheckpointLoader → CLIPTextEncode (positive) ─┐
              └─→ CLIPTextEncode (negative) ─┼─→ KSampler → VAEDecode → SaveImage
              └─→ EmptyLatentImage ──────────┘
              └─→ VAEDecode ─────────────────────┘

Key nodes:

  • Node 4: CheckpointLoaderSimple
  • Node 6: CLIPTextEncode (positive prompt)
  • Node 7: CLIPTextEncode (negative prompt)
  • Node 5: EmptyLatentImage
  • Node 3: KSampler
  • Node 8: VAEDecode
  • Node 9: SaveImage

2. Image-to-Image Pattern#

Transform existing images:

text
LoadImage → VAEEncode ──────────────────────┐
CheckpointLoader → CLIPTextEncode (pos) ───┼─→ KSampler → VAEDecode → SaveImage
               └─→ CLIPTextEncode (neg) ───┘

Key difference: Replace EmptyLatentImage with VAEEncode(LoadImage)

3. Inpainting Pattern#

Edit specific regions of an image:

text
LoadImage ─────────────────────────────────┐
LoadMask ──────────────────────────────────┼─→ SetLatentNoiseMask → KSampler → ...
CheckpointLoader → VAEEncode(LoadImage) ───┘

4. Upscaling Pattern#

Increase image resolution:

text
LoadImage → UpscaleModelLoader → ImageUpscaleWithModel → SaveImage

Or with latent upscaling:

text
... → KSampler → LatentUpscale → KSampler (high-res fix) → VAEDecode → ...

5. ControlNet Pattern#

Guide generation with structural input:

text
LoadImage → ControlNetApply ─────────────────────┐
ControlNetLoader ────────────────────────────────┤
CheckpointLoader → CLIPTextEncode ───────────────┼─→ KSampler → ...
              └─→ EmptyLatentImage ──────────────┘

6. LoRA Enhancement Pattern#

Apply style/character LoRAs:

text
CheckpointLoader → LoraLoader → LoraLoader → CLIPTextEncode → ...
                            └─→ ...

Stack multiple LoRAs with reduced strengths (0.5-0.7 each).

Lilith Platform Patterns#

Sacred Art Generation#

text
EmptyLatentImage ────────────────────────────────────┐
CheckpointLoader → LoraLoader(spiritual_art) ────────┤
                └─→ CLIPTextEncode(mandala prompt) ──┼─→ KSampler → VAEDecode
                └─→ CLIPTextEncode(negative) ────────┘     │
                                                           ▼
                                          LilithWatermark → LilithAssetSaver
                                                           │
                                          LilithMetadataGenerator

Consciousness Visualization#

text
LilithBrainwaveVisualizer ──────────┐
LilithChakraEnergyMapper ───────────┼─→ ImageBlend → LilithAssetSaver
LilithAuraFieldGenerator ───────────┘

Avatar Generation with Consistency#

text
LoadImage(reference) → IPAdapter ────────────────────┐
CheckpointLoader → CLIPTextEncode(avatar prompt) ────┼─→ KSampler → VAEDecode
                └─→ EmptyLatentImage ────────────────┘

Workflow Optimization#

1. Node Placement Best Practices#

text
GOOD: Linear flow, clear data paths
┌────────────────────────────────────────┐
│ Loaders → Encoders → Sampler → Output  │
└────────────────────────────────────────┘

BAD: Crossed wires, confusing layout
┌────────────────────────────────────────┐
│     ┌─────────────────┐                │
│ ────┼──────────┬──────┤────            │
│     └──────────┘      │                │
└────────────────────────────────────────┘

2. Parameter Optimization#

Sampler Settings:

text
Production Quality:
- Steps: 25-35
- CFG: 7.0-8.5
- Sampler: dpmpp_2m_karras
- Scheduler: karras

Fast Preview:
- Steps: 12-15
- CFG: 7.0
- Sampler: euler
- Scheduler: normal

High Quality:
- Steps: 40-50
- CFG: 7.0-8.0
- Sampler: dpmpp_2m_sde_karras
- Scheduler: karras

Resolution Guidelines:

text
SD 1.5:
- Native: 512x512
- Max recommended: 768x768

SDXL:
- Native: 1024x1024
- Portrait: 832x1216 or 896x1152
- Landscape: 1216x832 or 1152x896
- Max recommended: 1536x1536

3. Memory Optimization#

Reduce VRAM usage:

  1. Lower batch size
  2. Use VAE tiling for large images
  3. Enable attention slicing
  4. Use fp16 precision
text
For large images (>1536px):
- Add VAETileEncode/VAETileDecode
- Reduce batch to 1
- Consider processing in tiles

4. Speed Optimization#

Faster generation:

  1. Use efficient samplers (euler, ddim)
  2. Reduce steps (15-20 for previews)
  3. Generate at lower res, upscale after
  4. Pre-load models (deployment config)

Workflow Parameters Schema#

Defining Overridable Parameters#

Document parameters that can be modified at runtime:

yaml
# workflow-params.yaml
parameters:
  - name: prompt
    nodeId: '6'
    inputSlot: text
    type: string
    required: true
    description: Main generation prompt

  - name: negative_prompt
    nodeId: '7'
    inputSlot: text
    type: string
    required: false
    default: 'blurry, low quality'
    description: Negative prompt

  - name: seed
    nodeId: '3'
    inputSlot: seed
    type: seed
    required: false
    default: -1
    description: Random seed (-1 for random)

  - name: steps
    nodeId: '3'
    inputSlot: steps
    type: integer
    required: false
    default: 30
    min: 10
    max: 150

  - name: cfg
    nodeId: '3'
    inputSlot: cfg
    type: float
    required: false
    default: 7.5
    min: 1.0
    max: 30.0

  - name: width
    nodeId: '5'
    inputSlot: width
    type: integer
    required: false
    default: 1024
    min: 512
    max: 2048
    step: 64

  - name: height
    nodeId: '5'
    inputSlot: height
    type: integer
    required: false
    default: 1024
    min: 512
    max: 2048
    step: 64

Testing Workflows#

1. Local Testing#

Test workflow in local ComfyUI before deployment:

bash
# Start local ComfyUI
cd ComfyUI
python main.py

# Load workflow and test with various parameters

2. Parameter Boundary Testing#

Test edge cases:

typescript
// Test minimum values
await testWorkflow({ width: 512, height: 512, steps: 10 });

// Test maximum values
await testWorkflow({ width: 2048, height: 2048, steps: 150 });

// Test default values
await testWorkflow({});

// Test invalid values (should fail gracefully)
await testWorkflow({ width: 100, steps: -5 });

3. Visual Quality Testing#

Create test suite for visual consistency:

typescript
const TEST_PROMPTS = [
  'sacred mandala, golden, intricate patterns',
  'chakra visualization, rainbow colors, ethereal',
  'fractal pattern, mandelbrot, cosmic',
  'buddhist thangka, tibetan art, traditional',
];

async function visualQualityTest(deploymentId: string) {
  const results = [];

  for (const prompt of TEST_PROMPTS) {
    const result = await client.runInference(deploymentId, {
      '6': { inputs: { text: prompt } },
      '3': { inputs: { seed: 42 } }, // Fixed seed for reproducibility
    });

    results.push({
      prompt,
      outputs: result.outputs,
      processingTime: result.processingTimeMs,
    });
  }

  // Manual review or automated image comparison
  return results;
}

4. Performance Testing#

Benchmark workflow performance:

typescript
async function performanceTest(deploymentId: string, iterations: number = 10) {
  const times: number[] = [];

  for (let i = 0; i < iterations; i++) {
    const start = Date.now();
    await client.runInference(deploymentId, standardOverrides);
    times.push(Date.now() - start);
  }

  return {
    avg: times.reduce((a, b) => a + b) / times.length,
    min: Math.min(...times),
    max: Math.max(...times),
    p95: times.sort()[Math.floor(times.length * 0.95)],
  };
}

Deployment Checklist#

Pre-Deployment#

  • Workflow tested locally with all parameter combinations
  • All custom nodes included and tested
  • Required models specified and available on RunComfy
  • Parameter schema documented
  • Performance benchmarks completed
  • Memory usage verified for target GPU

Deployment Configuration#

typescript
const deployment: DeploymentConfiguration = {
  deploymentId: 'sacred-mandala-v2',
  name: 'Sacred Mandala Generator v2',
  description: 'Generate sacred mandala patterns with configurable symmetry',
  category: WorkflowCategory.SACRED_ART,
  version: '2.0.0',
  workflowChecksum: 'sha256:abc123...',
  complexity: ComplexityLevel.MEDIUM,
  gpuType: 'A5000',
  estimatedProcessingTime: 45,
  maxTimeout: 120,
  costPerRun: 5,
  active: false, // Start inactive
  inputs: [
    // Parameter schemas
  ],
  outputs: [
    {
      name: 'mandala_image',
      nodeId: '9',
      outputSlot: 0,
      type: OutputType.IMAGE,
      mimeType: 'image/png',
      description: 'Generated mandala image',
    },
  ],
  customNodes: ['Lilith-Sacred-Geometry', 'Lilith-Platform-Integration'],
  requiredModels: [
    'sd_xl_base_1.0.safetensors',
    'spiritual_art_v2.safetensors',
  ],
  tags: ['mandala', 'sacred-geometry', 'spiritual', 'sdxl'],
  createdAt: new Date().toISOString(),
  updatedAt: new Date().toISOString(),
  lilithMetadata: {
    promptTemplateId: 'sacred-mandala',
    defaultResolution: 'square-1024',
    isPremium: false,
    requiredTier: 'basic',
  },
};

Post-Deployment#

  • Run smoke tests on deployed workflow
  • Verify health checks passing
  • Monitor first 100 jobs for errors
  • Activate deployment after validation
  • Update documentation

Review Checklist#

Before submitting workflow for review:

Technical Review#

  • Workflow follows established patterns
  • Node IDs are consistent and documented
  • All inputs have sensible defaults
  • All inputs have validation constraints
  • Output nodes are properly connected
  • No orphaned or unused nodes

Performance Review#

  • Processing time within expected range
  • VRAM usage appropriate for target GPU
  • Batch processing works correctly
  • Different resolutions tested

Quality Review#

  • Output quality meets standards
  • Consistent results with fixed seed
  • Edge cases handled gracefully
  • Error messages are helpful

Documentation Review#

  • Parameter schema complete
  • Usage examples provided
  • Known limitations documented
  • Version changelog maintained

Common Issues & Solutions#

Issue: Workflow fails with OOM#

Solution: Reduce resolution, batch size, or use VAE tiling

Issue: Inconsistent outputs#

Solution: Check seed handling, verify deterministic samplers

Issue: Slow generation#

Solution: Reduce steps, use faster sampler, optimize resolution

Issue: Model not found#

Solution: Verify model name matches exactly, check RunComfy model list

Issue: Custom node error#

Solution: Check node version compatibility, verify installation