# ComfyUI Performance Tuning Guide

This guide covers performance optimization strategies for ComfyUI workflows in
the Oshun platform, including rate limiting, caching, scaling, and cost
optimization.

## Table of Contents

- [Performance Overview](#performance-overview)
- [Client-Side Optimization](#client-side-optimization)
- [Workflow Optimization](#workflow-optimization)
- [Provider Configuration](#provider-configuration)
- [Scaling Strategies](#scaling-strategies)
- [Caching](#caching)
- [Cost Optimization](#cost-optimization)
- [Monitoring and Metrics](#monitoring-and-metrics)
- [Troubleshooting Performance](#troubleshooting-performance)

---

## Performance Overview

### Key Performance Factors

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                    ComfyUI Performance Stack                                 │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│  Network Layer                                                               │
│  ├── API Latency: 50-200ms (request/response overhead)                      │
│  ├── Upload Time: Variable (input images, masks)                            │
│  └── Download Time: Variable (generated outputs)                            │
│                                                                              │
│  Queue Layer                                                                 │
│  ├── Queue Wait: 0-60s (depending on provider load)                         │
│  ├── Cold Start: 15-120s (if no warm workers)                               │
│  └── Worker Assignment: 1-5s                                                │
│                                                                              │
│  Execution Layer                                                             │
│  ├── Model Loading: 5-30s (cached: <1s)                                     │
│  ├── Sampling: 10-120s (varies by steps, resolution)                        │
│  ├── VAE Decode: 1-5s                                                       │
│  └── Post-processing: 1-10s (upscaling, effects)                            │
│                                                                              │
│  Total: 20s (cached, warm) to 5+ minutes (cold, complex workflow)           │
│                                                                              │
└─────────────────────────────────────────────────────────────────────────────┘
```

### Performance Targets

| Scenario              | Target Latency | Acceptable | Notes                       |
| --------------------- | -------------- | ---------- | --------------------------- |
| Preview (low quality) | <10s           | <20s       | Low steps, small resolution |
| Standard generation   | <30s           | <60s       | 30 steps, 1024x1024         |
| High quality          | <60s           | <120s      | 50+ steps, refinement       |
| Upscaling             | <30s           | <60s       | AI upscaler, 2-4x           |
| Complex workflow      | <120s          | <300s      | Multi-stage, ControlNet     |

---

## Client-Side Optimization

### Rate Limiting Configuration

Configure rate limiting to avoid hitting provider limits:

```typescript
import { RunComfyProvider } from '@oshun/comfy-provider';

const provider = new RunComfyProvider({
  apiKey: process.env.RUNCOMFY_API_KEY,

  // Rate limiting
  maxConcurrentJobs: 5, // Max parallel jobs
  queueSizeLimit: 100, // Max queued jobs
  enableQueue: true, // Enable local queue

  // Retry policy
  retryAttempts: 3, // Max retries
  retryBaseDelayMs: 1000, // Base delay (exponential backoff)

  // Timeouts
  defaultTimeoutMs: 300000, // 5 minute default
  pollingIntervalMs: 2000, // Poll every 2 seconds
  maxPollingAttempts: 150, // 5 min max polling
});
```

### Exponential Backoff

The provider uses exponential backoff with jitter:

```typescript
// Built-in retry calculation
function calculateDelay(attempt: number, baseDelay: number): number {
  // Exponential: 1s, 2s, 4s, ...
  const exponentialDelay = baseDelay * Math.pow(2, attempt);

  // Add jitter (0-50% of delay)
  const jitter = exponentialDelay * Math.random() * 0.5;

  // Cap at 30 seconds
  return Math.min(exponentialDelay + jitter, 30000);
}
```

### Circuit Breaker

Prevent cascade failures with circuit breaker pattern:

```typescript
interface CircuitBreakerConfig {
  failureThreshold: number; // Failures before opening (default: 5)
  successThreshold: number; // Successes to close (default: 2)
  timeout: number; // Time before half-open (default: 30000ms)
}

// The provider implements this internally
// Circuit states: CLOSED → OPEN → HALF_OPEN → CLOSED
```

### Connection Pooling

Reuse HTTP connections for better performance:

```typescript
// Provider automatically uses connection pooling
// For custom clients, use keep-alive:
import { Agent } from 'https';

const agent = new Agent({
  keepAlive: true,
  keepAliveMsecs: 30000,
  maxSockets: 10,
  maxFreeSockets: 5,
});

const client = new ComfyClient({
  httpAgent: agent,
});
```

---

## Workflow Optimization

### Reduce Sampling Steps

Balance quality vs. speed:

| Steps | Quality             | Time      | Use Case            |
| ----- | ------------------- | --------- | ------------------- |
| 10-15 | Draft               | Fast      | Previews, iteration |
| 20-30 | Good                | Medium    | Standard generation |
| 40-50 | High                | Slow      | Final output        |
| 50+   | Diminishing returns | Very slow | Rarely needed       |

```typescript
// Preview workflow
const previewConfig = {
  steps: 15,
  cfg: 7.0,
  sampler: 'euler', // Fastest
};

// Production workflow
const productionConfig = {
  steps: 30,
  cfg: 7.5,
  sampler: 'dpmpp_2m_sde', // Higher quality
};
```

### Choose Efficient Samplers

Sampler performance comparison:

| Sampler         | Steps Needed | Speed   | Quality       |
| --------------- | ------------ | ------- | ------------- |
| euler           | 30-40        | Fastest | Good          |
| euler_ancestral | 30-40        | Fast    | Good (varied) |
| dpm_2           | 20-30        | Medium  | Better        |
| dpmpp_2m        | 20-30        | Fast    | Better        |
| dpmpp_2m_sde    | 20-30        | Medium  | Best          |
| dpmpp_3m_sde    | 15-25        | Slow    | Best          |
| ddim            | 30-50        | Fast    | Good          |
| uni_pc          | 15-25        | Fast    | Better        |

```typescript
// For speed
{ sampler_name: 'euler', steps: 30 }

// For quality
{ sampler_name: 'dpmpp_2m_sde', scheduler: 'karras', steps: 25 }

// For consistency (deterministic)
{ sampler_name: 'ddim', steps: 30 }
```

### Optimize Resolution

Match resolution to model capabilities:

```typescript
// SDXL optimal resolutions (1024px base)
const SDXL_RESOLUTIONS = {
  square: { width: 1024, height: 1024 },
  portrait: { width: 832, height: 1216 },
  landscape: { width: 1216, height: 832 },
  wide: { width: 1344, height: 768 },
  tall: { width: 768, height: 1344 },
};

// SD 1.5 optimal resolutions (512px base)
const SD15_RESOLUTIONS = {
  square: { width: 512, height: 512 },
  portrait: { width: 448, height: 640 },
  landscape: { width: 640, height: 448 },
};

// For performance, generate at lower resolution, then upscale
const efficientWorkflow = {
  generateAt: { width: 768, height: 768 },
  upscaleTo: { width: 2048, height: 2048 },
};
```

### Batch Processing

Generate multiple images efficiently:

```typescript
// Single batch (more efficient)
{
  "class_type": "EmptyLatentImage",
  "inputs": {
    "width": 1024,
    "height": 1024,
    "batch_size": 4  // Generate 4 images in one pass
  }
}

// vs. 4 separate jobs (less efficient)
// Each job has overhead: queue, cold start, model loading
```

**Batch size recommendations:**

| GPU VRAM | Max Batch (1024px) | Max Batch (512px) |
| -------- | ------------------ | ----------------- |
| 8GB      | 1                  | 2-4               |
| 16GB     | 2-4                | 4-8               |
| 24GB     | 4-6                | 8-12              |
| 40GB     | 6-10               | 12-16             |

### Model Loading Optimization

Minimize model switching:

```typescript
// Bad: Different model per job (constant loading)
const jobs = [
  { model: 'modelA', prompt: '...' },
  { model: 'modelB', prompt: '...' },
  { model: 'modelA', prompt: '...' }, // Reloads modelA
];

// Good: Group by model (minimize loads)
const jobs = [
  { model: 'modelA', prompt: '...' },
  { model: 'modelA', prompt: '...' }, // Already loaded
  { model: 'modelB', prompt: '...' },
];

// Best: Batch with same model
const batch = {
  model: 'modelA',
  prompts: ['...', '...', '...'],
  batch_size: 3,
};
```

---

## Provider Configuration

### RunComfy Configuration

```typescript
const runcomfyConfig: RunComfyConfig = {
  apiKey: process.env.RUNCOMFY_API_KEY,

  // Performance tuning
  maxConcurrentJobs: 5,
  defaultTimeoutMs: 300000,
  pollingIntervalMs: 2000,
  maxPollingAttempts: 150,

  // Retry policy
  retryAttempts: 3,
  retryBaseDelayMs: 1000,

  // Queue management
  enableQueue: true,
  queueSizeLimit: 100,
};
```

### RunPod Configuration

```typescript
const runpodConfig: RunPodConfig = {
  apiKey: process.env.RUNPOD_API_KEY,
  endpointId: process.env.RUNPOD_ENDPOINT_ID,

  // Worker scaling
  minWorkers: 0, // Scale to zero when idle
  maxWorkers: 5, // Maximum concurrent workers
  idleTimeout: 120, // Seconds before worker shutdown

  // Request handling
  executionTimeout: 300,
  retryOnFail: true,
  maxRetries: 2,
};
```

### Multi-Provider Strategy

```typescript
import { ComfyCloudProvider, RoutingStrategy } from '@oshun/comfy-cloud';

const provider = new ComfyCloudProvider({
  providers: ['runcomfy', 'runpod', 'modal'],

  // Routing strategy
  routingStrategy: 'cost_optimized', // or 'latency_optimized', 'reliability'

  // Fallback configuration
  enableFailover: true,
  maxFailoverAttempts: 2,

  // Budget management
  dailyBudget: 50, // USD
  monthlyBudget: 500, // USD
  criticalJobReserve: 10, // % reserved for high-priority

  // Warm instance pool
  warmPoolSize: 2,
  warmPoolRefreshInterval: 300000, // 5 minutes
});
```

**Routing Strategies:**

| Strategy            | Optimizes For        | Trade-off               |
| ------------------- | -------------------- | ----------------------- |
| `cost_optimized`    | Lowest cost          | May have higher latency |
| `latency_optimized` | Fastest response     | Higher cost             |
| `reliability`       | Highest success rate | Variable cost           |
| `round_robin`       | Even distribution    | No optimization         |
| `failover_only`     | Primary provider     | Fallback on failure     |

---

## Scaling Strategies

### Horizontal Scaling

Scale workers based on queue depth:

```typescript
// RunPod auto-scaling configuration
const scalingPolicy = {
  // Queue-based scaling
  scaleUpThreshold: 5, // Queue depth to trigger scale-up
  scaleDownThreshold: 1, // Queue depth to trigger scale-down

  // Worker limits
  minWorkers: 0,
  maxWorkers: 10,

  // Cooldowns
  scaleUpCooldown: 60, // Seconds between scale-ups
  scaleDownCooldown: 300, // Seconds between scale-downs

  // Worker warm-up
  warmUpTime: 30, // Seconds for worker to be ready
};
```

### Warm Instance Pool

Keep instances warm to avoid cold starts:

```typescript
class WarmInstancePool {
  private pool: Map<string, InstanceInfo> = new Map();
  private minPoolSize: number = 2;

  async maintainPool(): Promise<void> {
    const currentSize = this.pool.size;

    if (currentSize < this.minPoolSize) {
      // Launch warm instances
      const needed = this.minPoolSize - currentSize;
      await this.launchInstances(needed);
    }

    // Refresh instances approaching timeout
    for (const [id, info] of this.pool) {
      if (info.idleTime > this.refreshThreshold) {
        await this.refreshInstance(id);
      }
    }
  }

  async getWarmInstance(): Promise<string | undefined> {
    for (const [id, info] of this.pool) {
      if (info.status === 'ready') {
        info.status = 'in_use';
        return id;
      }
    }
    return undefined; // No warm instance available
  }
}
```

### Request Prioritization

Prioritize critical jobs:

```typescript
interface PriorityQueue {
  high: JobRequest[]; // UI-blocking, user waiting
  normal: JobRequest[]; // Standard requests
  low: JobRequest[]; // Background tasks, batch jobs
}

class PrioritizedJobQueue {
  private queues: PriorityQueue = { high: [], normal: [], low: [] };

  enqueue(job: JobRequest): void {
    const priority = job.priority || 'normal';
    this.queues[priority].push(job);
  }

  dequeue(): JobRequest | undefined {
    // High priority first
    if (this.queues.high.length > 0) {
      return this.queues.high.shift();
    }
    if (this.queues.normal.length > 0) {
      return this.queues.normal.shift();
    }
    return this.queues.low.shift();
  }
}
```

---

## Caching

### Workflow Template Caching

Cache compiled workflows:

```typescript
const workflowCache = new Map<string, CompiledWorkflow>();
const CACHE_TTL = 3600000; // 1 hour

interface CompiledWorkflow {
  workflow: ComfyWorkflow;
  compiledAt: number;
  hash: string;
}

function getCachedWorkflow(templateId: string): ComfyWorkflow | null {
  const cached = workflowCache.get(templateId);

  if (cached && Date.now() - cached.compiledAt < CACHE_TTL) {
    return structuredClone(cached.workflow);
  }

  return null;
}

function cacheWorkflow(templateId: string, workflow: ComfyWorkflow): void {
  workflowCache.set(templateId, {
    workflow: structuredClone(workflow),
    compiledAt: Date.now(),
    hash: computeHash(workflow),
  });
}
```

### Model Availability Caching

Cache model discovery results:

```typescript
class ModelAvailabilityCache {
  private cache: Map<string, ModelInfo[]> = new Map();
  private ttl: number = 300000; // 5 minutes

  async getAvailableModels(provider: string): Promise<ModelInfo[]> {
    const cached = this.cache.get(provider);

    if (cached) {
      return cached;
    }

    const models = await this.fetchModels(provider);
    this.cache.set(provider, models);

    // Auto-expire
    setTimeout(() => this.cache.delete(provider), this.ttl);

    return models;
  }
}
```

### Output Caching

Cache generated outputs for identical requests:

```typescript
import { createHash } from 'crypto';

class OutputCache {
  private redis: Redis;
  private ttl: number = 86400; // 24 hours

  private computeKey(request: GenerationRequest): string {
    const normalized = {
      prompt: request.prompt.trim().toLowerCase(),
      negativePrompt: request.negativePrompt?.trim().toLowerCase(),
      model: request.model,
      width: request.width,
      height: request.height,
      steps: request.steps,
      cfg: request.cfg,
      seed: request.seed,
      sampler: request.sampler,
    };

    const hash = createHash('sha256')
      .update(JSON.stringify(normalized))
      .digest('hex');

    return `comfyui:output:${hash}`;
  }

  async get(request: GenerationRequest): Promise<string | null> {
    // Only cache deterministic requests (fixed seed)
    if (request.seed === -1) {
      return null;
    }

    const key = this.computeKey(request);
    return this.redis.get(key);
  }

  async set(request: GenerationRequest, outputUrl: string): Promise<void> {
    if (request.seed === -1) {
      return; // Don't cache random results
    }

    const key = this.computeKey(request);
    await this.redis.setex(key, this.ttl, outputUrl);
  }
}
```

---

## Cost Optimization

### GPU Selection

Choose appropriate GPU for workload:

| GPU       | VRAM | Cost/hr | Best For                  |
| --------- | ---- | ------- | ------------------------- |
| RTX 4090  | 24GB | $0.50   | SD 1.5, fast SDXL         |
| A40       | 48GB | $0.86   | SDXL, batch processing    |
| A100 40GB | 40GB | $1.58   | Flux, large batches       |
| A100 80GB | 80GB | $2.00   | Training, extreme batches |

```typescript
function selectGPU(requirements: JobRequirements): string {
  const { model, batchSize, resolution } = requirements;

  // Calculate VRAM needed
  const vramNeeded = estimateVRAM(model, batchSize, resolution);

  if (vramNeeded <= 8) return 'rtx_4090';
  if (vramNeeded <= 24) return 'a40';
  if (vramNeeded <= 40) return 'a100_40gb';
  return 'a100_80gb';
}

function estimateVRAM(
  model: string,
  batchSize: number,
  resolution: { width: number; height: number }
): number {
  const baseVRAM: Record<string, number> = {
    'sd_1.5': 4,
    'sd_2.1': 5,
    sdxl: 8,
    flux: 20,
  };

  const base = baseVRAM[model] || 8;
  const resolutionFactor =
    (resolution.width * resolution.height) / (1024 * 1024);
  const batchFactor = 1 + (batchSize - 1) * 0.5;

  return base * resolutionFactor * batchFactor;
}
```

### Idle Timeout Optimization

Configure timeouts based on traffic patterns:

```typescript
interface TrafficPattern {
  pattern: 'continuous' | 'bursty' | 'scheduled' | 'infrequent';
  idleTimeout: number; // seconds
  minWorkers: number;
}

const TRAFFIC_CONFIGS: Record<string, TrafficPattern> = {
  production: {
    pattern: 'continuous',
    idleTimeout: 120,
    minWorkers: 1,
  },
  staging: {
    pattern: 'bursty',
    idleTimeout: 60,
    minWorkers: 0,
  },
  development: {
    pattern: 'infrequent',
    idleTimeout: 30,
    minWorkers: 0,
  },
};
```

### Cost Tracking

```typescript
interface CostRecord {
  jobId: string;
  provider: string;
  gpu: string;
  executionTimeSeconds: number;
  cost: number;
  timestamp: Date;
}

class CostTracker {
  private records: CostRecord[] = [];

  async recordJob(job: CompletedJob): Promise<void> {
    const pricePerSecond = GPU_PRICING[job.gpu];
    const cost = job.executionTimeSeconds * pricePerSecond;

    const record: CostRecord = {
      jobId: job.id,
      provider: job.provider,
      gpu: job.gpu,
      executionTimeSeconds: job.executionTimeSeconds,
      cost,
      timestamp: new Date(),
    };

    this.records.push(record);
    await this.persistRecord(record);
  }

  getDailyCost(): number {
    const today = new Date().toDateString();
    return this.records
      .filter((r) => r.timestamp.toDateString() === today)
      .reduce((sum, r) => sum + r.cost, 0);
  }

  getMonthlyCost(): number {
    const thisMonth = new Date().getMonth();
    return this.records
      .filter((r) => r.timestamp.getMonth() === thisMonth)
      .reduce((sum, r) => sum + r.cost, 0);
  }
}
```

---

## Monitoring and Metrics

### Key Metrics

```typescript
interface ComfyUIMetrics {
  // Latency
  totalLatencyMs: number;
  queueTimeMs: number;
  executionTimeMs: number;
  networkTimeMs: number;

  // Throughput
  jobsPerMinute: number;
  imagesPerMinute: number;

  // Success rates
  successRate: number;
  retryRate: number;
  failureRate: number;

  // Resource usage
  activeWorkers: number;
  queueDepth: number;
  warmInstances: number;

  // Cost
  costPerJob: number;
  dailyCost: number;
  projectedMonthlyCost: number;
}
```

### Prometheus Metrics

```typescript
import { Registry, Counter, Histogram, Gauge } from 'prom-client';

const registry = new Registry();

// Job metrics
const jobsTotal = new Counter({
  name: 'comfyui_jobs_total',
  help: 'Total ComfyUI jobs',
  labelNames: ['provider', 'status', 'model'],
  registers: [registry],
});

const jobDuration = new Histogram({
  name: 'comfyui_job_duration_seconds',
  help: 'ComfyUI job duration',
  labelNames: ['provider', 'model'],
  buckets: [5, 10, 20, 30, 60, 120, 300],
  registers: [registry],
});

const queueDepth = new Gauge({
  name: 'comfyui_queue_depth',
  help: 'Current queue depth',
  labelNames: ['provider'],
  registers: [registry],
});

// Usage
function recordJobCompletion(job: CompletedJob): void {
  jobsTotal.labels(job.provider, 'success', job.model).inc();
  jobDuration.labels(job.provider, job.model).observe(job.duration);
}
```

### Health Checks

```typescript
interface HealthCheckResult {
  healthy: boolean;
  providers: Record<string, ProviderHealth>;
  queue: QueueHealth;
  cache: CacheHealth;
}

interface ProviderHealth {
  status: 'healthy' | 'degraded' | 'unhealthy';
  latencyMs: number;
  errorRate: number;
  activeWorkers: number;
  lastError?: string;
}

async function healthCheck(): Promise<HealthCheckResult> {
  const providers: Record<string, ProviderHealth> = {};

  for (const provider of ['runcomfy', 'runpod']) {
    try {
      const start = Date.now();
      const status = await checkProvider(provider);
      const latency = Date.now() - start;

      providers[provider] = {
        status: status.healthy ? 'healthy' : 'degraded',
        latencyMs: latency,
        errorRate: status.errorRate,
        activeWorkers: status.activeWorkers,
      };
    } catch (error) {
      providers[provider] = {
        status: 'unhealthy',
        latencyMs: -1,
        errorRate: 1,
        activeWorkers: 0,
        lastError: error.message,
      };
    }
  }

  return {
    healthy: Object.values(providers).some((p) => p.status === 'healthy'),
    providers,
    queue: await checkQueue(),
    cache: await checkCache(),
  };
}
```

---

## Troubleshooting Performance

### High Latency

**Symptoms:** Jobs taking longer than expected

**Diagnosis:**

```typescript
// Break down latency components
const metrics = await provider.getJobMetrics(jobId);

console.log('Queue time:', metrics.queueTimeMs);
console.log('Cold start:', metrics.coldStartMs);
console.log('Model load:', metrics.modelLoadMs);
console.log('Execution:', metrics.executionMs);
console.log('Upload:', metrics.uploadMs);
```

**Solutions:**

| Component  | Issue              | Solution                  |
| ---------- | ------------------ | ------------------------- |
| Queue time | High demand        | Scale workers, prioritize |
| Cold start | No warm instances  | Increase min workers      |
| Model load | Frequent switching | Batch by model            |
| Execution  | Too many steps     | Optimize workflow         |
| Upload     | Large files        | Compress, use CDN         |

### High Error Rate

**Symptoms:** Jobs failing frequently

**Diagnosis:**

```typescript
const analytics = await provider.getAnalytics();

console.log('Error distribution:');
for (const [code, count] of Object.entries(analytics.errorDistribution)) {
  console.log(`  ${code}: ${count}`);
}
```

**Common Errors and Solutions:**

| Error                  | Cause             | Solution                 |
| ---------------------- | ----------------- | ------------------------ |
| `TIMEOUT`              | Job too slow      | Reduce steps, resolution |
| `RATE_LIMITED`         | Too many requests | Reduce concurrency       |
| `INSUFFICIENT_CREDITS` | Budget exhausted  | Add credits              |
| `MODEL_NOT_FOUND`      | Invalid model     | Check model availability |
| `WORKFLOW_INVALID`     | Bad workflow      | Validate before submit   |
| `OOM`                  | Out of memory     | Reduce batch size        |

### Memory Issues

**Symptoms:** OOM errors, worker crashes

**Solutions:**

```typescript
// Reduce memory usage
const lowMemoryConfig = {
  // Smaller batches
  batchSize: 1,

  // Lower resolution
  width: 768,
  height: 768,

  // Use FP16 VAE
  vae: 'sdxl_vae_fp16.safetensors',

  // Limit LoRAs
  maxLoras: 2,

  // Use tiled VAE for large images
  tiledVAE: true,
  tileSize: 512,
};
```

### Queue Backlog

**Symptoms:** Long wait times, growing queue

**Solutions:**

```typescript
// Scale up workers
await provider.scaleWorkers({
  minWorkers: 2,
  maxWorkers: 10,
});

// Enable priority queue
const job = await provider.submitJob({
  workflow,
  priority: 1, // Higher priority
});

// Reject low-priority jobs during high load
if (queueDepth > threshold) {
  throw new Error('Service busy, try again later');
}
```

---

## Related Documentation

- [Workflow Documentation](./workflows.md)
- [Node Reference](./node-reference.md)
- [Custom Node Development](./custom-nodes.md)
- [RunComfy API Client](../../domains/lilith/extras/comfyui/runcomfy-api-client.md)