# GPU Jobs Routing Rules Configuration

This guide covers the flexible routing rules system for GPU job dispatch. The
routing rules allow you to configure job routing behavior without code changes.

## Table of Contents

- [Overview](#overview)
- [Configuration Structure](#configuration-structure)
- [Rule Definition](#rule-definition)
- [Provider Configuration](#provider-configuration)
- [Loading Configuration](#loading-configuration)
- [Examples](#examples)
- [Best Practices](#best-practices)

## Overview

The routing rules system provides:

- **File-based configuration**: JSON configuration files that can be modified
  without code changes
- **Rule priority**: Rules are evaluated in priority order, with higher priority
  rules taking precedence
- **Match conditions**: Filter rules based on job type, priority, time, cost,
  GPU requirements, and custom expressions
- **Provider selection**: Specify provider preferences, exclusions, and capacity
  thresholds
- **Runtime updates**: Configuration can be reloaded without restarting the
  service

## Configuration Structure

```typescript
interface RoutingRulesConfig {
  version: string; // Configuration version (e.g., "1.0.0")
  name: string; // Configuration name
  description?: string; // Optional description
  updatedAt?: string; // Last updated timestamp

  rules: RoutingRule[]; // Routing rules (evaluated in priority order)
  providers: ProviderConfig[]; // Provider-specific settings
  fallback: FallbackConfig; // Fallback configuration
  globalSettings: GlobalSettings; // Global settings
  featureFlags: Record<string, boolean>; // Feature flags
}
```

## Rule Definition

Each routing rule has match conditions and routing actions:

### Basic Rule Structure

```json
{
  "id": "unique-rule-id",
  "name": "Human-readable name",
  "description": "What this rule does",
  "priority": 100,
  "enabled": true,

  // Match conditions (all must match)
  "jobTypes": ["texture-upscale", "gaussian-splatting"],
  "priorities": ["high", "urgent"],

  // Routing actions
  "providers": ["runpod-serverless", "modal"],
  "excludeProviders": ["aws-ecs-spot"],
  "preferSpeed": true
}
```

### Match Conditions

| Condition         | Description                    | Example                                 |
| ----------------- | ------------------------------ | --------------------------------------- |
| `jobTypes`        | Job types this rule applies to | `["texture-upscale", "blender-render"]` |
| `priorities`      | Job priorities to match        | `["high", "urgent"]`                    |
| `timeRanges`      | Time-based activation          | See time ranges below                   |
| `costConditions`  | Cost thresholds                | `{"maxCost": 1.0}`                      |
| `gpuRequirements` | GPU requirements               | `{"minVRAM": 24}`                       |
| `metadataMatches` | Custom metadata matches        | `{"customerTier": "enterprise"}`        |
| `matchExpression` | Custom JavaScript expression   | `"job.priority === 'urgent'"`           |

### Time Range Configuration

```json
{
  "timeRanges": [
    {
      "startHour": 9,
      "endHour": 18,
      "daysOfWeek": [1, 2, 3, 4, 5],
      "timezone": "America/New_York"
    }
  ]
}
```

- `startHour`/`endHour`: 0-23 (24-hour format)
- `daysOfWeek`: 0=Sunday, 6=Saturday
- `timezone`: IANA timezone string

### Routing Actions

| Action               | Description                       | Example                          |
| -------------------- | --------------------------------- | -------------------------------- |
| `providers`          | Preferred providers in order      | `["runpod-serverless", "modal"]` |
| `excludeProviders`   | Providers to exclude              | `["aws-ecs-spot"]`               |
| `preferSpot`         | Prefer spot instances             | `true`                           |
| `preferSpeed`        | Prioritize speed over cost        | `true`                           |
| `maxCostOverride`    | Override max cost                 | `0.50`                           |
| `priorityBoost`      | Adjust job priority (-100 to 100) | `10`                             |
| `addTags`            | Tags to add to matched jobs       | `["enterprise", "fast-track"]`   |
| `capacityThresholds` | Capacity requirements             | See below                        |

## Provider Configuration

Configure provider-specific settings:

```json
{
  "providers": [
    {
      "provider": "runpod-serverless",
      "enabled": true,
      "weight": 100,
      "capacityThresholds": {
        "maxQueueDepth": 100,
        "maxWaitTimeSeconds": 300,
        "minAvailablePercent": 5
      },
      "costThresholds": {
        "maxCostPerHour": 2.0
      },
      "allowedJobTypes": ["texture-upscale", "gaussian-splatting"],
      "deniedJobTypes": ["video-encode"],
      "geoConfig": {
        "preferredRegions": ["us-east-1"]
      }
    }
  ]
}
```

### Provider Settings

| Setting               | Description                   |
| --------------------- | ----------------------------- |
| `weight`              | Load balancing weight (0-100) |
| `capacityThresholds`  | When to skip provider         |
| `costThresholds`      | Cost limits                   |
| `allowedJobTypes`     | Whitelist of job types        |
| `deniedJobTypes`      | Blacklist of job types        |
| `availabilityWindows` | Time-based availability       |
| `geoConfig`           | Geographic preferences        |

## Loading Configuration

### From File

```typescript
import { RoutingRulesLoader, RoutingRulesEngine } from '@yemaya/gpu-jobs';

// Load from JSON file
const config = RoutingRulesLoader.loadFromFile('./routing-config.json');
const engine = new RoutingRulesEngine(config);
```

### From Environment Variable

```bash
export GPU_ROUTING_CONFIG='{"version":"1.0.0",...}'
```

```typescript
const config = RoutingRulesLoader.loadFromEnv('GPU_ROUTING_CONFIG');
```

### With Fallback Chain

```typescript
const config = RoutingRulesLoader.load({
  configPath: './routing-config.json',
  envVar: 'GPU_ROUTING_CONFIG',
  defaultConfig: DEFAULT_ROUTING_RULES_CONFIG,
});
```

## Examples

### High Priority Fast Routing

Route urgent jobs to fastest providers:

```json
{
  "id": "urgent-fast",
  "name": "Urgent jobs fast routing",
  "priority": 100,
  "enabled": true,
  "priorities": ["urgent"],
  "providers": ["runpod-serverless", "modal"],
  "preferSpeed": true
}
```

### Cost Optimization for Low Priority

Route low priority jobs to cheapest providers:

```json
{
  "id": "low-priority-cheap",
  "name": "Low priority cost optimization",
  "priority": 40,
  "enabled": true,
  "priorities": ["low"],
  "providers": ["aws-ecs-spot"],
  "preferSpot": true,
  "maxCostOverride": 0.25
}
```

### Time-Based Routing

Use spot instances during off-peak hours:

```json
{
  "id": "off-peak-spot",
  "name": "Off-peak spot instances",
  "priority": 50,
  "enabled": true,
  "timeRanges": [
    {
      "startHour": 22,
      "endHour": 6,
      "daysOfWeek": [0, 1, 2, 3, 4, 5, 6]
    }
  ],
  "providers": ["aws-ecs-spot"],
  "preferSpot": true
}
```

### Large VRAM Requirements

Route high-memory jobs to appropriate GPUs:

```json
{
  "id": "large-vram",
  "name": "Large VRAM to A100",
  "priority": 90,
  "enabled": true,
  "gpuRequirements": {
    "minVRAM": 40
  },
  "providers": ["runpod-serverless"]
}
```

### Enterprise Customer Routing

Dedicated routing for enterprise customers:

```json
{
  "id": "enterprise",
  "name": "Enterprise dedicated routing",
  "priority": 110,
  "enabled": true,
  "metadataMatches": {
    "customerTier": "enterprise"
  },
  "providers": ["aws-ecs-ondemand"],
  "preferSpeed": true,
  "addTags": ["enterprise", "sla-guaranteed"]
}
```

## Best Practices

### 1. Priority Planning

- **100+**: Critical rules (enterprise, urgent)
- **80-99**: Job type specific rules
- **60-79**: Time-based rules
- **40-59**: Cost optimization rules
- **<40**: Catch-all rules

### 2. Rule Overlap

Rules can overlap - all matching rules contribute to the decision. Use
`excludeProviders` to prevent unwanted matches.

### 3. Capacity Thresholds

Set realistic capacity thresholds:

```json
{
  "capacityThresholds": {
    "maxQueueDepth": 100,
    "maxWaitTimeSeconds": 300,
    "minAvailablePercent": 10
  }
}
```

### 4. Testing Changes

1. Use `dryRun` in API to test routing decisions
2. Enable `debugLogging` in `globalSettings` for detailed logs
3. Use A/B testing for significant changes

### 5. Version Control

Keep routing configuration in version control:

```
configs/
  routing/
    production.json
    staging.json
    development.json
```

### 6. Monitoring

Monitor these metrics:

- Rule match rates
- Provider selection distribution
- Fallback usage rate
- Cache hit rates

## Related Documentation

- [GPU Jobs README](./README.md)
- [Cost Model](./cost-model.md)
- [Provider Configuration](./providers.md)
- [Sample Configuration](./routing-config-example.json)
