# GPU Jobs Cost Model

This guide covers the cost estimation system, pricing models, budget management,
and cost optimization strategies.

## Table of Contents

- [Overview](#overview)
- [Pricing Data](#pricing-data)
- [Cost Estimation Algorithm](#cost-estimation-algorithm)
- [Cost Breakdown](#cost-breakdown)
- [Budget Management](#budget-management)
- [Cost Optimization](#cost-optimization)
- [Reporting & Analytics](#reporting--analytics)
- [API Reference](#api-reference)

## Overview

The GPU Jobs cost estimation system provides accurate cost predictions before
job submission, enabling informed decisions about provider selection and budget
management.

### Key Features

- **Pre-dispatch estimation**: Get cost estimates before submitting jobs
- **Multi-provider comparison**: Compare costs across all configured providers
- **Budget enforcement**: Automatic rejection of jobs exceeding budgets
- **Real-time tracking**: Track actual vs. estimated costs
- **Cost optimization recommendations**: Suggestions for reducing GPU spend

## Pricing Data

### GPU Pricing Table (as of 2024)

| GPU       | RunPod Serverless | RunPod Pod | Modal    | AWS Spot | AWS On-Demand |
| --------- | ----------------- | ---------- | -------- | -------- | ------------- |
| RTX 3090  | $1.58/hr          | $0.69/hr   | -        | -        | -             |
| RTX 4090  | $2.48/hr          | $1.14/hr   | -        | -        | -             |
| T4        | -                 | -          | $0.59/hr | $0.16/hr | $0.53/hr      |
| L4        | -                 | -          | $1.10/hr | -        | -             |
| A10       | $2.09/hr          | $1.00/hr   | $1.10/hr | $0.30/hr | $1.01/hr      |
| A100 40GB | $5.00/hr          | $3.19/hr   | $3.88/hr | $2.46/hr | $8.19/hr      |
| A100 80GB | $7.99/hr          | $5.19/hr   | $5.63/hr | $3.69/hr | $12.29/hr     |
| H100      | -                 | $14.00/hr  | $8.12/hr | $9.83/hr | $32.77/hr     |

### Data Transfer Costs

| Provider | Ingress | Egress   |
| -------- | ------- | -------- |
| RunPod   | Free    | $0.05/GB |
| Modal    | Free    | $0.05/GB |
| AWS      | Free    | $0.09/GB |
| Vast.ai  | Varies  | Varies   |
| Local    | Free    | Free     |

### Storage Costs

| Provider   | Persistent Storage | Temporary |
| ---------- | ------------------ | --------- |
| RunPod Pod | $0.10/GB/month     | Included  |
| AWS EBS    | $0.08/GB/month     | Included  |
| Modal      | Free (ephemeral)   | N/A       |

## Cost Estimation Algorithm

### Basic Formula

```
Total Cost = Compute Cost + Data Transfer Cost + Platform Fee

Where:
  Compute Cost = (Duration × GPU Price) × GPU Count
  Data Transfer Cost = (Input Size × Ingress Rate) + (Output Size × Egress Rate)
  Platform Fee = Compute Cost × Platform Fee Rate
```

### Duration Estimation

Duration is estimated based on job type and GPU:

```typescript
interface DurationEstimate {
  baseSeconds: number; // Base processing time
  vramMultiplier: number; // Higher VRAM = faster
  gpuGenerationMultiplier: number; // Newer GPU = faster
}

const durationFactors: Record<GPUJobType, DurationEstimate> = {
  'texture-upscale': {
    baseSeconds: 30,
    vramMultiplier: 0.8, // 20% faster per 8GB VRAM
    gpuGenerationMultiplier: 0.9, // 10% faster per generation
  },
  'gaussian-splatting': {
    baseSeconds: 1800, // 30 minutes base
    vramMultiplier: 0.7, // 30% faster with more VRAM
    gpuGenerationMultiplier: 0.85,
  },
  'blender-render': {
    baseSeconds: 120, // Per frame base
    vramMultiplier: 0.75,
    gpuGenerationMultiplier: 0.8,
  },
  // ... other job types
};
```

### GPU Selection Algorithm

```typescript
function selectGPU(
  requirements: GPURequirements,
  preferCheapest: boolean = true
): GPUType {
  // 1. Filter GPUs meeting VRAM requirement
  const eligible = GPU_TYPES.filter((gpu) => gpu.vram >= requirements.minVRAM);

  // 2. Filter by feature requirements (tensor cores, etc.)
  const compatible = eligible.filter((gpu) =>
    meetsFeatureRequirements(gpu, requirements)
  );

  // 3. Apply preference
  if (
    requirements.preferredGPU &&
    compatible.includes(requirements.preferredGPU)
  ) {
    return requirements.preferredGPU;
  }

  // 4. Select based on strategy
  if (preferCheapest) {
    return compatible.sort((a, b) => a.pricePerHour - b.pricePerHour)[0];
  } else {
    return compatible.sort((a, b) => b.performance - a.performance)[0];
  }
}
```

### Cold Start Estimation

Cold start times are factored into duration estimates:

| Provider          | GPU Type | Average Cold Start |
| ----------------- | -------- | ------------------ |
| RunPod Serverless | Any      | 3-5 seconds        |
| RunPod Pod        | Any      | 30-60 seconds      |
| Modal             | T4/L4    | 5-10 seconds       |
| Modal             | A10/A100 | 10-15 seconds      |
| Modal             | H100     | 15-30 seconds      |
| AWS ECS           | Any      | 60-120 seconds     |
| Vast.ai           | Any      | 30-120 seconds     |
| Local             | N/A      | 0 seconds          |

## Cost Breakdown

### Detailed Breakdown Structure

```typescript
interface CostBreakdown {
  // Compute costs
  compute: {
    gpuTime: number; // GPU compute time cost
    cpuTime: number; // CPU compute time cost (if any)
    coldStart: number; // Cold start overhead cost
  };

  // Data transfer
  transfer: {
    ingress: number; // Input data download cost
    egress: number; // Output data upload cost
    internal: number; // Internal transfer cost
  };

  // Storage
  storage: {
    persistent: number; // Persistent volume cost
    temporary: number; // Temporary storage cost
  };

  // Platform fees
  platform: {
    baseFee: number; // Provider platform fee
    premiumFee: number; // Priority/SLA fee (if applicable)
  };

  // Total
  total: number;
  currency: 'USD';
}
```

### Example Breakdown

```typescript
// Example: 4K texture upscaling job
const breakdown: CostBreakdown = {
  compute: {
    gpuTime: 0.0413, // 60 seconds on RTX 4090 at $2.48/hr
    cpuTime: 0,
    coldStart: 0.0021, // 3 second cold start
  },
  transfer: {
    ingress: 0, // 50MB input (free)
    egress: 0.01, // 200MB output at $0.05/GB
    internal: 0,
  },
  storage: {
    persistent: 0,
    temporary: 0,
  },
  platform: {
    baseFee: 0, // No additional fee
    premiumFee: 0,
  },
  total: 0.0534,
  currency: 'USD',
};
```

## Budget Management

### Budget Configuration

```typescript
interface BudgetConfig {
  // Per-job limits
  maxCostPerJob?: number; // Max cost for any single job

  // Time-based limits
  hourlyBudget?: number; // Max spend per hour
  dailyBudget?: number; // Max spend per day
  weeklyBudget?: number; // Max spend per week
  monthlyBudget?: number; // Max spend per month

  // Alerts
  alertThresholds?: {
    warning: number; // Alert at this percentage (e.g., 0.8 = 80%)
    critical: number; // Critical alert (e.g., 0.95 = 95%)
  };

  // Enforcement
  enforcement?: 'soft' | 'hard'; // Soft = warn, Hard = reject

  // Rollover
  rolloverUnused?: boolean; // Carry unused budget to next period
}
```

### Budget Enforcement

```typescript
const dispatcher = createJobDispatcher({
  costEstimator,
  budgetConfig: {
    maxCostPerJob: 10.0,
    dailyBudget: 100.0,
    monthlyBudget: 1000.0,
    alertThresholds: {
      warning: 0.8,
      critical: 0.95,
    },
    enforcement: 'hard',
  },
});

// Job submission with budget check
try {
  await dispatcher.dispatch(job);
} catch (error) {
  if (error.code === 'BUDGET_EXCEEDED') {
    console.log(`Budget exceeded: ${error.message}`);
    console.log(`Current spend: $${error.currentSpend}`);
    console.log(`Budget limit: $${error.budgetLimit}`);
  }
}
```

### Budget Status API

```typescript
// Get current budget status
const status = await dispatcher.getBudgetStatus();

console.log('Budget Status:', {
  daily: {
    spent: status.daily.spent,
    limit: status.daily.limit,
    remaining: status.daily.remaining,
    percentUsed: status.daily.percentUsed,
  },
  monthly: {
    spent: status.monthly.spent,
    limit: status.monthly.limit,
    remaining: status.monthly.remaining,
    percentUsed: status.monthly.percentUsed,
  },
  alerts: status.alerts, // Active alerts
  forecast: status.forecast, // Projected spend
});
```

### Alert Webhooks

```typescript
dispatcher.on('budget-warning', (event) => {
  console.log(`Budget warning: ${event.budgetType} at ${event.percentUsed}%`);
  // Send notification
});

dispatcher.on('budget-critical', (event) => {
  console.log(`Budget critical: ${event.budgetType} at ${event.percentUsed}%`);
  // Escalate alert
});

dispatcher.on('budget-exceeded', (event) => {
  console.log(`Budget exceeded: ${event.budgetType}`);
  // Take action (pause jobs, etc.)
});
```

## Cost Optimization

### Provider Selection Strategies

```typescript
// 1. Cheapest provider (default)
const cheapest = createJobDispatcher({
  costEstimator,
  selectionStrategy: 'cheapest',
});

// 2. Fastest provider (minimize duration + cold start)
const fastest = createJobDispatcher({
  costEstimator,
  selectionStrategy: 'fastest',
});

// 3. Best value (cost per performance)
const bestValue = createJobDispatcher({
  costEstimator,
  selectionStrategy: 'best-value',
});

// 4. Custom strategy
const custom = createJobDispatcher({
  costEstimator,
  selectionStrategy: (estimates) => {
    // Custom logic to select provider
    return (
      estimates.find(
        (e) => e.provider === 'modal' && e.gpuType === 'a100-40gb'
      ) || estimates[0]
    );
  },
});
```

### Spot Instance Usage

```typescript
const job: GPUJobRequestBase = {
  type: 'gaussian-splatting',
  priority: 'normal',
  gpuRequirements: {
    minVRAM: 40,
    canUseSpot: true, // Allow spot/preemptible
    estimatedDuration: 3600,
  },
  // ...
};

// Spot savings can be 60-80% vs. on-demand
const estimate = costEstimator.estimate(job);
console.log(`On-demand cost: $${estimate.onDemandCost}`);
console.log(`Spot cost: $${estimate.spotCost}`);
console.log(`Savings: ${estimate.spotSavingsPercent}%`);
```

### Job Batching

```typescript
// Batch multiple small jobs to reduce cold start overhead
const batcher = createJobBatcher({
  dispatcher,
  maxBatchSize: 10,
  maxWaitTime: 5000, // Max wait before dispatching
  batchBy: 'job-type', // Batch by job type
});

// Submit jobs - they'll be batched automatically
await batcher.submit(job1);
await batcher.submit(job2);
await batcher.submit(job3);

// Flush immediately if needed
await batcher.flush();
```

### Warm Pool Pre-provisioning

```typescript
const warmPool = createWarmPoolManager({
  dispatcher,
  pools: {
    'texture-upscale': {
      minWorkers: 2,
      maxWorkers: 10,
      gpuType: 'rtx-4090',
      provider: 'runpod-serverless',
      idleTimeout: 300, // Keep warm for 5 minutes
    },
    'blender-render': {
      minWorkers: 1,
      maxWorkers: 5,
      gpuType: 'a10',
      provider: 'modal',
      idleTimeout: 600,
    },
  },
});

// Jobs use warm workers instantly (no cold start)
const result = await dispatcher.dispatch(job);
```

### Cost Optimization Recommendations

```typescript
// Get optimization recommendations
const recommendations = await costEstimator.getOptimizationRecommendations({
  jobHistory: await getRecentJobs(30), // Last 30 days
  currentConfig: dispatcher.getConfig(),
});

recommendations.forEach((rec) => {
  console.log(`${rec.type}: ${rec.description}`);
  console.log(`  Potential savings: $${rec.potentialSavings}/month`);
  console.log(`  Implementation: ${rec.implementation}`);
});

// Example recommendations:
// - "Switch to spot instances": Save $150/month by using spot for low-priority jobs
// - "Use warm pools": Save $50/month by reducing cold starts
// - "Batch similar jobs": Save $30/month by batching texture-upscale jobs
// - "Right-size GPU selection": Save $75/month by using A10 instead of A100 for jobs under 16GB VRAM
```

## Reporting & Analytics

### Usage Reports

```typescript
import { getGPUJobsMonitoringService } from '@yemaya/gpu-jobs';

const monitoring = getGPUJobsMonitoringService();

// Daily report
const dailyReport = await monitoring.getDailyReport(new Date());
console.log('Daily Report:', {
  totalJobs: dailyReport.totalJobs,
  totalCost: dailyReport.totalCost,
  averageCostPerJob: dailyReport.averageCostPerJob,
  costByProvider: dailyReport.costByProvider,
  costByJobType: dailyReport.costByJobType,
  costByUser: dailyReport.costByUser,
  estimateAccuracy: dailyReport.estimateAccuracy,
});

// Monthly report
const monthlyReport = await monitoring.getMonthlyReport(2024, 11);
console.log('Monthly Report:', {
  totalCost: monthlyReport.totalCost,
  trend: monthlyReport.trend, // vs. previous month
  topUsers: monthlyReport.topUsers,
  topJobTypes: monthlyReport.topJobTypes,
  savingsAchieved: monthlyReport.savingsAchieved,
});
```

### Cost Metrics

```typescript
// Real-time metrics
const metrics = monitoring.getCostMetrics();

console.log('Cost Metrics:', {
  // Running costs
  currentHourCost: metrics.currentHourCost,
  currentDayCost: metrics.currentDayCost,
  runningJobsCost: metrics.runningJobsCost,

  // Historical
  last24HoursCost: metrics.last24HoursCost,
  last7DaysCost: metrics.last7DaysCost,
  last30DaysCost: metrics.last30DaysCost,

  // Estimates
  projectedDailyCost: metrics.projectedDailyCost,
  projectedMonthlyCost: metrics.projectedMonthlyCost,

  // Accuracy
  estimateAccuracy: metrics.estimateAccuracy, // Percentage
  avgEstimateDeviation: metrics.avgEstimateDeviation,
});
```

### Estimate Accuracy Tracking

```typescript
// Track estimate accuracy over time
const accuracyReport = await monitoring.getEstimateAccuracyReport({
  startDate: new Date('2024-10-01'),
  endDate: new Date('2024-10-31'),
  groupBy: 'provider',
});

accuracyReport.forEach((entry) => {
  console.log(`${entry.provider}:`);
  console.log(`  Average accuracy: ${entry.avgAccuracy}%`);
  console.log(`  Over-estimates: ${entry.overEstimatePercent}%`);
  console.log(`  Under-estimates: ${entry.underEstimatePercent}%`);
  console.log(`  Max deviation: $${entry.maxDeviation}`);
});
```

## API Reference

### GPUCostEstimator

```typescript
class GPUCostEstimator {
  // Estimate cost for a single job
  estimate(job: GPUJobRequestBase): CostEstimate;

  // Estimate for specific provider
  estimateForProvider(
    job: GPUJobRequestBase,
    provider: GPUProviderType,
    gpuType: GPUType
  ): CostEstimate;

  // Compare all providers
  estimateAllProviders(job: GPUJobRequestBase): CostEstimate[];

  // Get cheapest option
  getCheapestOption(job: GPUJobRequestBase): CostEstimate;

  // Get fastest option
  getFastestOption(job: GPUJobRequestBase): CostEstimate;

  // Calculate batch cost
  estimateBatch(jobs: GPUJobRequestBase[]): BatchCostEstimate;

  // Get optimization recommendations
  getOptimizationRecommendations(
    config: OptimizationConfig
  ): Promise<Recommendation[]>;

  // Update pricing data
  updatePricing(provider: GPUProviderType, pricing: ProviderPricing): void;
}
```

### CostEstimate Interface

```typescript
interface CostEstimate {
  // Core estimates
  estimatedCost: number; // Total estimated cost in USD
  estimatedDuration: number; // Estimated duration in seconds

  // Provider info
  provider: GPUProviderType;
  gpuType: GPUType;
  gpuCount: number;

  // Detailed breakdown
  breakdown: CostBreakdown;

  // Confidence
  confidence: 'high' | 'medium' | 'low';
  confidenceFactors: string[];

  // Alternatives
  alternatives?: {
    provider: GPUProviderType;
    gpuType: GPUType;
    cost: number;
    duration: number;
  }[];

  // Spot pricing (if available)
  spotCost?: number;
  spotSavingsPercent?: number;
}
```

### Budget Status Interface

```typescript
interface BudgetStatus {
  // Current period status
  hourly: PeriodStatus;
  daily: PeriodStatus;
  weekly: PeriodStatus;
  monthly: PeriodStatus;

  // Active alerts
  alerts: BudgetAlert[];

  // Forecast
  forecast: {
    endOfDay: number;
    endOfWeek: number;
    endOfMonth: number;
  };

  // Running jobs
  runningJobsCost: number;
  pendingJobsCost: number;
}

interface PeriodStatus {
  spent: number;
  limit: number | null;
  remaining: number | null;
  percentUsed: number;
  status: 'ok' | 'warning' | 'critical' | 'exceeded';
}

interface BudgetAlert {
  type: 'warning' | 'critical' | 'exceeded';
  budgetType: 'hourly' | 'daily' | 'weekly' | 'monthly' | 'per-job';
  message: string;
  timestamp: Date;
  currentValue: number;
  threshold: number;
}
```

## Best Practices

### 1. Always Estimate Before Dispatch

```typescript
// Good: Check cost before dispatch
const estimate = dispatcher.estimateCost(job);
if (estimate.estimatedCost > acceptableThreshold) {
  console.log('Job too expensive, consider alternatives');
  console.log('Alternatives:', estimate.alternatives);
} else {
  await dispatcher.dispatch(job);
}
```

### 2. Set Appropriate Budgets

```typescript
// Per-project budget tracking
const projectBudget = createBudgetTracker({
  projectId: 'project-123',
  monthlyBudget: 500,
  alertWebhook: 'https://api.example.com/budget-alerts',
});
```

### 3. Use Spot When Possible

```typescript
// Enable spot for non-critical jobs
const job: GPUJobRequestBase = {
  // ...
  gpuRequirements: {
    canUseSpot: priority !== 'urgent',
  },
};
```

### 4. Right-Size GPU Selection

```typescript
// Don't over-provision
const requirements = {
  minVRAM: calculateMinVRAM(jobInput), // Calculate actual need
  // NOT: minVRAM: 80  // Always using A100 80GB
};
```

### 5. Monitor Estimate Accuracy

```typescript
// Track and improve estimates
dispatcher.on('job-completed', async (event) => {
  const accuracy = event.actualCost / event.estimatedCost;
  if (accuracy < 0.8 || accuracy > 1.2) {
    // Investigate significant deviations
    await reportEstimateDeviation(event);
  }
});
```

## Related Documentation

- [GPU Jobs README](./README.md)
- [Provider Configuration](./providers.md)
- [Troubleshooting Guide](./troubleshooting.md)
- [API Reference](./api.md)
