Eve library

GPU Jobs Cost Model

Guide

Revision 1

On this page

This guide covers the cost estimation system, pricing models, budget management, and cost optimization strategies.

Table of Contents#

Overview#

The GPU Jobs cost estimation system provides accurate cost predictions before job submission, enabling informed decisions about provider selection and budget management.

Key Features#

  • Pre-dispatch estimation: Get cost estimates before submitting jobs
  • Multi-provider comparison: Compare costs across all configured providers
  • Budget enforcement: Automatic rejection of jobs exceeding budgets
  • Real-time tracking: Track actual vs. estimated costs
  • Cost optimization recommendations: Suggestions for reducing GPU spend

Pricing Data#

GPU Pricing Table (as of 2024)#

GPU RunPod Serverless RunPod Pod Modal AWS Spot AWS On-Demand
RTX 3090 $1.58/hr $0.69/hr - - -
RTX 4090 $2.48/hr $1.14/hr - - -
T4 - - $0.59/hr $0.16/hr $0.53/hr
L4 - - $1.10/hr - -
A10 $2.09/hr $1.00/hr $1.10/hr $0.30/hr $1.01/hr
A100 40GB $5.00/hr $3.19/hr $3.88/hr $2.46/hr $8.19/hr
A100 80GB $7.99/hr $5.19/hr $5.63/hr $3.69/hr $12.29/hr
H100 - $14.00/hr $8.12/hr $9.83/hr $32.77/hr

Data Transfer Costs#

Provider Ingress Egress
RunPod Free $0.05/GB
Modal Free $0.05/GB
AWS Free $0.09/GB
Vast.ai Varies Varies
Local Free Free

Storage Costs#

Provider Persistent Storage Temporary
RunPod Pod $0.10/GB/month Included
AWS EBS $0.08/GB/month Included
Modal Free (ephemeral) N/A

Cost Estimation Algorithm#

Basic Formula#

text
Total Cost = Compute Cost + Data Transfer Cost + Platform Fee

Where:
  Compute Cost = (Duration × GPU Price) × GPU Count
  Data Transfer Cost = (Input Size × Ingress Rate) + (Output Size × Egress Rate)
  Platform Fee = Compute Cost × Platform Fee Rate

Duration Estimation#

Duration is estimated based on job type and GPU:

typescript
interface DurationEstimate {
  baseSeconds: number; // Base processing time
  vramMultiplier: number; // Higher VRAM = faster
  gpuGenerationMultiplier: number; // Newer GPU = faster
}

const durationFactors: Record<GPUJobType, DurationEstimate> = {
  'texture-upscale': {
    baseSeconds: 30,
    vramMultiplier: 0.8, // 20% faster per 8GB VRAM
    gpuGenerationMultiplier: 0.9, // 10% faster per generation
  },
  'gaussian-splatting': {
    baseSeconds: 1800, // 30 minutes base
    vramMultiplier: 0.7, // 30% faster with more VRAM
    gpuGenerationMultiplier: 0.85,
  },
  'blender-render': {
    baseSeconds: 120, // Per frame base
    vramMultiplier: 0.75,
    gpuGenerationMultiplier: 0.8,
  },
  // ... other job types
};

GPU Selection Algorithm#

typescript
function selectGPU(
  requirements: GPURequirements,
  preferCheapest: boolean = true
): GPUType {
  // 1. Filter GPUs meeting VRAM requirement
  const eligible = GPU_TYPES.filter((gpu) => gpu.vram >= requirements.minVRAM);

  // 2. Filter by feature requirements (tensor cores, etc.)
  const compatible = eligible.filter((gpu) =>
    meetsFeatureRequirements(gpu, requirements)
  );

  // 3. Apply preference
  if (
    requirements.preferredGPU &&
    compatible.includes(requirements.preferredGPU)
  ) {
    return requirements.preferredGPU;
  }

  // 4. Select based on strategy
  if (preferCheapest) {
    return compatible.sort((a, b) => a.pricePerHour - b.pricePerHour)[0];
  } else {
    return compatible.sort((a, b) => b.performance - a.performance)[0];
  }
}

Cold Start Estimation#

Cold start times are factored into duration estimates:

Provider GPU Type Average Cold Start
RunPod Serverless Any 3-5 seconds
RunPod Pod Any 30-60 seconds
Modal T4/L4 5-10 seconds
Modal A10/A100 10-15 seconds
Modal H100 15-30 seconds
AWS ECS Any 60-120 seconds
Vast.ai Any 30-120 seconds
Local N/A 0 seconds

Cost Breakdown#

Detailed Breakdown Structure#

typescript
interface CostBreakdown {
  // Compute costs
  compute: {
    gpuTime: number; // GPU compute time cost
    cpuTime: number; // CPU compute time cost (if any)
    coldStart: number; // Cold start overhead cost
  };

  // Data transfer
  transfer: {
    ingress: number; // Input data download cost
    egress: number; // Output data upload cost
    internal: number; // Internal transfer cost
  };

  // Storage
  storage: {
    persistent: number; // Persistent volume cost
    temporary: number; // Temporary storage cost
  };

  // Platform fees
  platform: {
    baseFee: number; // Provider platform fee
    premiumFee: number; // Priority/SLA fee (if applicable)
  };

  // Total
  total: number;
  currency: 'USD';
}

Example Breakdown#

typescript
// Example: 4K texture upscaling job
const breakdown: CostBreakdown = {
  compute: {
    gpuTime: 0.0413, // 60 seconds on RTX 4090 at $2.48/hr
    cpuTime: 0,
    coldStart: 0.0021, // 3 second cold start
  },
  transfer: {
    ingress: 0, // 50MB input (free)
    egress: 0.01, // 200MB output at $0.05/GB
    internal: 0,
  },
  storage: {
    persistent: 0,
    temporary: 0,
  },
  platform: {
    baseFee: 0, // No additional fee
    premiumFee: 0,
  },
  total: 0.0534,
  currency: 'USD',
};

Budget Management#

Budget Configuration#

typescript
interface BudgetConfig {
  // Per-job limits
  maxCostPerJob?: number; // Max cost for any single job

  // Time-based limits
  hourlyBudget?: number; // Max spend per hour
  dailyBudget?: number; // Max spend per day
  weeklyBudget?: number; // Max spend per week
  monthlyBudget?: number; // Max spend per month

  // Alerts
  alertThresholds?: {
    warning: number; // Alert at this percentage (e.g., 0.8 = 80%)
    critical: number; // Critical alert (e.g., 0.95 = 95%)
  };

  // Enforcement
  enforcement?: 'soft' | 'hard'; // Soft = warn, Hard = reject

  // Rollover
  rolloverUnused?: boolean; // Carry unused budget to next period
}

Budget Enforcement#

typescript
const dispatcher = createJobDispatcher({
  costEstimator,
  budgetConfig: {
    maxCostPerJob: 10.0,
    dailyBudget: 100.0,
    monthlyBudget: 1000.0,
    alertThresholds: {
      warning: 0.8,
      critical: 0.95,
    },
    enforcement: 'hard',
  },
});

// Job submission with budget check
try {
  await dispatcher.dispatch(job);
} catch (error) {
  if (error.code === 'BUDGET_EXCEEDED') {
    console.log(`Budget exceeded: ${error.message}`);
    console.log(`Current spend: $${error.currentSpend}`);
    console.log(`Budget limit: $${error.budgetLimit}`);
  }
}

Budget Status API#

typescript
// Get current budget status
const status = await dispatcher.getBudgetStatus();

console.log('Budget Status:', {
  daily: {
    spent: status.daily.spent,
    limit: status.daily.limit,
    remaining: status.daily.remaining,
    percentUsed: status.daily.percentUsed,
  },
  monthly: {
    spent: status.monthly.spent,
    limit: status.monthly.limit,
    remaining: status.monthly.remaining,
    percentUsed: status.monthly.percentUsed,
  },
  alerts: status.alerts, // Active alerts
  forecast: status.forecast, // Projected spend
});

Alert Webhooks#

typescript
dispatcher.on('budget-warning', (event) => {
  console.log(`Budget warning: ${event.budgetType} at ${event.percentUsed}%`);
  // Send notification
});

dispatcher.on('budget-critical', (event) => {
  console.log(`Budget critical: ${event.budgetType} at ${event.percentUsed}%`);
  // Escalate alert
});

dispatcher.on('budget-exceeded', (event) => {
  console.log(`Budget exceeded: ${event.budgetType}`);
  // Take action (pause jobs, etc.)
});

Cost Optimization#

Provider Selection Strategies#

typescript
// 1. Cheapest provider (default)
const cheapest = createJobDispatcher({
  costEstimator,
  selectionStrategy: 'cheapest',
});

// 2. Fastest provider (minimize duration + cold start)
const fastest = createJobDispatcher({
  costEstimator,
  selectionStrategy: 'fastest',
});

// 3. Best value (cost per performance)
const bestValue = createJobDispatcher({
  costEstimator,
  selectionStrategy: 'best-value',
});

// 4. Custom strategy
const custom = createJobDispatcher({
  costEstimator,
  selectionStrategy: (estimates) => {
    // Custom logic to select provider
    return (
      estimates.find(
        (e) => e.provider === 'modal' && e.gpuType === 'a100-40gb'
      ) || estimates[0]
    );
  },
});

Spot Instance Usage#

typescript
const job: GPUJobRequestBase = {
  type: 'gaussian-splatting',
  priority: 'normal',
  gpuRequirements: {
    minVRAM: 40,
    canUseSpot: true, // Allow spot/preemptible
    estimatedDuration: 3600,
  },
  // ...
};

// Spot savings can be 60-80% vs. on-demand
const estimate = costEstimator.estimate(job);
console.log(`On-demand cost: $${estimate.onDemandCost}`);
console.log(`Spot cost: $${estimate.spotCost}`);
console.log(`Savings: ${estimate.spotSavingsPercent}%`);

Job Batching#

typescript
// Batch multiple small jobs to reduce cold start overhead
const batcher = createJobBatcher({
  dispatcher,
  maxBatchSize: 10,
  maxWaitTime: 5000, // Max wait before dispatching
  batchBy: 'job-type', // Batch by job type
});

// Submit jobs - they'll be batched automatically
await batcher.submit(job1);
await batcher.submit(job2);
await batcher.submit(job3);

// Flush immediately if needed
await batcher.flush();

Warm Pool Pre-provisioning#

typescript
const warmPool = createWarmPoolManager({
  dispatcher,
  pools: {
    'texture-upscale': {
      minWorkers: 2,
      maxWorkers: 10,
      gpuType: 'rtx-4090',
      provider: 'runpod-serverless',
      idleTimeout: 300, // Keep warm for 5 minutes
    },
    'blender-render': {
      minWorkers: 1,
      maxWorkers: 5,
      gpuType: 'a10',
      provider: 'modal',
      idleTimeout: 600,
    },
  },
});

// Jobs use warm workers instantly (no cold start)
const result = await dispatcher.dispatch(job);

Cost Optimization Recommendations#

typescript
// Get optimization recommendations
const recommendations = await costEstimator.getOptimizationRecommendations({
  jobHistory: await getRecentJobs(30), // Last 30 days
  currentConfig: dispatcher.getConfig(),
});

recommendations.forEach((rec) => {
  console.log(`${rec.type}: ${rec.description}`);
  console.log(`  Potential savings: $${rec.potentialSavings}/month`);
  console.log(`  Implementation: ${rec.implementation}`);
});

// Example recommendations:
// - "Switch to spot instances": Save $150/month by using spot for low-priority jobs
// - "Use warm pools": Save $50/month by reducing cold starts
// - "Batch similar jobs": Save $30/month by batching texture-upscale jobs
// - "Right-size GPU selection": Save $75/month by using A10 instead of A100 for jobs under 16GB VRAM

Reporting & Analytics#

Usage Reports#

typescript
import { getGPUJobsMonitoringService } from '@yemaya/gpu-jobs';

const monitoring = getGPUJobsMonitoringService();

// Daily report
const dailyReport = await monitoring.getDailyReport(new Date());
console.log('Daily Report:', {
  totalJobs: dailyReport.totalJobs,
  totalCost: dailyReport.totalCost,
  averageCostPerJob: dailyReport.averageCostPerJob,
  costByProvider: dailyReport.costByProvider,
  costByJobType: dailyReport.costByJobType,
  costByUser: dailyReport.costByUser,
  estimateAccuracy: dailyReport.estimateAccuracy,
});

// Monthly report
const monthlyReport = await monitoring.getMonthlyReport(2024, 11);
console.log('Monthly Report:', {
  totalCost: monthlyReport.totalCost,
  trend: monthlyReport.trend, // vs. previous month
  topUsers: monthlyReport.topUsers,
  topJobTypes: monthlyReport.topJobTypes,
  savingsAchieved: monthlyReport.savingsAchieved,
});

Cost Metrics#

typescript
// Real-time metrics
const metrics = monitoring.getCostMetrics();

console.log('Cost Metrics:', {
  // Running costs
  currentHourCost: metrics.currentHourCost,
  currentDayCost: metrics.currentDayCost,
  runningJobsCost: metrics.runningJobsCost,

  // Historical
  last24HoursCost: metrics.last24HoursCost,
  last7DaysCost: metrics.last7DaysCost,
  last30DaysCost: metrics.last30DaysCost,

  // Estimates
  projectedDailyCost: metrics.projectedDailyCost,
  projectedMonthlyCost: metrics.projectedMonthlyCost,

  // Accuracy
  estimateAccuracy: metrics.estimateAccuracy, // Percentage
  avgEstimateDeviation: metrics.avgEstimateDeviation,
});

Estimate Accuracy Tracking#

typescript
// Track estimate accuracy over time
const accuracyReport = await monitoring.getEstimateAccuracyReport({
  startDate: new Date('2024-10-01'),
  endDate: new Date('2024-10-31'),
  groupBy: 'provider',
});

accuracyReport.forEach((entry) => {
  console.log(`${entry.provider}:`);
  console.log(`  Average accuracy: ${entry.avgAccuracy}%`);
  console.log(`  Over-estimates: ${entry.overEstimatePercent}%`);
  console.log(`  Under-estimates: ${entry.underEstimatePercent}%`);
  console.log(`  Max deviation: $${entry.maxDeviation}`);
});

API Reference#

GPUCostEstimator#

typescript
class GPUCostEstimator {
  // Estimate cost for a single job
  estimate(job: GPUJobRequestBase): CostEstimate;

  // Estimate for specific provider
  estimateForProvider(
    job: GPUJobRequestBase,
    provider: GPUProviderType,
    gpuType: GPUType
  ): CostEstimate;

  // Compare all providers
  estimateAllProviders(job: GPUJobRequestBase): CostEstimate[];

  // Get cheapest option
  getCheapestOption(job: GPUJobRequestBase): CostEstimate;

  // Get fastest option
  getFastestOption(job: GPUJobRequestBase): CostEstimate;

  // Calculate batch cost
  estimateBatch(jobs: GPUJobRequestBase[]): BatchCostEstimate;

  // Get optimization recommendations
  getOptimizationRecommendations(
    config: OptimizationConfig
  ): Promise<Recommendation[]>;

  // Update pricing data
  updatePricing(provider: GPUProviderType, pricing: ProviderPricing): void;
}

CostEstimate Interface#

typescript
interface CostEstimate {
  // Core estimates
  estimatedCost: number; // Total estimated cost in USD
  estimatedDuration: number; // Estimated duration in seconds

  // Provider info
  provider: GPUProviderType;
  gpuType: GPUType;
  gpuCount: number;

  // Detailed breakdown
  breakdown: CostBreakdown;

  // Confidence
  confidence: 'high' | 'medium' | 'low';
  confidenceFactors: string[];

  // Alternatives
  alternatives?: {
    provider: GPUProviderType;
    gpuType: GPUType;
    cost: number;
    duration: number;
  }[];

  // Spot pricing (if available)
  spotCost?: number;
  spotSavingsPercent?: number;
}

Budget Status Interface#

typescript
interface BudgetStatus {
  // Current period status
  hourly: PeriodStatus;
  daily: PeriodStatus;
  weekly: PeriodStatus;
  monthly: PeriodStatus;

  // Active alerts
  alerts: BudgetAlert[];

  // Forecast
  forecast: {
    endOfDay: number;
    endOfWeek: number;
    endOfMonth: number;
  };

  // Running jobs
  runningJobsCost: number;
  pendingJobsCost: number;
}

interface PeriodStatus {
  spent: number;
  limit: number | null;
  remaining: number | null;
  percentUsed: number;
  status: 'ok' | 'warning' | 'critical' | 'exceeded';
}

interface BudgetAlert {
  type: 'warning' | 'critical' | 'exceeded';
  budgetType: 'hourly' | 'daily' | 'weekly' | 'monthly' | 'per-job';
  message: string;
  timestamp: Date;
  currentValue: number;
  threshold: number;
}

Best Practices#

1. Always Estimate Before Dispatch#

typescript
// Good: Check cost before dispatch
const estimate = dispatcher.estimateCost(job);
if (estimate.estimatedCost > acceptableThreshold) {
  console.log('Job too expensive, consider alternatives');
  console.log('Alternatives:', estimate.alternatives);
} else {
  await dispatcher.dispatch(job);
}

2. Set Appropriate Budgets#

typescript
// Per-project budget tracking
const projectBudget = createBudgetTracker({
  projectId: 'project-123',
  monthlyBudget: 500,
  alertWebhook: 'https://api.example.com/budget-alerts',
});

3. Use Spot When Possible#

typescript
// Enable spot for non-critical jobs
const job: GPUJobRequestBase = {
  // ...
  gpuRequirements: {
    canUseSpot: priority !== 'urgent',
  },
};

4. Right-Size GPU Selection#

typescript
// Don't over-provision
const requirements = {
  minVRAM: calculateMinVRAM(jobInput), // Calculate actual need
  // NOT: minVRAM: 80  // Always using A100 80GB
};

5. Monitor Estimate Accuracy#

typescript
// Track and improve estimates
dispatcher.on('job-completed', async (event) => {
  const accuracy = event.actualCost / event.estimatedCost;
  if (accuracy < 0.8 || accuracy > 1.2) {
    // Investigate significant deviations
    await reportEstimateDeviation(event);
  }
});