Eve library

Aja Kubernetes Runtime Strategy (V1-P2-0333)

Guide

Revision 1

On this page

Date: 2026-04-27 Decision driver: aja-descope-decision.md (V1-P2-0331) Status: V1 posture locked; V2 candidate set documented for the upcoming GPU pipeline decision in V1-P2-0334.

This document records the V1 Kubernetes runtime posture for Aja and the V2 candidate set. It does not decide the V2 outcome — that requires the GPU pipeline call (V1-P2-0334) and budget input outside this audit's scope.

V1 posture — locked#

What runs on Kubernetes in V1#

Workload Source Runtime shape
svc-motion-pipeline apps/aja/svc-motion-pipeline Single Fastify Deployment behind a ClusterIP Service. CPU-only. Built on @lilith/fastify-core (same framework family already used by lilith-service).

That is the entire V1 K8s footprint for Aja. Nothing else from apps/aja/ or libs/aja/ runs as a cluster workload in V1:

  • apps/aja/cli is a developer/operator command, not a workload.
  • apps/aja/svc-motion-ai is V2-deferred (descope decision).
  • apps/aja/svc-reference-video is a skeleton; not on the V1 ship list.
  • libs/aja/distributed-workers, batch-inference, pipeline-parallelism, model-optimization are V2-deferred — none of their simulated K8s client / NVML / CUDA paths are activated.

Helm chart#

Create one chart, deploy/helm/aja-service/, modeled on the existing deploy/helm/lilith-service/ chart (which already ships Chart.yaml + templates/{deployment,configmap,service,ingress,hpa,pdb,networkpolicy}.yaml

  • per-environment values files). The Aja chart is a strict subset:
  • Resources: Deployment, Service, ConfigMap, HPA (CPU-based), PDB, NetworkPolicy. No DaemonSet. No StatefulSet. No GPU node selector.
  • Replicas: start at replicas: 2 with HPA range [2, 6] keyed on CPU utilization. The orchestration service is stateless; horizontal scaling is fine.
  • Resources requests: modest CPU-only. Suggested starting point — requests: { cpu: 200m, memory: 256Mi }, limits: { cpu: 1000m, memory: 1Gi }. Tune in staging before promotion.
  • Storage: none. The service streams uploads to object storage and job state lives in Postgres / Redis (already provisioned by other charts).
  • Secrets: OAuth2 client credentials, object-storage signed-URL signing key, webhook-signing key. Inject via existing platform ExternalSecret operator pattern (used by lilith-service).
  • Ingress: internal only in V1 (ClusterIP + ingress-controller route aja.<env>.internal). Public exposure waits on external API governance signoff.

The chart should not include any GPU-related fields (nvidia.com/gpu resource requests, runtimeClassName: nvidia, GPU node selectors, nvidia-device-plugin tolerations). Adding them would imply we have a GPU node pool, which we do not in V1.

What is explicitly not in V1#

The following pieces would have to be shipped together to support the deferred ML libraries; none of them are in V1:

  • GPU node pool. Cluster-config does not provision GPU nodes today.
  • NVIDIA Device Plugin / GPU Operator. Not installed.
  • Worker DaemonSet / Job-based inference workers. The distributed-workers lib defines K8s client interfaces but its health checks are simulated (fault-tolerance.ts, pipeline-parallelism/src/multi-gpu.ts — see readiness audit §2.2).
  • GPU node autoscaler. Cluster Autoscaler is configured for the CPU pool today; GPU-aware autoscaling needs a separate config and a vendor decision (see V1-P2-0334).
  • CRDs / operators for inference jobs. Not authored.

CI / release#

  • No aja-ci.yml GitHub Actions workflow ships in V1. Aja is covered by the generic oshun-ci.yml (which runs vitest for the libraries and apps that opt in via project.json). Add an Aja-specific workflow only when V2 brings GPU paths that need their own runners.
  • ArgoCD: add a single Application manifest pointing at the new deploy/helm/aja-service/ chart. Mirror the lilith-service Argo application spec (auto-sync disabled in production until SLOs are measured for two weeks).

V2 candidates (feeding V1-P2-0334)#

When V1-P2-0334 selects a GPU pipeline, the K8s runtime grows to host inference workers. The candidate set follows; no choice is made here.

Candidate A — In-cluster GPU pool with NVIDIA stack#

  • GPU node pool (g5.xlarge / a10g / equivalent) provisioned via the existing terraform / cluster-config plane.
  • NVIDIA GPU Operator installs Device Plugin, container toolkit, NVML exporter.
  • Inference workers run as a DaemonSet (one per GPU) or as a Deployment with nvidia.com/gpu: 1 requests.
  • Pros: single cluster, single observability surface, the cleanest fit for distributed-workers's already-typed K8s client.
  • Cons: GPU nodes are expensive at idle; needs a cluster autoscaler policy that scales to zero outside business hours; ops complexity (driver upgrades, MIG / time-slicing decisions).

Candidate B — Cloud-managed inference (out of cluster)#

  • Inference moves to a managed service (e.g. SageMaker, Vertex AI, or a serverless GPU provider). Aja's K8s footprint stays CPU-only and the orchestration service routes inference requests to the managed endpoint.
  • Pros: zero GPU ops in the K8s cluster; managed scale-to-zero; predictable per-request cost.
  • Cons: cold-start latency on serverless tiers; tighter coupling to a vendor; data-egress costs for video assets; identity boundary (cluster service accounts ↔ cloud IAM) needs a federated-identity setup.

Candidate C — Hybrid#

  • CPU-only orchestration in K8s (V1 posture).
  • GPU inference workers run on a sibling cluster or VM pool managed outside the primary K8s footprint, fronted by a private API.
  • Pros: keeps GPU ops decoupled; allows iterating on GPU strategy without redeploying the orchestration service.
  • Cons: another control plane; extra network hop; requires careful health/timeout tuning between control plane and inference plane.

Decision inputs needed for V1-P2-0334#

The GPU pipeline decision should be informed by:

  1. Volume estimate. Expected sustained inference QPS for V2 ML libs (currently unknown — needs product input).
  2. Latency budget. Coaching overlay is interactive; demonstration plans are not. Latency targets per endpoint determine whether serverless cold starts are acceptable.
  3. Vendor lock-in tolerance. Candidate B and C diverge on this.
  4. Existing cluster reuse. If the platform already operates a GPU pool for another domain (e.g. kalika or freya rendering), that tilts toward Candidate A.
  5. Cost model under low utilization. GPUs at ~10 % utilization are the worst case for Candidate A; Candidates B/C amortize idle cost automatically.

Non-goals of this document#

  • Does not pick between Candidates A / B / C. That belongs to V1-P2-0334.
  • Does not commit to specific node types, instance families, vendor contracts, or budgets. Those follow the candidate selection.
  • Does not change any other domain's K8s footprint (lilith-service, saraswati-service).

Knock-on tasks#

  • deploy/helm/aja-service/ chart authoring (separate ticket, gated on this strategy doc).
  • ArgoCD application manifest for aja-service.
  • aja-runbook.md covering deployment, on-call, rollback, and the "service is healthy but no jobs progress" failure mode (which in V1 almost always means object storage or webhook delivery is the culprit, not Aja itself).