Eve library

Aja Readiness Audit (V1-P2-0330)

Guide

Revision 1

On this page

Date: 2026-04-27 Scope: libs/aja/* and apps/aja/* Audit method: directory-by-directory inventory, representative source-file inspection, adversarial-scan vocabulary check (in production, would be loaded, Math.random() in implementation paths, simulated runtimes, missing model data), deployment-infrastructure search.

This document feeds the V1-P2-0331 descope-vs-complete decision and the follow-on tasks V1-P2-0332 (canonical SDK contract), V1-P2-0333 (Kubernetes runtime), and V1-P2-0334 (GPU pipeline).

TL;DR#

Aja currently ships ~254k LOC of TypeScript across 37 libraries plus ~7.9k LOC of Python in the data-science SDK, but a substantial fraction of the ML-adjacent code is simulated: GPU paths, ONNX inference, SMPL model loading, K8s health checks, and TensorRT compilation are stubbed with explicit "in production this would..." markers and Math.random() joint confidences. Deployment infrastructure (Dockerfiles, Helm charts, GPU node pools, CI workflow) for Aja is absent, and libs/contracts/src/aja/ does not exist.

Recommendation: descope to a thin V1 MVP centered on the job-orchestration service plus SDKs and format conversion (which are real and shippable), deferring all ML-inference / GPU / distributed-worker functionality to V2. See §7 Recommendation.

1. Inventory#

1.1 libs/aja/* — 37 subprojects, ~254k LOC TypeScript + 7.9k LOC Python#

All 37 subprojects have package.json and source code; none are strictly empty. Test files: 25 across the domain (*.test.ts or test_*.py), covering ~9 of the 37 libs (~24 % of libraries have any tests at all).

Subproject Approx LOC Tests Classification
fitness-animation ~55,800 yes (3) REAL — kinematics, fatigue, mocap, muscle, physics, secondary motion
human-mesh-recovery ~16,200 no MIXED — real FK/loss math; SMPL data is placeholder (§2.2)
motion-processing ~16,600 yes (1) REAL — filters, gap-filling, retiming
neural-retargeting ~16,150 no MIXED — typed ONNX interfaces, weights initialized but never loaded
motion-formats ~10,650 no REAL — BVH/FBX/GLB/USD/Alembic parser/serializer surface
batch-inference ~8,740 yes (5) MIXED — real batching/queue logic; GPU memory queries simulated
animation-blending ~8,160 no REAL — additive layers, idle blending, clip transitions
motion-pipeline-sdk-python ~7,950 yes REAL — async/sync clients, training, jupyter, data-science helpers
motion-pipeline-sdk ~6,930 yes REAL — TS client, OAuth2, retry, streaming, batch
distributed-workers ~6,740 no MIXED — K8s client typed but health checks simulated
27 others ~2k–4k each partial STUB-leaning to MIXED — see §2

(LOC figures are wc -l of *.ts files excluding node_modules/dist; they include type files, comments, blank lines, and tests.)

1.2 apps/aja/* — 5 apps#

App Stack Status
cli Commander.js + chalk + inquirer Real CLI; commands process, convert, inspect, debug, config, jobs, health delegate to the SDK client. Functions only insofar as svc-motion-pipeline does.
svc-motion-ai Hono on :3040 Routes for /embodied-instruction, /motion-retargeting, /skeletal-animation-motion, /video-analysis, /video-to-motion exist; underlying service implementations are MIXED (depend on the inference libs above).
svc-motion-pipeline Fastify (@lilith/fastify-core) Real service skeleton — job manager, configuration repository, webhook service, polling service, progress streaming, download service. Health endpoint inherited from the framework.
svc-reference-video (minimal) Skeleton; not a V1 ship target.
docs Markdown Comprehensive but aspirational — README + API + integration + examples; describes end-state, not current implementation.

1.3 Documentation#

  • libs/aja/README.md — single 200+ line domain README. Aspirational; lists libraries, layered architecture, supported pipelines, and ML methods (HMR 2.0, CLIFF, PyMAF-X, 4D-Humans, WHAM) none of which are actually implemented.
  • No per-subproject READMEs.
  • apps/aja/docs — comprehensive but reflects intended end-state.

2. Stub vs Real classification#

Adversarial scan summary across libs/aja/**/*.ts (excluding tests/dist):

  • "in production" markers: 17 hits
  • Broader stub-indicator markers (simulated, placeholder, would be loaded, would use, TODO, FIXME): 125 hits
  • Math.random() calls in implementation source (not tests): 124 hits

These are not all stubs — gap-filling legitimately uses noise, and several hits are inside test fixtures inside source folders. But the concentration in human-mesh-recovery, neural-retargeting, pipeline-parallelism, and model-optimization lines up with the qualitative classifications below.

2.1 REAL — domain-specific algorithms, would survive adversarial scan#

  • motion-pipeline-sdk — Full TS client: OAuth2 token refresh, retry with exponential backoff, NDJSON streaming, batch operations. No external deps beyond fetch. ~7k LOC.
  • motion-pipeline-sdk-python — Real httpx/pydantic async + sync clients, optional torch integration for the training module, Jupyter helpers. ~8k LOC.
  • motion-formats — BVH/FBX/GLB/USD/Alembic parser/serializer scaffolding (~10.6k LOC). Format-specific algorithms.
  • fitness-animation — Largest library. Inverse kinematics, mocap ingestion, muscle simulation, physics, secondary motion. ~56k LOC, 3 test suites.
  • motion-processing — Filters, smoothing, gap-filling, foot-sliding detection, retiming. (Note one Math.random() use in gap-filling.ts:845-847 is a legitimate noise injection for synthetic data, not a stubbed algorithm.)
  • animation-blending — Additive layers, idle blending, clip transitions.
  • apps/aja/cli — Real CLI structure built on Commander.js; depends on the orchestration service.
  • apps/aja/svc-motion-pipeline — Fastify orchestration service skeleton using @lilith/fastify-core for health/metrics. Job manager, webhook delivery, streaming download.

2.2 MIXED — real structure but load-bearing components are simulated#

  • human-mesh-recovery/src/smpl-recovery.ts:48:

    // Model data (would be loaded from NPZ files in production)

    The SMPLModel class has real forward kinematics, body fitting, and loss functions, but meanShape, shapeDirs, etc. are initialized as empty Float32Arrays. smpl-recovery.ts:794 populates joint confidences with 0.9 + Math.random() * 0.1 — i.e. mesh recovery is a stub returning fabricated results.

  • neural-retargeting — ~16k LOC of transformer/skeleton-graph math, but the ONNX runtime is only typed (interfaces in inference.ts:35-65), never imported or invoked. Without onnxruntime linked, no actual inference happens.

  • distributed-workers/src/fault-tolerance.ts:

    // Simulate health check (in production, would call worker health endpoint)

    Defines K8s client classes but never makes a real Kubernetes API request.

  • batch-inference/src/memory-manager.ts:206:

    // This would be replaced with actual GPU queries in production

    Real CPU-side batching scheduler, but every GPU memory probe returns a hardcoded value.

  • pipeline-parallelism/src/multi-gpu.ts — 7 explicit "in production" markers across :89, :219, :313, :413, :442, :498, :689. Multi-GPU initialization, peer-access, NVML utilization, and CUDA copy paths are all comments rather than calls.

  • model-optimization/src/tensorrt-compiler.ts:267:

    // Build the engine (simulated - in production would use native bindings)

    TensorRT compilation produces a synthetic engine descriptor.

  • apps/aja/svc-motion-ai — Hono routes exist with proper schemas (coachingOverlaySchema, sessionHandoffSchema, demonstrationRequestSchema) but the service classes they depend on come from MIXED libraries above, so the service inherits their gaps.

2.3 STUB — placeholder/simulated only#

  • content-moderation/src/upload-scanner.ts:367:

    // This is a simulation - in production, this would call an ML model

    NSFW/violence detection is a hash-based scoring placeholder.

  • video-chunking/src/video-splitter.ts:56:

    // Placeholder - in production would analyze actual video frames

  • pose-lifting/src/depth-estimation.ts:210, blend-shape-retargeting/src/muscle-deformation.ts:514, multi-view-reconstruction/src/calibration.ts:354 — same pattern: the function is implemented as a fallback that ignores its real inputs.

  • model-optimization/src/mixed-precision.ts:839 — calibration data generation is (Math.random() - 0.5) * 2 rather than an ingested representative dataset. This is a stub: it claims to calibrate but cannot improve any model that depends on its output.

  • content-security, content-watermarking, consent-management, data-retention, asset-storage — CRUD/config-only patterns, no domain-specific algorithms. Nothing here would fail an adversarial scan because there is no algorithmic substance to scan.

  • skeleton-mapping, proportional-adaptation, blend-shape-retargeting, depth-sensing, pose-lifting, multi-view-reconstruction — type definitions present, light implementations, no end-to-end working path.

2.4 EMPTY#

None strictly empty (every directory has at least index.ts + types.ts), but consent-management, data-retention, and content-watermarking are near-skeletons with only configuration types and CRUD wrappers around in-memory stores.

3. Cross-cutting concerns#

3.1 ML / GPU / native code#

  • ~95 % of Aja code is pure TypeScript.
  • No CUDA, cuDNN, TensorRT, or NVML bindings are linked. All GPU references are comments.
  • No ONNX Runtime dependency, despite neural-retargeting typing one.
  • No Python ML inference outside the optional training module of the Python SDK.
  • No model weights / checkpoints in the repo (no .pth, .onnx, .npz for SMPL, .pb, no checkpoint directory).

3.2 Polyglot story#

  • Dominant: TypeScript (NestJS / Hono / Fastify).
  • Secondary: Python (motion-pipeline-sdk-python for data-science workflows; optional torch).
  • Absent: Rust, C++, Cython, Go, native bindings.

3.3 Deployment infrastructure#

  • Helm: deploy/helm/ contains cluster-config, environments, lilith-service, saraswati-service. No aja-service chart.
  • Dockerfiles: zero Dockerfile* under apps/aja/.
  • K8s manifests / ArgoCD apps: no Aja entries.
  • GPU node pools: zero nvidia.com/gpu resource requests, GPU node selectors, or NVIDIA runtime config anywhere in deploy/.
  • CI: .github/workflows/ has oshun-ci.yml, iris-ci.yml, psyche-ci.yml, tara-ci.yml. No aja-ci.yml. Aja is covered only by the generic monorepo CI, which does not exercise GPU paths.

3.4 Contracts#

libs/contracts/src/ contains common/, events/, llm/, tts/. There is no aja/ subdirectory. V1-P2-0332 explicitly anticipates creating one "if retained"; the audit confirms the directory is not pre-staged.

4. Test coverage#

  • 9 of 37 libs have any test files (~24 %).
  • 0 tests for neural-retargeting, human-mesh-recovery, motion-formats, multi-view-reconstruction, skeleton-mapping, pose-lifting, proportional-adaptation, blend-shape-retargeting, depth-sensing, content-security, content-watermarking, and the rest of the long tail.
  • No app-level E2E tests connecting cli → svc-motion-pipeline → workers.
  • No cross-language test verifying the Python SDK against the TypeScript service.

5. README vs reality#

libs/aja/README.md advertises:

"Comprehensive tools for processing, analyzing, and transforming human motion data."

…and explicitly names methods including HMR 2.0, CLIFF, PyMAF-X, 4D-Humans, WHAM, "deep learning models", "semantic motion retargeting", and a production-grade pipeline across fitness / dance / sports / medical / gaming / VFX domains.

Reality: the SMPL recovery class is initialized with empty arrays (smpl-recovery.ts:48). The neural retargeting interface to ONNX is never loaded. There are no model files. The README describes the V2 vision, not the V1 code.

6. Top blockers for V1 ship#

  1. No GPU inference runtime. batch-inference, neural-retargeting, pipeline-parallelism, and model-optimization all assume an inference substrate that does not exist (no onnxruntime, no CUDA, no TensorRT).
  2. No SMPL / parametric-body model data. human-mesh-recovery cannot produce real meshes without model NPZs.
  3. Simulated K8s health and fault tolerance. distributed-workers/src/fault-tolerance.ts and pipeline-parallelism/src/multi-gpu.ts cannot scale workers — every "health check" is a comment.
  4. Empty libs/contracts/src/aja/. No canonical SDK contract; downstream teams cannot consume Aja stably.
  5. No Aja deployment artifacts. Zero Dockerfile, zero Helm chart, zero ArgoCD app, zero GPU node-pool configuration.
  6. 24 % test coverage; zero E2E tests. Critical paths (neural-retargeting, human-mesh-recovery, motion-formats) have no tests.
  7. No GPU pipeline decision. V1-P2-0334 lists CUDA-via-NVML / ROCm / cloud-only as alternatives; none are picked.
  8. No K8s runtime decision. V1-P2-0333 expects a strategy doc; none exists.
  9. Aspiration / reality mismatch in the domain README. Sets expectations that the code cannot meet, which will burn integrators.
  10. Polyglot integration unverified. Python SDK ↔ TypeScript service data-format compatibility (numpy arrays ↔ Float32Array) untested.

7. Recommendation#

Descope Aja to a minimal V1 MVP. Do not attempt to ship the full domain in V1.

7.1 Reasoning#

  • Substance vs aspiration gap: the volume of code is real, but load-bearing inference paths are simulated. Shipping the current state without GPU/ML wiring would mean shipping a domain whose advertised value proposition (motion AI) is fabricated at the algorithmic level.
  • External dependencies missing: SMPL models, ONNX Runtime, GPU node pools, K8s operators. Each is an out-of-monorepo procurement / integration decision. Stacking those decisions in V1 imports significant schedule risk.
  • Test and contract debt: 76 % of libs are untested; the canonical contract directory is absent. Stabilizing the public surface alone is multi-week work.
  • Other domains are ahead. lilith and saraswati already have Helm charts; Aja would be the only Tier-A domain shipping without one.

7.2 Proposed V1 MVP scope (the parts that are real today)#

Ship:

  • libs/aja/motion-pipeline-sdk (TypeScript client)
  • libs/aja/motion-pipeline-sdk-python (Python client)
  • libs/aja/motion-formats (format conversion only — no ML)
  • apps/aja/svc-motion-pipeline (Fastify orchestration; jobs/webhooks/ delivery; no inference workers)
  • apps/aja/cli (wraps the SDK)
  • apps/aja/docs (rewritten to match V1 reality, not V2 vision)

Add for V1:

  • libs/contracts/src/aja/ — canonical job / format / status contracts that the SDK + service share (fulfills V1-P2-0332).
  • A V1 Helm chart for svc-motion-pipeline only (no GPU node pool, no worker daemonset). Document that ML workers are V2.

Defer to V2:

  • distributed-workers, batch-inference, pipeline-parallelism, model-optimization (all GPU paths)
  • neural-retargeting, human-mesh-recovery, pose-lifting, depth-sensing, multi-view-reconstruction, blend-shape-retargeting, proportional-adaptation, skeleton-mapping, semantic-retargeting (all ML inference)
  • content-moderation, content-security, content-watermarking, consent-management, data-retention (until the surrounding ML/storage story is real)
  • apps/aja/svc-motion-ai (depends on the deferred libs above)
  • apps/aja/svc-reference-video (skeleton, not V1 material)

7.3 Exit criteria revisions for V1-MET-197/198/199#

V1-P2-0331 is the locking task; the proposed adjustments are:

  • V1-MET-197 (originally end-to-end motion AI pipeline) → restrict to end-to-end job orchestration through the Fastify service plus SDK round-trip; explicitly mark inference as out-of-scope.
  • V1-MET-198 (originally distributed-worker scaling) → defer entirely to V2 once a GPU pipeline (V1-P2-0334) is selected.
  • V1-MET-199 (originally GPU model optimization) → defer entirely to V2.

7.4 What this audit does NOT decide#

The descope/keep decision belongs to the user via V1-P2-0331. This audit provides the evidence base: code volume, stub density, deployment posture, and test coverage. The ML-runtime decision (V1-P2-0334), the K8s decision (V1-P2-0333), and the contract scaffold (V1-P2-0332) are downstream of that locking call.