# Aja Domain — Architecture > **Aja** — Motion AI and Animation Platform Aja is a motion-intelligence platform that transforms raw video or motion capture input into clean, retargeted, production-ready animation data. Named after the Yoruba Orisha Aja — guide through the forest — the domain navigates the complex path from noisy, real-world capture to high-quality animation assets. In practical terms, a caller submits a video file or a motion capture recording to Aja. The platform cleans and validates the data, estimates 3D skeletal pose, runs quality scoring, retargets the result to any target character skeleton, and delivers the finished animation in the requested file format. The same infrastructure that handles a single clip can scale out horizontally to process thousands of files in parallel. Downstream domains — game studios, VFX pipelines, AI coaching applications — consume the resulting BVH, FBX, or glTF files through integration adapters. The domain consists of **5 application projects** (`apps/aja/`) and **39 library projects** (`libs/aja/`). The motion-pipeline service, its SDK, and the `motion-integration` library were authored under the "Lilith Motion Pipeline" name and retain `@lilith/...` labels in code comments — and one package name (`@lilith/svc-reference-video`). The domain itself is Aja. --- ## Library Organization The 39 libraries in `libs/aja/` are organized into seven functional clusters. Each cluster below lists the libraries it contains and the concern they address. ``` libs/aja/ ├── motion-formats/ # Animation format I/O (BVH, FBX, glTF/GLB, USD, Alembic, C3D, TRC, ASF/AMC, JSON) ├── motion-processing/ # Filtering, smoothing, interpolation, foot sliding correction ├── motion-quality/ # Quality metrics: smoothness, plausibility, ground-truth, perceptual ├── motion-validation/ # Data validation and error detection ├── motion-integration/ # Cross-domain integration adapters (Yemaya, Isis, Bellona, Sophia) │ ├── pose-lifting/ # 2D keypoints → 3D skeletal pose ├── depth-sensing/ # Monocular and stereo depth estimation ├── multi-view-reconstruction/ # Multi-camera 3D reconstruction ├── human-mesh-recovery/ # SMPL/SMPL-X parametric body mesh recovery │ ├── skeleton-mapping/ # Skeleton hierarchy conversion and normalization ├── optimization-ik/ # Inverse kinematics solvers for retargeting ├── avatar-library/ # Avatar definitions, hierarchy presets, rig configs ├── avatar-integration/ # Avatar binding and animation playback ├── avatar-preview-ui/ # React component for 3D avatar preview ├── bone-mapping-ui/ # React drag-and-drop bone mapping interface │ ├── semantic-retargeting/ # Intent-preserving motion transfer ├── proportional-adaptation/ # Body proportion adaptation during retargeting ├── blend-shape-retargeting/ # Facial blend shape extraction and remapping ├── neural-retargeting/ # ML-based motion retargeting ├── animation-blending/ # Motion blending, cross-fades, layering │ ├── domain-motion-pipelines/ # Pre-configured pipelines (yoga, fitness, dance, martial-arts) ├── fitness-animation/ # Fitness-specific analysis (reps, form, ROM) ├── film-pipeline/ # Film/VFX-grade motion deliverable packaging │ ├── motion-pipeline-sdk/ # TypeScript SDK for the pipeline service ├── motion-pipeline-sdk-python/ # Python SDK (aja-motion-pipeline) ├── batch-inference/ # Batch job orchestration ├── video-chunking/ # Automatic long-video splitting ├── model-optimization/ # Model compression and quantization ├── pipeline-parallelism/ # Multi-stage parallel processing ├── distributed-workers/ # Worker pool coordination ├── result-aggregation/ # Combine distributed results ├── pipeline-cache/ # LRU caching for pipeline stages ├── asset-storage/ # Tiered storage management, CDN delivery, signed URL access for motion assets │ ├── privacy-protection/ # Privacy-preserving processing ├── consent-management/ # Consent lifecycle tracking ├── content-security/ # Data security protocols ├── content-watermarking/ # Digital watermark embedding ├── content-moderation/ # Content screening └── data-retention/ # Retention policy enforcement ``` **Total: 39 library projects** --- ## Service Architecture Aja's runtime surface is two HTTP services, one library module, and a CLI tool. The table below lists each application project with its package name, runtime framework, default port, and core responsibility. ``` apps/aja/ svc-motion-pipeline/ @aja/svc-motion-pipeline Fastify :8090 Core pipeline orchestration svc-motion-ai/ @aja/svc-motion-ai Hono :3040 Motion AI endpoints svc-reference-video/ @lilith/svc-reference-video library — Reference video subsystem cli/ @aja/cli Commander — CLI tools docs/ (unnamed) — — Markdown documentation site ``` There are **5 application projects**. `svc-reference-video` ships as a TypeScript library (its `package.json` has `main: dist/index.js` and no HTTP server entry); `docs` is a static Markdown documentation tree. ### `svc-motion-pipeline` (Fastify — port 8090) The central pipeline orchestrator. It is the entry point for all job submissions and the source of truth for job state. Its responsibilities are: - Accept job submissions (single file, batch, URL) - Manage the job queue (via Redis/BullMQ) - Coordinate pipeline stage execution - Deliver results and status updates - Apply data retention and consent policies ### `svc-motion-ai` (Hono — port 3040) The Motion AI service handles the inference-heavy operations that require GPU resources. It composes five internal services (`VideoToMotionService`, `MotionRetargetingService`, `SkeletalAnimationMotionService`, `VideoAnalysisService`, `EmbodiedInstructionService`) and exposes REST routes for: - video-to-motion processing (`POST /api/v1/video-to-motion/process`) - motion retargeting (`POST /api/v1/retargeting/retarget`) - procedural skeletal animation (`POST /api/v1/skeletal/animate`) - video analysis (`POST /api/v1/analysis/analyze`) - embodied-instruction demonstration plans, coaching overlays, and session handoffs (`POST /api/v1/embodied-instruction/*`), plus a capabilities discovery endpoint The service also contains internal module trees for live motion capture, mobile live capture, and motion enhancement (synthesis / style transfer / super-resolution / physics refinement) that the internal services use; these modules are not all directly exposed as routes. Live-capture streaming code lives in these modules — there is no live-capture route on the service. ### `svc-reference-video` (`@lilith/svc-reference-video` — library) A library module (not a server) that manages a searchable collection of reference videos used for comparison, benchmarking, and instructional content. Its responsibilities are: - Ingest reference videos (upload, URL, YouTube, Vimeo) - Scene detection, metadata extraction, person/activity detection - Categorization, tagging, and segmentation - Search over the reference library (CLIP-embedding, Meilisearch, and Qdrant backends are present in tests) - Annotation, collaboration (workspaces, comments, reviews, assignments), and version history ### `cli` (Commander) Command-line tools for developers and pipeline operators. The CLI wraps the TypeScript SDK (`@aja/motion-pipeline-sdk`) and exposes `aja process`, `aja convert`, `aja inspect`, `aja debug`, `aja config`, `aja jobs`, and `aja health` commands. --- ## Processing Pipeline Architecture The diagram below maps each pipeline stage to the library that implements it, showing how data flows from raw input through to a finished animation file. ``` 1. INPUT STAGE ├── Video upload (file, URL, stream) ├── MoCap import (C3D, TRC, BVH, FBX, ASF/AMC) └── Live camera feed (WebSocket) │ ▼ 2. VIDEO ANALYSIS STAGE ├── @aja/pose-lifting 2D → 3D pose ├── @aja/depth-sensing Depth from mono/stereo ├── @aja/multi-view-reconstruction Multi-camera 3D └── @aja/human-mesh-recovery SMPL-X full body mesh │ ▼ 3. MOTION PROCESSING STAGE ├── @aja/motion-processing Filter, smooth, interpolate ├── @aja/motion-validation Validate data integrity └── @aja/motion-quality Score quality dimensions │ ▼ 4. RETARGETING STAGE ├── @aja/skeleton-mapping Map bone hierarchies ├── @aja/proportional-adaptation Adapt body proportions ├── @aja/semantic-retargeting Intent-preserving transfer ├── @aja/neural-retargeting ML-based transfer ├── @aja/blend-shape-retargeting Facial blend shapes └── @aja/animation-blending Blend and transition │ ▼ 5. EXPORT STAGE └── @aja/motion-formats Write to BVH, FBX, GLB, USD, ... ``` The capability map above illustrates the full potential of the pipeline. The orchestrated pipeline in `svc-motion-pipeline` uses a concrete `PipelineStage` enum of **11 stages**: `ingestion`, `validation`, `preprocessing`, `pose-estimation`, `skeleton-fitting`, `domain-analysis`, `quality-assessment`, `retargeting`, `format-conversion`, `postprocessing`, `delivery`. The V1 job processor (`createPipelineProcessor`, task `V1-P2-1775`) runs `ingestion`, `validation`, `preprocessing`, `quality-assessment`, `format-conversion`, `postprocessing`, and `delivery` using `@aja/motion-formats`. The four ML-only stages — `pose-estimation`, `skeleton-fitting`, `domain-analysis`, `retargeting` — are **V2-deferred** per descope decision `V1-P2-0331`; when a pipeline config enables them the processor marks each `'skipped'` with a structured log line and continues. --- ## Core Design Patterns Five architectural decisions shape how every library in Aja is built. These patterns enforce consistency, enable horizontal scaling, and protect user data. ### 1. Stage-Based Pipeline with Caching The pipeline is broken into discrete, cacheable stages. Each stage has a well-defined input and output type. `@aja/pipeline-cache` caches stage outputs using a content hash key, so a second job with the same input video does not repeat expensive pose estimation. Stages are designed to be independently testable and replaceable. ### 2. Quality-Gated Output Every pipeline run produces a `QualityReport` alongside the motion output. Applications consuming Aja can configure a minimum quality threshold — jobs below the threshold are flagged for manual review rather than automatically delivered. Quality scoring happens as a dedicated stage after processing and before retargeting, allowing early detection of inputs unlikely to produce usable output. ### 3. Format Universality `@aja/motion-formats` is the single source of truth for all format I/O. The library implements a common internal representation (`MotionClip`) that all other libraries work with, and handles serialization to/from all supported formats. This prevents format-specific code from leaking into processing or retargeting libraries. ### 4. Distributed Horizontal Scaling Heavy AI workloads are distributed via `@aja/distributed-workers` and `@aja/pipeline-parallelism`. Worker pools can be scaled horizontally on Kubernetes. `@aja/video-chunking` ensures long videos are split into processable segments before dispatch, enabling each chunk to be processed independently and in parallel. `@aja/result-aggregation` assembles the per-chunk results into a coherent output clip. ### 5. Privacy by Design `@aja/privacy-protection` is the first output consumer in the pipeline. When `privacyMode` is enabled, the source video frames are deleted from storage immediately after pose extraction completes — before any other output is written. Only anonymized skeletal data flows downstream. `@aja/consent-management` records and enforces consent per subject, allowing withdrawn consent to trigger retroactive data deletion. --- ## Library Dependency Graph (Key Paths) Understanding which libraries depend on which others is important for making changes safely. The graph below shows the most important dependency paths, derived from the `dependencies` declared in each library's `package.json`. `@aja/motion-formats` is the foundation: it has no internal Aja dependencies and everything else ultimately builds on top of it. ``` @aja/motion-pipeline-sdk └── uuid, eventsource (no @aja deps; calls svc-motion-pipeline via HTTP) @aja/neural-retargeting ├── @aja/skeleton-mapping └── @aja/optimization-ik @aja/skeleton-mapping └── @aja/motion-formats (re-exports core primitives) @aja/motion-quality └── @aja/motion-formats (re-exports core primitives) @aja/domain-motion-pipelines └── @oshun/types (defines its own motion types) @aja/film-pipeline └── @aja/motion-formats (type-only import of AnimationClip / MotionFormat) @aja/human-mesh-recovery └── tslib (no @aja deps) apps/aja/svc-motion-ai └── @aja/{motion-formats, motion-processing, motion-quality, skeleton-mapping, optimization-ik, pose-lifting, depth-sensing, human-mesh-recovery, multi-view-reconstruction, neural-retargeting, domain-motion-pipelines} apps/aja/svc-motion-pipeline └── @aja/{domain-motion-pipelines, motion-formats, motion-processing, motion-quality, motion-validation} @aja/motion-formats (no internal dependencies — foundation library) ``` --- ## Dependencies on Other Oshun Domains Aja is a **producer domain**: it consumes shared infrastructure from `@oshun/*` but does not depend on any other Oshun product domain. Instead, downstream product domains pull motion data from Aja when they need it, decoupling their release cycles from Aja's pipeline changes. The integration boundary is implemented in `@aja/motion-integration`, which ships adapter modules for exactly four domains. Each adapter translates Aja's `AnimationClip` and quality data into the format and protocol expected by the receiving domain. The `IntegrationDomain` type is `'yemaya' | 'isis' | 'bellona' | 'sophia'`, and each has a dedicated adapter under `motion-integration/src/{domain}/`. | Domain | Integration | | ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `yemaya` | `@aja/motion-integration/yemaya` adapter — film production handoff | | `isis` | `@aja/motion-integration/isis` adapter — 3D asset creation handoff | | `bellona` | `@aja/motion-integration/bellona` adapter — build/artifact handoff | | `sophia` | `@aja/motion-integration/sophia` adapter — research/knowledge handoff | | Shared `@oshun/*` | `svc-motion-pipeline` declares `@oshun/cache`, `@oshun/database`, `@oshun/event-bus`, `@oshun/queue`; `svc-motion-ai` declares `@oshun/config`, `@oshun/logging`, `@oshun/errors`, `@oshun/health` | The Yemaya boundary exists because film production requires Alembic or FBX deliverables formatted for DCC tools (Maya, Houdini, Nuke) with a production session context — a concern that belongs to Yemaya, not to Aja's core pipeline. The Isis boundary keeps 3D asset assembly separate from motion processing: Isis receives the finished animation paired with a mesh and integrates it into its asset library. The Bellona boundary enables deterministic build pipelines — motion artifacts are versioned and published to Bellona's registry so that dependent builds are reproducible. ### Consuming Domains Aja is a **producer domain** — it processes raw input into standardized motion assets consumed by downstream domains. The Yemaya `remote-film-capture` library is a verified consumer: it imports Aja libraries directly (`aja-retargeting-integration.ts`, `aja-film-pipeline-integration.ts`, `aja-multi-view-integration.ts`, and others under `libs/yemaya/remote-film-capture/src/`). Aja does not depend on other Oshun product domains, only on shared infrastructure. --- ## Build and Test Configuration All Aja projects share the same build toolchain and tag conventions. The key executor values and direct invocations for use when Nx is unavailable (e.g., due to worktree conflicts) are below. - **Build executor**: `@nx/js:tsc` - **Lint executor**: `@nx/eslint:lint` - **Test executor**: `@nx/vite:test` (Vitest for TypeScript) - **Tags**: `scope:aja`, `layer:domain`, `type:lib` Library `project.json` `name` values carry a `lilith-` prefix (e.g. `lilith-motion-formats`) reflecting the original "Lilith Motion Pipeline" authorship; the published package names are `@aja/...`. Direct invocations when Nx is unavailable: ```bash # Type check cd libs/aja/ && npx tsc --noEmit # Run TypeScript tests cd libs/aja/ && npx vitest run # Start the pipeline service directly (entry: src/server.ts) cd apps/aja/svc-motion-pipeline && npx tsx src/server.ts # Start the AI service directly (entry: src/server.ts) cd apps/aja/svc-motion-ai && npx tsx src/server.ts ``` ## Source Verification This architecture document was checked against the source under `apps/aja/*` and `libs/aja/*`: 39 library `package.json` files, the 5 app projects, the service entry points (`server.ts`), the `PipelineStage` enum and `V2_DEFERRED_STAGES` set in `svc-motion-pipeline`, the `motion-integration` `IntegrationDomain` type, and the verified Yemaya consumer under `libs/yemaya/remote-film-capture/`. Project / package-name discrepancies (`@lilith/...` labels) are noted where they occur.