Eve library

Aja Domain Architecture

Guide

Revision 1

On this page

Canonical source: the canonical architecture document for this domain is deep-dive/architecture.md, rendered in the docs center. This file is its operational companion (service topology, APIs, schemas, runtime detail) — reconciled by linking, not duplicated (§2, §13).

Aja -- Motion AI and Animation Pipeline Platform

Named after the Yoruba Orisha Aja (forest guide), this domain provides an end-to-end motion processing pipeline: from raw video or motion capture input through pose estimation, skeletal retargeting, quality assessment, and format conversion to production-ready animation files.


System Overview#

Aja is a motion-intelligence platform consisting of 4 service applications and 38 libraries organized into 7 functional categories. The platform accepts video, motion capture data, or live camera feeds and transforms them into clean, retargeted animation data in industry-standard formats.

Core Value Proposition#

  1. Video-to-Motion -- Extract skeletal animation from ordinary video using AI pose estimation, depth sensing, and human mesh recovery.
  2. Motion Quality -- Automatically assess, clean, and validate motion data through noise reduction, physics cleanup, foot sliding correction, and QA pipelines.
  3. Format Universality -- Import and export across 13+ animation formats (BVH, FBX, GLB, USD, Alembic, C3D, TRC, ASF/AMC, Collada, MDD, PC2, OBJ sequences, custom JSON).
  4. Domain Specialization -- Pre-configured pipelines for yoga, fitness, dance, martial arts, and general motion with domain-specific analysis.

Service Architecture#

Applications#

text
apps/aja/
  svc-motion-pipeline/    Fastify     :8090    Core pipeline orchestration
  svc-motion-ai/          Hono        :3040    AI motion processing
  svc-reference-video/    (library)   --       Video reference management
  cli/                    Commander   --       Command-line tools
Application Framework Port Description
svc-motion-pipeline Fastify 8090 Pipeline orchestration, job management, configuration, result delivery (webhooks, SSE, polling, downloads)
svc-motion-ai Hono 3040 AI-powered motion processing: video-to-motion conversion, retargeting, skeletal animation, live capture, mobile capture, motion enhancement, video analysis
svc-reference-video -- (library) -- Reference video library: ingestion, metadata extraction, scene detection, categorization, search (CLIP/vector), annotation, collaboration
cli Commander.js -- CLI tool (aja / aja-motion) with commands: process, convert, inspect, debug, config, jobs, health

svc-motion-pipeline -- API Endpoints#

This is the primary orchestration service. Built on Fastify with @lilith/fastify-core.

Health and Status:

Method Path Description
GET /health Service health (components, uptime)
GET /v1/pipeline/stats Queue statistics (waiting, active, completed, failed)

Pipeline Configuration:

Method Path Description
GET /v1/pipeline/configs List configurations (filter by domain, quality preset)
GET /v1/pipeline/configs/:configId Get configuration by ID
GET /v1/pipeline/configs/by-name/:name Get configuration by name
POST /v1/pipeline/configs Create configuration (JSON)
POST /v1/pipeline/configs/yaml Create configuration (YAML)
GET /v1/pipeline/configs/:configId/yaml Export configuration as YAML
DELETE /v1/pipeline/configs/:configId Delete configuration
GET /v1/pipeline/presets List quality presets (draft, standard, high, ultra)

Job Management:

Method Path Description
POST /v1/pipeline/jobs Submit processing job (returns 202 with pollUrl/streamUrl)
GET /v1/pipeline/jobs List jobs (filter by status, domain, tags; pagination)
GET /v1/pipeline/jobs/:jobId Get job status (polling endpoint)
POST /v1/pipeline/jobs/:jobId/cancel Cancel job
POST /v1/pipeline/jobs/:jobId/retry Retry failed job
PATCH /v1/pipeline/jobs/:jobId/priority Update job priority
POST /v1/pipeline/jobs/batch Batch operations (cancel, retry, prioritize)

Result Delivery:

Method Path Description
GET /v1/pipeline/jobs/:jobId/outputs List available outputs (completed jobs)
POST /v1/pipeline/jobs/:jobId/download Generate signed download URL
GET /v1/pipeline/jobs/:jobId/stream SSE progress stream
GET /v1/pipeline/streams/stats Active stream connection count

Webhooks:

Method Path Description
GET /v1/pipeline/jobs/:jobId/webhooks Get webhook deliveries for job
GET /v1/pipeline/webhooks/:deliveryId Get webhook delivery status
POST /v1/pipeline/webhooks/:deliveryId/retry Retry webhook delivery
GET /v1/pipeline/webhooks/stats Webhook service statistics

svc-motion-ai -- API Endpoints#

Built on Hono with CORS and logger middleware.

Method Path Description
GET /health Health check
POST /api/v1/video-to-motion/process Convert video to motion data
POST /api/v1/retargeting/retarget Retarget motion to different skeleton
POST /api/v1/skeletal/animate Generate skeletal animation
POST /api/v1/analysis/analyze Analyze video content

The service internally contains 7 modules:

  • video-to-motion -- Video preprocessing, pose estimation engine, keypoint normalization, temporal processing
  • live-motion-capture -- Webcam capture, realtime pose estimation, realtime retargeting, streaming pose lift, live avatar driving
  • mobile-live-capture -- iOS capture, Android capture, on-device processing, mobile streaming
  • motion-retargeting -- Skeleton-to-skeleton retargeting service
  • motion-enhancement -- Motion super-resolution, style transfer, physics refinement, motion synthesis
  • skeletal-animation-motion -- Skeletal animation generation
  • video-analysis -- Video content analysis

CLI Commands#

The aja CLI connects to the pipeline service via @aja/motion-pipeline-sdk.

Command Description
aja process [input...] Submit videos for motion extraction (supports glob, URL, interactive mode)
aja convert <input> <output> Batch format conversion between motion formats
aja inspect <file> Quality inspection and validation of motion files
aja debug Pipeline debugging and troubleshooting
aja config Manage pipeline configurations
aja jobs View and manage processing jobs
aja health Check API service health

Library Architecture#

Dependency Graph (Simplified)#

text
                         +--------------------+
                         | svc-motion-pipeline|
                         +---------+----------+
                                   |
         +------------+------------+------------+
         |            |            |            |
   motion-formats  motion-    motion-    domain-motion-
                   processing quality    pipelines
         |            |            |            |
         +------+-----+            |     +------+------+
                |                  |     |      |      |
          motion-validation        |   yoga  fitness  dance  martial-arts
                                   |
                         +---------+---------+
                         |         |         |
                   skeleton-  optimization- avatar-
                   mapping    ik            library

                         +------------------+
                         | svc-motion-ai    |
                         +---------+--------+
                                   |
     +----------+----------+-------+--------+----------+
     |          |          |       |        |          |
  pose-     depth-    human-mesh  multi-   neural-  motion-formats
  lifting   sensing   recovery    view-    retargeting
                                  recon

Category 1: Core Motion Processing (5 libs)#

Library Package Purpose
motion-formats @aja/motion-formats Multi-format import/export with parsers/writers for BVH, FBX, GLB/glTF, USD/USDA/USDC/USDZ, Alembic, C3D, TRC, ASF/AMC. Includes importAnimation(), exportAnimation(), convertAnimation(), detectFormat(), and motion clip database.
motion-processing @aja/motion-processing 6-stage processing pipeline: noise reduction (FFT, wavelets, adaptive), foot sliding correction (IK solvers: TwoBone, FABRIK, CCD), physics cleanup (CoM, angular momentum, balance, joint torque limits), retiming (time warps, beat sync), gap filling (short/long, keyframe insertion), segmentation (action, transitions, loop points). Exposed as MotionProcessingPipeline class and quickCleanup()/fullProcessing() convenience functions.
motion-quality @aja/motion-quality Quality assessment: basic metrics (jitter, foot sliding, bone length, joint angles, smoothness, plausibility), perceptual metrics (naturalness, style consistency, temporal coherence, visual quality), ground truth comparison (MPJPE, PA-MPJPE, PCK, AUC), and automated QA pipeline with pass/reject/review decisions.
motion-validation @aja/motion-validation Data validation and error detection for motion data.
motion-integration @aja/motion-integration Integration layer connecting processing stages.

Category 2: Video Analysis (4 libs)#

Library Package Purpose
pose-lifting @aja/pose-lifting 2D-to-3D pose lifting from single or multi-view video
depth-sensing @aja/depth-sensing Monocular and stereo depth estimation
multi-view-reconstruction @aja/multi-view-reconstruction Multi-camera 3D reconstruction
human-mesh-recovery @aja/human-mesh-recovery SMPL/SMPL-X parametric mesh recovery from video

Category 3: Skeletal and Avatar (6 libs)#

Library Package Purpose
skeleton-mapping @aja/skeleton-mapping Skeleton hierarchy conversion between different standards
optimization-ik @aja/optimization-ik Inverse kinematics solvers (analytical + iterative)
avatar-library @aja/avatar-library Avatar definitions and presets
avatar-integration @aja/avatar-integration Avatar binding and animation
avatar-preview-ui @aja/avatar-preview-ui React components for avatar preview
bone-mapping-ui @aja/bone-mapping-ui Interactive bone mapping UI

Category 4: Retargeting and Blending (5 libs)#

Library Package Purpose
semantic-retargeting @aja/semantic-retargeting Semantic-aware motion retargeting
proportional-adaptation @aja/proportional-adaptation Body proportion adaptation across different morphologies
blend-shape-retargeting @aja/blend-shape-retargeting Facial blend shape retargeting
neural-retargeting @aja/neural-retargeting ML-based retargeting using neural networks
animation-blending @aja/animation-blending Motion blending, transitions, and interpolation

Category 5: Domain Pipelines (2 libs)#

Library Package Purpose
domain-motion-pipelines @aja/domain-motion-pipelines Factory for domain-specific pipelines (yoga, fitness, dance, martial-arts). Each pipeline provides detection, recognition, form analysis, and tracking specific to its domain.
fitness-animation @aja/fitness-animation Fitness-specific analysis (exercise recognition, rep counting, form analysis)

Domain Pipeline Capabilities:

Pipeline Key Services
Yoga Asana detection/recognition, alignment analysis, breath detection, drishti tracking, bandha analysis, meditation state detection, pranayama recognition, sequence tracking
Fitness Exercise recognition, rep counting with phase detection, form analysis with real-time feedback, workout tracking (sets/reps), equipment detection
Dance Style detection (ballet, hip-hop, salsa, etc.), beat detection, beat-movement synchronization, choreography segmentation, expression analysis, musicality scoring
Martial Arts Style detection (boxing, karate, BJJ, etc.), strike detection/classification, power/speed estimation, stance analysis, guard position analysis, footwork patterns, combat sequence tracking

Category 6: Infrastructure (8 libs)#

Library Package Purpose
motion-pipeline-sdk @aja/motion-pipeline-sdk TypeScript SDK for pipeline service API
motion-pipeline-sdk-python @aja/motion-pipeline-sdk-python Python SDK with Jupyter notebook support
batch-inference @aja/batch-inference Batch processing for large motion datasets
video-chunking @aja/video-chunking Video splitting for parallel processing
model-optimization @aja/model-optimization ML model compression and optimization
pipeline-parallelism @aja/pipeline-parallelism Distributed pipeline stage execution
distributed-workers @aja/distributed-workers Worker coordination for distributed processing
result-aggregation @aja/result-aggregation Result combining from distributed workers

Category 7: Security and Compliance (6 libs)#

Library Package Purpose
privacy-protection @aja/privacy-protection Privacy-preserving processing (face blurring, body anonymization)
consent-management @aja/consent-management Consent tracking and enforcement
content-security @aja/content-security Security protocols for motion data
content-watermarking @aja/content-watermarking Digital watermarking for motion files
content-moderation @aja/content-moderation Content filtering and safety
data-retention @aja/data-retention Retention policy enforcement

Utility (1 lib)#

Library Package Purpose
pipeline-cache @aja/pipeline-cache LRU caching for pipeline stages

Data Flow#

Video-to-Animation Pipeline#

text
  Input                  Video Analysis         Motion Processing
  ======                 ==============         ================

  +--------+   chunk    +-------------+  raw    +-------------+
  | Video  | --------->| Pose        |-------->| Noise       |
  | File   |   split   | Estimation  | poses   | Reduction   |
  +--------+           +-------------+         +------+------+
                              |                       |
  +--------+           +------v------+         +------v------+
  | MoCap  |           | Depth       |         | Gap         |
  | C3D/TRC| --------->| Sensing     |         | Filling     |
  +--------+  import   +------+------+         +------+------+
                              |                       |
  +--------+           +------v------+         +------v------+
  | Anim   |           | Human Mesh  |         | Foot Slide  |
  | BVH/FBX| --------->| Recovery    |         | Correction  |
  +--------+  import   +------+------+         +------+------+
                              |                       |
  +--------+           +------v------+         +------v------+
  | Live   |  stream   | Multi-View  |         | Physics     |
  | Camera | --------->| Reconstruct |         | Cleanup     |
  +--------+           +-------------+         +------+------+
                                                      |

  Retargeting              Domain Analysis        Output
  ===========              ===============        ======

  +-------------+         +-------------+        +--------+
  | Skeleton    |-------->| Yoga/Fitness/|------->| BVH    |
  | Mapping     |         | Dance/       |        +--------+
  +------+------+         | Martial Arts |        +--------+
         |                +------+------+  export | FBX    |
  +------v------+               |          ----->| +--------+
  | Proportional|         +------v------+        +--------+
  | Adaptation  |         | Quality     |        | GLB    |
  +------+------+         | Assessment  |        +--------+
         |                +------+------+        +--------+
  +------v------+               |                | USD    |
  | Neural      |         +------v------+        +--------+
  | Retarget    |         | QA Pipeline |        +--------+
  +------+------+         | Pass/Reject |        |Alembic |
         |                +-------------+        +--------+
  +------v------+
  | Animation   |
  | Blending    |
  +-------------+

Job Lifecycle#

text
  Client                svc-motion-pipeline             svc-motion-ai
  ======                ===================             =============

  POST /jobs --------> JobManager.submit()
                        |
                        v
                    PriorityQueue.enqueue()
                        |
                        v (processing loop)
                    JobProcessor.processJob()
                        |
                        +-- updateStage("ingestion")
                        +-- updateStage("validation")
                        +-- updateStage("pose-estimation") --> AI Service
                        +-- updateStage("domain-analysis")
                        +-- updateStage("quality-assessment")
                        +-- updateStage("retargeting")
                        +-- updateStage("format-conversion")
                        |
                        v
                    completeJob(result)
                        |
                        +---> Webhook delivery
                        +---> SSE stream
                        +---> Poll response
                        +---> Signed download URL

Pipeline Stages#

The processing pipeline consists of up to 11 configurable stages:

Stage Description Configurable
ingestion Accept and validate input files timeout, retry
validation Validate data integrity and format timeout, retry
preprocessing Prepare data for processing timeout, retry
pose-estimation AI pose estimation from video timeout, retry, parallel
skeleton-fitting Fit skeleton to estimated poses timeout, retry
domain-analysis Domain-specific analysis (yoga/fitness/dance/martial-arts) timeout, retry
quality-assessment Quality metrics and QA decisions timeout, retry
retargeting Map motion to target skeleton timeout, retry
format-conversion Convert to output formats timeout, retry
postprocessing Final cleanup and optimization timeout, retry
delivery Store results and notify timeout, retry

Each stage supports:

  • Enable/disable toggle
  • Per-stage timeout
  • Retry with exponential/linear/fixed backoff
  • Stage dependencies (must-complete-before)
  • Parallel execution flag

Quality Presets#

Preset Processing Time Target Quality Frame Rate Key Features
draft 0.5x baseline 0.6 24 fps Basic noise reduction
standard 1.0x baseline 0.8 30 fps Noise reduction + smoothing + jitter correction
high 2.0x baseline 0.9 60 fps All standard + foot sliding + contact detection + physics
ultra 4.0x baseline 0.95 120 fps All high + IK cleanup + multi-view fusion

Cross-Domain Dependencies#

Shared @oshun/* Libraries Used#

Library Used By
@oshun/cache svc-motion-pipeline
@oshun/database svc-motion-pipeline
@oshun/event-bus svc-motion-pipeline
@oshun/queue svc-motion-pipeline
@oshun/config svc-motion-ai
@oshun/logging svc-motion-ai
@oshun/errors svc-motion-ai
@oshun/health svc-motion-ai

Shared @lilith/* Libraries Used#

Library Used By
@lilith/fastify-core svc-motion-pipeline (service server bootstrap)

Technology Stack#

Layer Technologies
Pipeline Service Fastify 4.x, @lilith/fastify-core, uuid, js-yaml, zod
AI Service Hono, @hono/node-server
CLI Commander.js 12.x, chalk 5.x, inquirer 9.x, ora 8.x, yaml
SDKs TypeScript SDK, Python SDK with Jupyter support
Motion Formats Custom parsers/writers for 13+ formats
ML Models Pose estimation, depth sensing, mesh recovery (SMPL-X)
Job Management In-memory priority queue with event-driven processing
Result Delivery Webhooks (HMAC signed), SSE streaming, polling, signed download URLs
Build Nx with @nx/js:tsc and nx:run-commands
Testing Vitest
Runtime Node.js with ESM modules

Deployment Architecture#

text
                    +------------------+
                    |   Load Balancer  |
                    +--------+---------+
                             |
              +--------------+--------------+
              |                             |
     +--------v--------+          +--------v--------+
     | svc-motion-     |          | svc-motion-ai   |
     | pipeline :8090  |          |            :3040 |
     +--------+--------+          +--------+--------+
              |                            |
              |   +-----------+            |
              +-->| Redis     |            |
              |   | (queue/   |            |
              |   |  cache)   |            |
              |   +-----------+            |
              |                            |
              |   +-----------+            |
              +-->| S3/MinIO  |<-----------+
              |   | (storage) |
              |   +-----------+
              |
              |   +-----------+
              +-->| PostgreSQL|
                  | (jobs/    |
                  |  configs) |
                  +-----------+

Services are deployed via the shared ECS deployment pipeline (deploy-ecs.yml). The pipeline service coordinates job execution and stores results in S3-compatible storage. Redis handles job queuing and caching. PostgreSQL stores job metadata and pipeline configurations.


Configuration#

Environment Variables#

bash
# svc-motion-pipeline
PORT=8090                          # Service port
HOST=0.0.0.0                      # Bind address
MAX_CONCURRENT_JOBS=10             # Concurrent job limit
DEFAULT_JOB_TIMEOUT_MS=1800000    # 30-minute default timeout

# svc-motion-ai
PORT=3040                          # Service port

# CLI
AJA_API_URL=http://localhost:3000  # Pipeline API URL
AJA_API_KEY=...                    # API key for authentication

Pipeline Configuration (YAML)#

Pipelines can be defined as YAML:

yaml
apiVersion: v1
kind: PipelineDefinition
metadata:
  name: yoga-high-quality
  labels:
    domain: yoga
    quality: high
spec:
  domain: yoga
  qualityPreset: high
  stages:
    - name: ingestion
      type: ingestion
      enabled: true
      timeout: 60s
      retry:
        maxAttempts: 3
        backoff: exponential
        initialDelay: 1s
        maxDelay: 30s
    - name: pose-estimation
      type: pose-estimation
      enabled: true
      dependsOn: [ingestion]
      timeout: 300s
  input:
    formats: [mp4, mov, avi]
    frameRate:
      min: 24
      max: 120
    duration:
      min: 1s
      max: 3600s
    maxSize: 2GB
  output:
    format: fbx
    additionalFormats: [bvh, glb]
    compression: gzip
    includeAnalytics: true
  settings:
    timeout: 1800s
    failFast: true

Development#

bash
# Start pipeline service in dev mode
pnpm nx dev @aja/svc-motion-pipeline

# Start AI service in dev mode
pnpm nx dev aja-svc-motion-ai

# Run CLI
pnpm nx dev @aja/cli -- process video.mp4 --preset high --domain yoga

# Run tests for specific lib
pnpm nx test @aja/motion-formats

# Build all Aja projects
pnpm nx run-many --target=build --projects=tag:scope:aja

# Lint all Aja projects
pnpm nx run-many --target=lint --projects=tag:scope:aja