Eve library

Aja - Motion & Animation Domain

Other

Revision 1

On this page

Aja is the motion, animation, and kinematics domain of the Oshun monorepo. Named after the Yoruba Orisha of the forest who guides travelers through wilderness, Aja provides comprehensive tools for processing, analyzing, and transforming human motion data.

Overview#

Aja provides a complete motion processing pipeline from video input to production-ready animation output, including:

  • Video Analysis: Pose estimation, depth sensing, multi-view reconstruction
  • Motion Capture: Human mesh recovery, pose lifting, temporal modeling
  • Motion Processing: Filtering, retargeting, blending, compression
  • Quality Assessment: Metrics, validation, error detection
  • Avatar Integration: Skeleton mapping, proportional adaptation, blend shapes
  • Pipeline Orchestration: End-to-end processing with configurable stages

Libraries#

Core Motion Processing#

Library Package Description
motion-formats @aja/motion-formats BVH, FBX, GLB, C3D format parsing and serialization
motion-processing @aja/motion-processing Filters, smoothing, interpolation, normalization
motion-quality @aja/motion-quality Quality metrics, assessment, and scoring
motion-validation @aja/motion-validation Motion data validation and error detection
motion-integration @aja/motion-integration Integration layer for motion pipeline

Video Analysis#

Library Package Description
pose-lifting @aja/pose-lifting 2D to 3D pose lifting algorithms
depth-sensing @aja/depth-sensing Monocular and stereo depth estimation
multi-view-reconstruction @aja/multi-view-reconstruction Multi-camera 3D reconstruction
human-mesh-recovery @aja/human-mesh-recovery SMPL/SMPL-X mesh recovery from video

Skeletal & Avatar#

Library Package Description
skeleton-mapping @aja/skeleton-mapping Skeleton hierarchy mapping and conversion
optimization-ik @aja/optimization-ik Inverse kinematics solvers
avatar-library @aja/avatar-library Avatar definitions and presets
avatar-integration @aja/avatar-integration Avatar binding and animation application
avatar-preview-ui @aja/avatar-preview-ui React components for avatar preview
bone-mapping-ui @aja/bone-mapping-ui Interactive bone mapping interface

Retargeting & Blending#

Library Package Description
semantic-retargeting @aja/semantic-retargeting Semantic motion retargeting
proportional-adaptation @aja/proportional-adaptation Body proportion adaptation
blend-shape-retargeting @aja/blend-shape-retargeting Facial blend shape retargeting
neural-retargeting @aja/neural-retargeting ML-based motion retargeting
animation-blending @aja/animation-blending Motion blending and transitions

Domain-Specific Pipelines#

Library Package Description
domain-motion-pipelines @aja/domain-motion-pipelines Pre-configured pipelines for fitness, dance, sports, etc.
fitness-animation @aja/fitness-animation Specialized fitness motion analysis

SDKs#

Library Package Description
motion-pipeline-sdk @aja/motion-pipeline-sdk TypeScript SDK for pipeline service
motion-pipeline-sdk-python aja-motion-pipeline Python SDK with Jupyter, data science, ML training support

Installation#

bash
# Install all Aja libraries
pnpm install

# Build all Aja libraries
pnpm nx run-many -t build -p @aja/*

# Test all Aja libraries
pnpm nx run-many -t test -p @aja/*

Usage#

Basic Motion Processing#

typescript
import { parseBVH, serializeGLB } from '@aja/motion-formats';
import { applyButterworth, normalizeMotion } from '@aja/motion-processing';
import { assessQuality } from '@aja/motion-quality';

// Load motion data
const motion = parseBVH(bvhContent);

// Process
const filtered = applyButterworth(motion, { cutoffHz: 6 });
const normalized = normalizeMotion(filtered);

// Assess quality
const metrics = assessQuality(normalized);
console.log(`Quality score: ${metrics.overallScore}`);

// Export
const glbBuffer = serializeGLB(normalized);

Video-to-Motion Pipeline#

typescript
import { MotionPipelineClient, SDKConfig } from '@aja/motion-pipeline-sdk';

const client = new MotionPipelineClient(
  SDKConfig.create({
    baseUrl: 'https://api.aja.example.com',
    auth: { type: 'api-key', apiKey: 'your-key' },
  })
);

// Submit video for processing
const job = await client
  .jobBuilder()
  .fromUrl('https://example.com/video.mp4')
  .withConfig('yoga-high')
  .priority('high')
  .submit();

// Wait for completion
const result = await client.waitForJob(job.jobId);

Python SDK#

python
from aja_motion_pipeline import (
    MotionPipelineClient,
    SDKConfig,
    AuthConfig,
    AuthType,
)

client = MotionPipelineClient(
    SDKConfig(
        base_url="https://api.aja.example.com",
        auth=AuthConfig(type=AuthType.API_KEY, api_key="your-key"),
    )
)

result = (
    client.job_builder()
    .from_url("https://example.com/video.mp4")
    .with_config("yoga-high")
    .submit()
)

Avatar Integration#

typescript
import { mapSkeleton, createMapping } from '@aja/skeleton-mapping';
import { applyMotion } from '@aja/avatar-integration';
import { getAvatar } from '@aja/avatar-library';

// Get avatar definition
const avatar = getAvatar('mixamo-ybot');

// Create skeleton mapping
const mapping = createMapping(motion.skeleton, avatar.skeleton);

// Apply motion to avatar
const avatarMotion = applyMotion(motion, mapping, avatar);

Architecture#

text
┌─────────────────────────────────────────────────────────────────┐
│                         Input Sources                           │
│  ┌─────────┐  ┌─────────┐  ┌─────────┐  ┌─────────────────┐   │
│  │  Video  │  │  BVH    │  │  FBX    │  │  Motion Capture │   │
│  └────┬────┘  └────┬────┘  └────┬────┘  └────────┬────────┘   │
└───────┼────────────┼────────────┼────────────────┼─────────────┘
        │            │            │                │
        ▼            ▼            ▼                ▼
┌─────────────────────────────────────────────────────────────────┐
│                    Video Analysis Layer                         │
│  ┌──────────────┐  ┌──────────────┐  ┌────────────────────┐   │
│  │ Pose Lifting │  │ Depth Sense  │  │ Multi-View Recon   │   │
│  └──────┬───────┘  └──────┬───────┘  └─────────┬──────────┘   │
│         │                 │                    │               │
│         ▼                 ▼                    ▼               │
│  ┌───────────────────────────────────────────────────────┐    │
│  │              Human Mesh Recovery (SMPL-X)             │    │
│  └───────────────────────────┬───────────────────────────┘    │
└──────────────────────────────┼─────────────────────────────────┘
                               │
                               ▼
┌─────────────────────────────────────────────────────────────────┐
│                   Motion Processing Layer                       │
│  ┌─────────────┐  ┌──────────────┐  ┌──────────────────┐      │
│  │  Filtering  │→│  Validation  │→│  Quality Check   │      │
│  └─────────────┘  └──────────────┘  └──────────────────┘      │
└─────────────────────────────┬───────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                   Retargeting Layer                             │
│  ┌─────────────────┐  ┌────────────────┐  ┌────────────────┐  │
│  │ Skeleton Mapping │  │ Proportional   │  │ Neural         │  │
│  │                  │  │ Adaptation     │  │ Retargeting    │  │
│  └────────┬─────────┘  └───────┬────────┘  └───────┬────────┘  │
│           │                    │                   │           │
│           ▼                    ▼                   ▼           │
│  ┌───────────────────────────────────────────────────────┐    │
│  │                  Animation Blending                   │    │
│  └───────────────────────────┬───────────────────────────┘    │
└──────────────────────────────┼─────────────────────────────────┘
                               │
                               ▼
┌─────────────────────────────────────────────────────────────────┐
│                      Output Formats                             │
│  ┌─────────┐  ┌─────────┐  ┌─────────┐  ┌─────────────────┐   │
│  │  BVH    │  │  FBX    │  │  GLB    │  │  Engine-Native  │   │
│  └─────────┘  └─────────┘  └─────────┘  └─────────────────┘   │
└─────────────────────────────────────────────────────────────────┘

Domain Pipelines#

Aja includes pre-configured pipelines optimized for specific domains:

  • Fitness: Exercise form analysis, rep counting, range of motion
  • Dance: Choreography capture, style transfer, synchronization
  • Sports: Performance analysis, technique breakdown, comparison
  • Medical: Gait analysis, rehabilitation tracking, clinical assessment
  • Gaming: Real-time motion capture, gesture recognition
  • Film/VFX: High-fidelity capture, facial performance, body doubles

Quality Metrics#

The quality assessment system evaluates:

  • Temporal Smoothness: Jitter, acceleration consistency
  • Anatomical Plausibility: Joint limits, bone lengths, self-penetration
  • Ground Contact: Foot sliding, ground plane alignment
  • Semantic Accuracy: Pose matching, action recognition confidence

License#

MIT