Aja is the motion, animation, and kinematics domain of the Oshun monorepo. Named after the Yoruba Orisha of the forest who guides travelers through wilderness, Aja provides comprehensive tools for processing, analyzing, and transforming human motion data.
Overview#
Aja provides a complete motion processing pipeline from video input to production-ready animation output, including:
- Video Analysis: Pose estimation, depth sensing, multi-view reconstruction
- Motion Capture: Human mesh recovery, pose lifting, temporal modeling
- Motion Processing: Filtering, retargeting, blending, compression
- Quality Assessment: Metrics, validation, error detection
- Avatar Integration: Skeleton mapping, proportional adaptation, blend shapes
- Pipeline Orchestration: End-to-end processing with configurable stages
Libraries#
Core Motion Processing#
| Library | Package | Description |
|---|---|---|
| motion-formats | @aja/motion-formats |
BVH, FBX, GLB, C3D format parsing and serialization |
| motion-processing | @aja/motion-processing |
Filters, smoothing, interpolation, normalization |
| motion-quality | @aja/motion-quality |
Quality metrics, assessment, and scoring |
| motion-validation | @aja/motion-validation |
Motion data validation and error detection |
| motion-integration | @aja/motion-integration |
Integration layer for motion pipeline |
Video Analysis#
| Library | Package | Description |
|---|---|---|
| pose-lifting | @aja/pose-lifting |
2D to 3D pose lifting algorithms |
| depth-sensing | @aja/depth-sensing |
Monocular and stereo depth estimation |
| multi-view-reconstruction | @aja/multi-view-reconstruction |
Multi-camera 3D reconstruction |
| human-mesh-recovery | @aja/human-mesh-recovery |
SMPL/SMPL-X mesh recovery from video |
Skeletal & Avatar#
| Library | Package | Description |
|---|---|---|
| skeleton-mapping | @aja/skeleton-mapping |
Skeleton hierarchy mapping and conversion |
| optimization-ik | @aja/optimization-ik |
Inverse kinematics solvers |
| avatar-library | @aja/avatar-library |
Avatar definitions and presets |
| avatar-integration | @aja/avatar-integration |
Avatar binding and animation application |
| avatar-preview-ui | @aja/avatar-preview-ui |
React components for avatar preview |
| bone-mapping-ui | @aja/bone-mapping-ui |
Interactive bone mapping interface |
Retargeting & Blending#
| Library | Package | Description |
|---|---|---|
| semantic-retargeting | @aja/semantic-retargeting |
Semantic motion retargeting |
| proportional-adaptation | @aja/proportional-adaptation |
Body proportion adaptation |
| blend-shape-retargeting | @aja/blend-shape-retargeting |
Facial blend shape retargeting |
| neural-retargeting | @aja/neural-retargeting |
ML-based motion retargeting |
| animation-blending | @aja/animation-blending |
Motion blending and transitions |
Domain-Specific Pipelines#
| Library | Package | Description |
|---|---|---|
| domain-motion-pipelines | @aja/domain-motion-pipelines |
Pre-configured pipelines for fitness, dance, sports, etc. |
| fitness-animation | @aja/fitness-animation |
Specialized fitness motion analysis |
SDKs#
| Library | Package | Description |
|---|---|---|
| motion-pipeline-sdk | @aja/motion-pipeline-sdk |
TypeScript SDK for pipeline service |
| motion-pipeline-sdk-python | aja-motion-pipeline |
Python SDK with Jupyter, data science, ML training support |
Installation#
bash
# Install all Aja libraries
pnpm install
# Build all Aja libraries
pnpm nx run-many -t build -p @aja/*
# Test all Aja libraries
pnpm nx run-many -t test -p @aja/*
Usage#
Basic Motion Processing#
typescript
import { parseBVH, serializeGLB } from '@aja/motion-formats';
import { applyButterworth, normalizeMotion } from '@aja/motion-processing';
import { assessQuality } from '@aja/motion-quality';
// Load motion data
const motion = parseBVH(bvhContent);
// Process
const filtered = applyButterworth(motion, { cutoffHz: 6 });
const normalized = normalizeMotion(filtered);
// Assess quality
const metrics = assessQuality(normalized);
console.log(`Quality score: ${metrics.overallScore}`);
// Export
const glbBuffer = serializeGLB(normalized);
Video-to-Motion Pipeline#
typescript
import { MotionPipelineClient, SDKConfig } from '@aja/motion-pipeline-sdk';
const client = new MotionPipelineClient(
SDKConfig.create({
baseUrl: 'https://api.aja.example.com',
auth: { type: 'api-key', apiKey: 'your-key' },
})
);
// Submit video for processing
const job = await client
.jobBuilder()
.fromUrl('https://example.com/video.mp4')
.withConfig('yoga-high')
.priority('high')
.submit();
// Wait for completion
const result = await client.waitForJob(job.jobId);
Python SDK#
python
from aja_motion_pipeline import (
MotionPipelineClient,
SDKConfig,
AuthConfig,
AuthType,
)
client = MotionPipelineClient(
SDKConfig(
base_url="https://api.aja.example.com",
auth=AuthConfig(type=AuthType.API_KEY, api_key="your-key"),
)
)
result = (
client.job_builder()
.from_url("https://example.com/video.mp4")
.with_config("yoga-high")
.submit()
)
Avatar Integration#
typescript
import { mapSkeleton, createMapping } from '@aja/skeleton-mapping';
import { applyMotion } from '@aja/avatar-integration';
import { getAvatar } from '@aja/avatar-library';
// Get avatar definition
const avatar = getAvatar('mixamo-ybot');
// Create skeleton mapping
const mapping = createMapping(motion.skeleton, avatar.skeleton);
// Apply motion to avatar
const avatarMotion = applyMotion(motion, mapping, avatar);
Architecture#
text
┌─────────────────────────────────────────────────────────────────┐
│ Input Sources │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────────────┐ │
│ │ Video │ │ BVH │ │ FBX │ │ Motion Capture │ │
│ └────┬────┘ └────┬────┘ └────┬────┘ └────────┬────────┘ │
└───────┼────────────┼────────────┼────────────────┼─────────────┘
│ │ │ │
▼ ▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────┐
│ Video Analysis Layer │
│ ┌──────────────┐ ┌──────────────┐ ┌────────────────────┐ │
│ │ Pose Lifting │ │ Depth Sense │ │ Multi-View Recon │ │
│ └──────┬───────┘ └──────┬───────┘ └─────────┬──────────┘ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌───────────────────────────────────────────────────────┐ │
│ │ Human Mesh Recovery (SMPL-X) │ │
│ └───────────────────────────┬───────────────────────────┘ │
└──────────────────────────────┼─────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ Motion Processing Layer │
│ ┌─────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ Filtering │→│ Validation │→│ Quality Check │ │
│ └─────────────┘ └──────────────┘ └──────────────────┘ │
└─────────────────────────────┬───────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ Retargeting Layer │
│ ┌─────────────────┐ ┌────────────────┐ ┌────────────────┐ │
│ │ Skeleton Mapping │ │ Proportional │ │ Neural │ │
│ │ │ │ Adaptation │ │ Retargeting │ │
│ └────────┬─────────┘ └───────┬────────┘ └───────┬────────┘ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌───────────────────────────────────────────────────────┐ │
│ │ Animation Blending │ │
│ └───────────────────────────┬───────────────────────────┘ │
└──────────────────────────────┼─────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ Output Formats │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────────────┐ │
│ │ BVH │ │ FBX │ │ GLB │ │ Engine-Native │ │
│ └─────────┘ └─────────┘ └─────────┘ └─────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
Domain Pipelines#
Aja includes pre-configured pipelines optimized for specific domains:
- Fitness: Exercise form analysis, rep counting, range of motion
- Dance: Choreography capture, style transfer, synchronization
- Sports: Performance analysis, technique breakdown, comparison
- Medical: Gait analysis, rehabilitation tracking, clinical assessment
- Gaming: Real-time motion capture, gesture recognition
- Film/VFX: High-fidelity capture, facial performance, body doubles
Quality Metrics#
The quality assessment system evaluates:
- Temporal Smoothness: Jitter, acceleration consistency
- Anatomical Plausibility: Joint limits, bone lengths, self-penetration
- Ground Contact: Foot sliding, ground plane alignment
- Semantic Accuracy: Pose matching, action recognition confidence
License#
MIT