Canonical source: the canonical features document for this domain is
deep-dive/features.md, rendered in the docs
center. This file is its operational companion (service topology, APIs,
schemas, runtime detail) — reconciled by linking, not duplicated (§2, §13).
Motion AI and Animation Platform
Aja transforms ordinary video into production-ready animation. Upload a video,
and Aja extracts skeletal motion, cleans and retargets it, assesses quality, and
exports in any major animation format. The platform also supports live motion
capture, domain-specific movement analysis (fitness, dance, sports, medical),
and batch processing at scale. This document describes every capability
available to users.
Table of Contents#
- Video-to-Motion Conversion
- Motion Capture Processing
- Live Motion Capture
- Body, Hand & Face Tracking
- Motion Retargeting
- Motion Processing & Cleanup
- Motion Quality Assessment
- Format Conversion
- Domain-Specific Pipelines
- Reference Video Management
- AI Motion Features
- Skeleton & Avatar Management
- Batch Processing & Scale
- Pipeline Configuration
- Job Management
- Result Delivery
- CLI Tools
- SDKs & Developer Tools
- Privacy & Security
1. Video-to-Motion Conversion#
The core Aja feature: convert any video recording into skeletal animation data.
| Feature |
Description |
| Video formats |
MP4, MOV, AVI, and other standard video formats |
| Frame rate range |
24 FPS to 120 FPS input |
| Duration range |
1 second to 60 minutes |
| Maximum file size |
Up to 2 GB per video file |
| URL input |
Submit videos by URL in addition to file upload |
| Glob patterns |
Process multiple files matching a pattern (CLI) |
1.2 Pose Estimation#
| Feature |
Description |
| AI pose detection |
Automatic detection of human body poses from video frames |
| 2D to 3D pose lifting |
Convert 2D screen-space poses into full 3D skeletal data |
| Keypoint normalization |
Standardize detected keypoints across frames for consistency |
| Temporal processing |
Analyze motion across time for smooth, coherent results |
| Multi-person detection |
Detect and track multiple people in a single video |
1.3 Depth Analysis#
| Feature |
Description |
| Monocular depth estimation |
Estimate depth from a single camera view |
| Stereo depth estimation |
Compute depth from stereo camera pairs |
| Depth-enhanced pose |
Use depth information to improve 3D pose accuracy |
1.4 Multi-View Reconstruction#
| Feature |
Description |
| Multi-camera input |
Combine footage from multiple camera angles |
| 3D reconstruction |
Build a complete 3D motion model from multiple views |
| Camera calibration |
Automatic or manual camera parameter configuration |
1.5 Human Mesh Recovery#
| Feature |
Description |
| SMPL mesh recovery |
Reconstruct a full 3D body mesh using the SMPL parametric model |
| SMPL-X mesh recovery |
Extended mesh recovery including hands and face |
| Body shape estimation |
Estimate body proportions and shape parameters from video |
| Mesh-to-skeleton |
Convert recovered mesh data into skeletal animation |
2. Motion Capture Processing#
Import and process data from professional motion capture systems.
| Format |
Description |
| C3D |
Biomechanics standard from optical marker-based systems |
| TRC |
Marker trajectory files from systems like Vicon, OptiTrack |
| ASF/AMC |
Acclaim skeleton and motion files |
| BVH |
Biovision hierarchy (commonly used for MoCap exchange) |
| FBX |
Autodesk FBX with embedded animation data |
2.2 MoCap Data Processing#
| Feature |
Description |
| Marker data import |
Read raw marker positions from optical capture systems |
| Gap filling |
Automatically fill missing marker data in occluded frames |
| Noise reduction |
Remove tracking noise from raw capture data |
| Skeleton fitting |
Fit a standard skeleton to raw marker or keypoint data |
3. Live Motion Capture#
Capture motion in real time from cameras and devices.
3.1 Desktop Live Capture#
| Feature |
Description |
| Webcam capture |
Start a live capture session from any connected webcam |
| Real-time pose estimation |
Process frames at interactive rates for instant motion data |
| Real-time retargeting |
Apply motion to a target avatar as it is captured |
| Streaming pose lift |
Live 2D-to-3D pose lifting during capture |
| Live avatar driving |
Drive a 3D avatar in real time with your body movements |
3.2 Mobile Live Capture#
| Feature |
Description |
| iOS capture |
Capture motion using iPhone or iPad cameras |
| Android capture |
Capture motion using Android device cameras |
| On-device processing |
Run pose estimation directly on the mobile device |
| Mobile streaming |
Stream captured motion data from mobile to the pipeline server |
4. Body, Hand & Face Tracking#
4.1 Full-Body Tracking#
| Feature |
Description |
| Whole-body skeleton |
Extract a full skeleton hierarchy from head to feet |
| Joint positions |
3D position of every tracked joint per frame |
| Joint rotations |
Rotation data for each joint in the hierarchy |
| Bone lengths |
Measured bone lengths for accurate proportions |
4.2 Hand Tracking#
| Feature |
Description |
| Hand keypoint detection |
Detect individual finger and hand joint positions |
| Hand gesture recognition |
Identify common hand gestures in the motion data |
4.3 Face Tracking#
| Feature |
Description |
| Facial blend shape extraction |
Extract blend shape weights for facial animation |
| Blend shape retargeting |
Map extracted blend shapes to a target avatar's face rig |
| Expression analysis |
Analyze facial expressions in the source video |
5. Motion Retargeting#
Transfer motion from one skeleton to another, regardless of body proportions.
5.1 Skeleton-to-Skeleton Retargeting#
| Feature |
Description |
| Skeleton mapping |
Map bones between different skeleton hierarchies (e.g., Mixamo to Unreal) |
| Automatic bone matching |
AI-assisted matching of bones between source and target skeletons |
| Interactive bone mapping UI |
Visual drag-and-drop interface for manual bone mapping |
5.2 Proportional Adaptation#
| Feature |
Description |
| Body proportion scaling |
Adapt motion for different body sizes (e.g., child to adult, short to tall) |
| Limb length adjustment |
Correct motion artifacts caused by differing limb proportions |
| Morphology adaptation |
Handle extreme differences in body shape between source and target |
5.3 Semantic Retargeting#
| Feature |
Description |
| Intent preservation |
Preserve the meaning and style of the motion during retargeting |
| Semantic-aware mapping |
Understand what a motion "means" (e.g., a wave, a walk) and retarget accordingly |
| Style consistency |
Maintain the character's movement style across retargeted clips |
5.4 Neural Retargeting#
| Feature |
Description |
| ML-based retargeting |
Use trained neural networks for higher-quality retargeting |
| Learned motion priors |
Networks trained on large motion datasets understand natural movement |
| Complex morphology handling |
Handle challenging retargeting cases (e.g., human to non-human) |
5.5 Facial Blend Shape Retargeting#
| Feature |
Description |
| Source-to-target blend shapes |
Map blend shapes between different facial rigs |
| Apple ARKit to custom rig |
Retarget Apple face tracking data to custom avatar rigs |
| Expression preservation |
Maintain expression fidelity across different face models |
6. Motion Processing & Cleanup#
Clean, smooth, and refine motion data for production use.
6.1 Noise Reduction#
| Method |
Description |
| FFT filtering |
Frequency-domain noise removal |
| Wavelet denoising |
Multi-scale noise reduction preserving motion detail |
| Adaptive filtering |
Noise reduction that adapts to the motion's characteristics |
| Method |
Description |
| IK-based correction |
Inverse kinematics solvers pin feet to the ground plane |
| TwoBone IK |
Analytical two-bone solver for leg chains |
| FABRIK |
Forward And Backward Reaching Inverse Kinematics for natural correction |
| CCD (Cyclic Coordinate Descent) |
Iterative IK solver for complex chains |
| Ground contact detection |
Automatically detect when feet should be planted |
| Plane alignment |
Align foot contacts to the ground surface |
6.3 Physics Cleanup#
| Feature |
Description |
| Center of mass correction |
Ensure physically plausible balance and weight distribution |
| Angular momentum conservation |
Correct unrealistic angular momentum in acrobatic movements |
| Balance validation |
Verify that the character maintains physical balance |
| Joint torque limits |
Enforce realistic joint force limits |
6.4 Retiming#
| Feature |
Description |
| Time warping |
Speed up or slow down sections of motion |
| Beat synchronization |
Align motion timing to musical beats |
| Speed curve editing |
Custom speed profiles for motion sections |
6.5 Gap Filling#
| Feature |
Description |
| Short gap interpolation |
Fill small gaps (a few frames) with smooth interpolation |
| Long gap prediction |
Predict motion for extended missing sections using context |
| Keyframe insertion |
Insert keyframes to guide gap-filling in specific areas |
6.6 Segmentation#
| Feature |
Description |
| Action segmentation |
Automatically split a motion clip into distinct actions |
| Transition detection |
Identify transition points between movements |
| Loop point detection |
Find frames where a motion can seamlessly loop |
7. Motion Quality Assessment#
Automatically evaluate the quality of motion data.
7.1 Basic Quality Metrics#
| Metric |
Description |
| Jitter detection |
Measure high-frequency noise in joint positions |
| Foot sliding |
Quantify how much feet slide when they should be planted |
| Bone length consistency |
Verify bone lengths remain constant across frames |
| Joint angle validity |
Check that joint angles stay within anatomical limits |
| Smoothness score |
Measure overall motion smoothness |
| Plausibility score |
Assess whether the motion looks physically natural |
7.2 Perceptual Quality Metrics#
| Metric |
Description |
| Naturalness |
How natural the motion appears to a human viewer |
| Style consistency |
Whether the movement style remains consistent throughout |
| Temporal coherence |
Smoothness of motion over time without sudden jumps |
| Visual quality |
Overall visual assessment of the rendered motion |
7.3 Ground Truth Comparison#
| Metric |
Description |
| MPJPE |
Mean Per Joint Position Error (mm) against reference |
| PA-MPJPE |
Procrustes-Aligned MPJPE (scale/rotation invariant) |
| PCK |
Percentage of Correct Keypoints within a threshold |
| AUC |
Area Under the Curve for PCK at varying thresholds |
7.4 Automated QA Pipeline#
| Feature |
Description |
| Pass / Reject / Review decisions |
Automatic quality gating based on configurable thresholds |
| Quality reports |
Detailed per-metric breakdown for every processed file |
| Quality presets |
Draft (0.6 target), Standard (0.8), High (0.9), Ultra (0.95) |
| Configurable thresholds |
Set minimum quality scores per metric |
Import and export motion data in 13+ industry-standard formats.
| Format |
Extension |
Read |
Write |
Description |
| BVH |
.bvh |
Yes |
Yes |
Biovision Hierarchy -- widely used for MoCap exchange |
| FBX |
.fbx |
Yes |
Yes |
Autodesk interchange format with skeletal and mesh data |
| glTF / GLB |
.gltf, .glb |
Yes |
Yes |
Khronos 3D format for web and real-time engines |
| USD / USDA / USDC / USDZ |
.usd, .usda, .usdc, .usdz |
Yes |
Yes |
Pixar Universal Scene Description |
| Alembic |
.abc |
Yes |
Yes |
Open interchange for baked animation and geometry |
| C3D |
.c3d |
Yes |
Yes |
Biomechanics standard for marker data |
| TRC |
.trc |
Yes |
Yes |
Marker trajectory format |
| ASF/AMC |
.asf, .amc |
Yes |
Yes |
Acclaim skeleton and motion |
| Collada |
.dae |
Yes |
Yes |
XML-based 3D interchange |
| MDD |
.mdd |
Yes |
Yes |
Point cache animation |
| PC2 |
.pc2 |
Yes |
Yes |
Point cache format |
| OBJ Sequences |
.obj (numbered) |
Yes |
Yes |
Per-frame OBJ geometry sequences |
| Custom JSON |
.json |
Yes |
Yes |
Aja's native JSON motion representation |
8.2 Conversion Features#
| Feature |
Description |
| Batch conversion |
Convert multiple files between formats in one operation |
| Auto-detect format |
Automatically identify the input format from file contents |
| Compression options |
Apply gzip or other compression to output files |
| Additional output formats |
Export the same motion in multiple formats simultaneously |
| Motion clip database |
Store and retrieve motion clips from a local database |
9. Domain-Specific Pipelines#
Pre-configured processing pipelines optimized for specific movement domains.
9.1 Yoga Pipeline#
| Feature |
Description |
| Asana detection |
Identify yoga poses (asanas) in the motion data |
| Asana recognition |
Classify which specific asana is being performed |
| Alignment analysis |
Evaluate body alignment against ideal form |
| Breath detection |
Detect breathing patterns from body movement |
| Drishti tracking |
Track gaze direction during yoga practice |
| Bandha analysis |
Analyze engagement of energy locks (bandhas) |
| Meditation state detection |
Identify still, meditative poses |
| Pranayama recognition |
Detect specific breathing exercise patterns |
| Sequence tracking |
Follow the progression through a yoga sequence |
9.2 Fitness Pipeline#
| Feature |
Description |
| Exercise recognition |
Identify which exercise is being performed |
| Repetition counting |
Count reps with phase detection (concentric/eccentric) |
| Form analysis |
Evaluate exercise form with real-time feedback |
| Range of motion |
Measure joint range of motion during exercises |
| Workout tracking |
Track sets, reps, and rest periods across a workout |
| Equipment detection |
Detect use of weights, bands, or machines |
9.3 Dance Pipeline#
| Feature |
Description |
| Style detection |
Identify dance style (ballet, hip-hop, salsa, contemporary, etc.) |
| Beat detection |
Detect musical beats from associated audio or motion rhythm |
| Beat-movement synchronization |
Analyze how well movements align with musical beats |
| Choreography segmentation |
Break a performance into distinct choreographic phrases |
| Expression analysis |
Evaluate expressiveness and performance quality |
| Musicality scoring |
Score the dancer's musicality and rhythmic accuracy |
9.4 Martial Arts Pipeline#
| Feature |
Description |
| Style detection |
Identify martial art style (boxing, karate, BJJ, muay thai, etc.) |
| Strike detection |
Detect and classify individual strikes (punch, kick, elbow, knee) |
| Power estimation |
Estimate strike power from motion dynamics |
| Speed estimation |
Measure strike and movement speed |
| Stance analysis |
Evaluate fighting stance and weight distribution |
| Guard position analysis |
Assess defensive hand and body positioning |
| Footwork patterns |
Analyze movement patterns and foot positioning |
| Combat sequence tracking |
Follow the progression of attack and defense combinations |
9.5 Sports Pipeline#
| Feature |
Description |
| Performance analysis |
Break down athletic performance metrics |
| Technique breakdown |
Analyze specific techniques (swing, throw, stride) |
| Comparison to reference |
Compare against reference performances |
9.6 Medical / Rehabilitation Pipeline#
| Feature |
Description |
| Gait analysis |
Clinical-grade walking gait assessment |
| Rehabilitation tracking |
Monitor rehabilitation exercise progress over time |
| Clinical assessment |
Generate reports suitable for clinical review |
| Range of motion measurement |
Precise joint ROM measurements |
9.7 Film / VFX Pipeline#
| Feature |
Description |
| High-fidelity capture |
Maximum quality settings for film production |
| Facial performance capture |
Detailed facial animation from video |
| Multi-camera support |
Process synchronized multi-camera footage |
9.8 Gaming Pipeline#
| Feature |
Description |
| Real-time capture |
Low-latency processing for interactive applications |
| Gesture recognition |
Detect hand and body gestures for game input |
| Loop-ready output |
Automatically find and prepare looping animation clips |
10. Reference Video Management#
Organize, search, and collaborate on reference videos.
10.1 Video Library#
| Feature |
Description |
| Video ingestion |
Upload and import reference videos into the library |
| Metadata extraction |
Automatically extract duration, resolution, frame rate, codec information |
| Scene detection |
Detect and segment individual scenes within a video |
| Categorization |
Organize videos by domain, style, movement type, and custom tags |
10.2 Search & Discovery#
| Feature |
Description |
| CLIP-based vector search |
Semantic search using CLIP embeddings for visual similarity |
| Text search |
Search by title, description, and tags |
| Filter by metadata |
Filter by duration, resolution, domain, and custom attributes |
10.3 Annotation & Collaboration#
| Feature |
Description |
| Frame annotations |
Add notes and markers to specific frames |
| Temporal annotations |
Mark and label time ranges within videos |
| Collaborative review |
Multiple users can annotate and comment on the same video |
11. AI Motion Features#
11.1 Motion Enhancement#
| Feature |
Description |
| Motion super-resolution |
Increase the frame rate and detail of motion data using AI |
| Style transfer |
Apply the movement style of one clip to another |
| Physics refinement |
Use physics-aware AI to make motion more physically plausible |
| Motion synthesis |
Generate new motion sequences from text descriptions or constraints |
11.2 Video Analysis#
| Feature |
Description |
| Video content analysis |
Analyze video content to identify movement types, scenes, and subjects |
| Action recognition |
Classify actions and activities in video footage |
11.3 Animation Blending#
| Feature |
Description |
| Motion blending |
Smoothly blend between two motion clips |
| Transition generation |
Create natural transitions between different movements |
| Interpolation |
Generate in-between frames for smooth animation |
| Blend weights |
Control the mix ratio between source motions |
11.4 Inverse Kinematics#
| Feature |
Description |
| Analytical IK solvers |
Fast, closed-form solutions for common limb chains |
| Iterative IK solvers |
General-purpose solvers (FABRIK, CCD) for complex chains |
| Constraint handling |
IK solutions respect joint limits and reach constraints |
| End-effector targeting |
Position hands, feet, or head at specific target locations |
12. Skeleton & Avatar Management#
12.1 Avatar Library#
| Feature |
Description |
| Avatar presets |
Pre-configured avatars for common use cases |
| Avatar definitions |
Standardized avatar descriptions with skeleton hierarchy |
| Custom avatar import |
Import your own avatar models for retargeting |
12.2 Avatar Preview#
| Feature |
Description |
| React preview components |
Embed avatar previews in web applications |
| Motion playback preview |
Preview processed motion on an avatar before export |
| Interactive inspection |
Rotate, zoom, and scrub through animated previews |
12.3 Bone Mapping UI#
| Feature |
Description |
| Interactive bone mapping |
Visual interface to map bones between source and target skeletons |
| Drag-and-drop assignment |
Map bones by dragging connections in the UI |
| Mapping templates |
Save and reuse bone mappings for common skeleton pairs |
| Auto-mapping suggestions |
AI-suggested bone matches based on hierarchy analysis |
12.4 Avatar Integration#
| Feature |
Description |
| Avatar binding |
Bind processed motion data to avatar rigs |
| Avatar animation |
Drive avatar animation from motion data |
| Cross-platform avatars |
Support avatars from Unity, Unreal, Mixamo, and other platforms |
13. Batch Processing & Scale#
13.1 Batch Operations#
| Feature |
Description |
| Batch job submission |
Submit multiple videos or files for processing in a single request |
| Batch format conversion |
Convert many files between formats in one operation |
| Batch cancel |
Cancel multiple jobs at once |
| Batch retry |
Retry multiple failed jobs at once |
| Batch priority update |
Change priority for multiple jobs simultaneously |
13.2 Distributed Processing#
| Feature |
Description |
| Video chunking |
Split long videos into segments for parallel processing |
| Pipeline parallelism |
Execute pipeline stages in parallel where possible |
| Distributed workers |
Coordinate processing across multiple worker instances |
| Result aggregation |
Combine results from distributed workers into a single output |
| Worker coordination |
Manage worker health, assignment, and load balancing |
| Feature |
Description |
| Model optimization |
Compress and optimize ML models for faster inference |
| Pipeline caching |
LRU cache for pipeline stage results to avoid recomputation |
| Concurrent job limits |
Configurable maximum concurrent job count (default: 10) |
14. Pipeline Configuration#
14.1 Pipeline Stages#
The processing pipeline consists of up to 11 configurable stages:
| Stage |
Description |
| Ingestion |
Accept and validate input files |
| Validation |
Verify data integrity and format correctness |
| Preprocessing |
Prepare data for processing (normalization, resampling) |
| Pose estimation |
AI-based human pose detection from video |
| Skeleton fitting |
Fit a skeleton model to estimated poses |
| Domain analysis |
Run domain-specific analysis (yoga, fitness, dance, etc.) |
| Quality assessment |
Compute quality metrics and make pass/reject decisions |
| Retargeting |
Map motion to a target skeleton |
| Format conversion |
Convert to requested output format(s) |
| Postprocessing |
Final cleanup and optimization |
| Delivery |
Store results and send notifications |
14.2 Per-Stage Configuration#
| Option |
Description |
| Enable / disable |
Toggle any stage on or off |
| Timeout |
Maximum time allowed for the stage |
| Retry policy |
Exponential, linear, or fixed backoff with configurable attempts |
| Dependencies |
Specify which stages must complete before this one runs |
| Parallel execution |
Allow a stage to run in parallel with others |
14.3 Quality Presets#
| Preset |
Processing Time |
Target Quality |
Frame Rate |
Features |
| Draft |
0.5x baseline |
0.60 |
24 FPS |
Basic noise reduction only |
| Standard |
1.0x baseline |
0.80 |
30 FPS |
Noise reduction + smoothing + jitter correction |
| High |
2.0x baseline |
0.90 |
60 FPS |
All standard + foot sliding + contact detection + physics |
| Ultra |
4.0x baseline |
0.95 |
120 FPS |
All high + IK cleanup + multi-view fusion |
| Feature |
Description |
| JSON configuration |
Define pipeline configs in JSON |
| YAML configuration |
Define pipeline configs in YAML (Kubernetes-style manifests) |
| Named configurations |
Save and retrieve configs by name for reuse |
| Domain-labeled configs |
Tag configurations by domain (yoga, fitness, dance, etc.) |
| Config export |
Export configurations as YAML for version control |
15. Job Management#
15.1 Job Lifecycle#
| State |
Description |
| Queued |
Job is waiting in the priority queue |
| Processing |
Job is actively being processed through pipeline stages |
| Completed |
Job finished successfully with output files |
| Failed |
Job encountered an error |
| Cancelled |
Job was cancelled by the user |
15.2 Job Operations#
| Feature |
Description |
| Submit job |
Start a new processing job (returns immediately with job ID) |
| Check status |
Poll the job status at any time |
| Cancel job |
Stop a running or queued job |
| Retry job |
Retry a failed job |
| Update priority |
Change a job's priority in the queue |
15.3 Job Filtering & Search#
| Feature |
Description |
| Filter by status |
Show only queued, processing, completed, or failed jobs |
| Filter by domain |
Show jobs for a specific domain (yoga, fitness, etc.) |
| Filter by tags |
Search jobs by custom tags |
| Pagination |
Page through large job lists |
15.4 Queue Statistics#
| Metric |
Description |
| Waiting count |
Jobs waiting in the queue |
| Active count |
Jobs currently being processed |
| Completed count |
Total completed jobs |
| Failed count |
Total failed jobs |
16. Result Delivery#
16.1 Output Retrieval#
| Method |
Description |
| Polling |
Check job status via GET endpoint until complete |
| SSE streaming |
Subscribe to real-time Server-Sent Events for stage-by-stage progress |
| Webhooks |
Receive an HTTP POST notification when a job completes or fails |
| Signed download URLs |
Generate time-limited signed URLs for secure result download |
16.2 Output Contents#
| Feature |
Description |
| Motion files |
Processed animation in the requested format(s) |
| Additional formats |
Optionally export in multiple formats simultaneously |
| Analytics data |
Domain-specific analysis results (if enabled) |
| Quality report |
Per-metric quality assessment of the output |
| Compression |
Results can be gzip-compressed for faster download |
16.3 Webhook Management#
| Feature |
Description |
| Webhook delivery history |
View all webhook deliveries for a job |
| Delivery status |
Track whether webhooks were successfully received |
| Retry failed webhooks |
Manually retry a failed webhook delivery |
| Webhook statistics |
Service-wide webhook delivery metrics |
| HMAC signing |
Webhook payloads are signed for security verification |
A command-line interface for interacting with the Aja platform from a terminal.
17.1 Commands#
| Command |
Description |
aja process [input...] |
Submit videos for motion extraction; supports file paths, globs, URLs, and interactive mode |
aja convert <input> <output> |
Convert between motion formats (batch support) |
aja inspect <file> |
Inspect and validate a motion file; display quality metrics |
aja debug |
Debug pipeline issues and troubleshoot errors |
aja config |
Manage pipeline configurations (list, create, edit, delete) |
aja jobs |
View and manage processing jobs (list, status, cancel, retry) |
aja health |
Check API service health and connectivity |
17.2 CLI Features#
| Feature |
Description |
| Interactive mode |
Interactive prompts for guided job submission |
| Progress spinners |
Visual progress feedback during operations |
| Color output |
Syntax-highlighted, color-coded terminal output |
| Pipeline presets |
Specify quality preset on the command line (draft, standard, high, ultra) |
| Domain selection |
Select the processing domain (yoga, fitness, dance, etc.) |
| API key auth |
Authenticate with the pipeline service via API key |
| Configurable API URL |
Point the CLI at any pipeline service instance |
18.1 TypeScript SDK#
| Feature |
Description |
| Full API client |
Typed client for all pipeline service endpoints |
| Job submission |
Programmatically submit and manage processing jobs |
| Config management |
Create and manage pipeline configurations |
| Result retrieval |
Download processed results |
| SSE streaming |
Subscribe to job progress events |
| Type-safe |
Full TypeScript type definitions for all API types |
18.2 Python SDK#
| Feature |
Description |
| Python API client |
Full client for pipeline service interaction |
| Jupyter notebook support |
Designed for use in Jupyter/Colab notebooks |
| ML workflow integration |
Integrate motion processing into Python ML pipelines |
| Data science friendly |
Return types compatible with NumPy, pandas, and common data science tools |
18.3 API Endpoints#
| Category |
Key Endpoints |
Description |
| Health |
GET /health |
Service health and uptime |
| Pipeline stats |
GET /v1/pipeline/stats |
Queue statistics |
| Configs |
GET/POST/DELETE /v1/pipeline/configs |
Pipeline configuration CRUD |
| Config presets |
GET /v1/pipeline/presets |
List quality presets |
| Jobs |
POST /v1/pipeline/jobs |
Submit processing job |
| Job status |
GET /v1/pipeline/jobs/:jobId |
Poll job status |
| Job control |
POST .../cancel, POST .../retry, PATCH .../priority |
Manage running jobs |
| Batch ops |
POST /v1/pipeline/jobs/batch |
Batch cancel, retry, prioritize |
| Outputs |
GET /v1/pipeline/jobs/:jobId/outputs |
List job outputs |
| Download |
POST /v1/pipeline/jobs/:jobId/download |
Get signed download URL |
| SSE stream |
GET /v1/pipeline/jobs/:jobId/stream |
Real-time progress events |
| Webhooks |
GET .../webhooks, POST .../retry |
Webhook delivery management |
| Video-to-motion |
POST /api/v1/video-to-motion/process |
Direct video processing |
| Retargeting |
POST /api/v1/retargeting/retarget |
Direct retargeting |
| Skeletal animation |
POST /api/v1/skeletal/animate |
Skeletal animation generation |
| Video analysis |
POST /api/v1/analysis/analyze |
Video content analysis |
19. Privacy & Security#
19.1 Privacy Protection#
| Feature |
Description |
| Face blurring |
Automatically blur faces in processed video to protect identity |
| Body anonymization |
Remove identifying visual features from body data |
| Privacy-preserving processing |
Process motion data without retaining personally identifiable information |
| Configurable privacy levels |
Choose between different levels of privacy protection |
19.2 Consent Management#
| Feature |
Description |
| Consent tracking |
Record and verify consent for each person in the motion data |
| Consent enforcement |
Block processing of data without proper consent records |
| Consent audit trail |
Maintain a complete history of consent grants and revocations |
19.3 Content Security#
| Feature |
Description |
| Digital watermarking |
Embed invisible watermarks in output motion files for provenance tracking |
| Content moderation |
Filter inappropriate or harmful content before processing |
| Security protocols |
Encrypted data in transit and at rest |
| Signed webhook payloads |
HMAC-signed webhook deliveries for verification |
19.4 Data Retention#
| Feature |
Description |
| Retention policies |
Define how long input and output data is stored |
| Automatic cleanup |
Expired data is automatically deleted per retention policy |
| Policy enforcement |
System enforces retention rules and prevents access to expired data |
| Audit logging |
Complete audit trail of data creation, access, and deletion |
Feature Summary#
| Category |
Feature Count |
Highlights |
| Video-to-Motion |
15+ |
AI pose estimation, 2D-to-3D lifting, depth sensing, SMPL-X mesh recovery |
| MoCap Processing |
5+ |
C3D, TRC, ASF/AMC import; marker gap filling; skeleton fitting |
| Live Capture |
5+ |
Desktop webcam, iOS, Android; real-time retargeting; live avatar driving |
| Body/Hand/Face |
10+ |
Full-body skeleton, hand keypoints, facial blend shapes |
| Retargeting |
15+ |
Skeleton mapping, proportional adaptation, neural retargeting, blend shapes |
| Motion Cleanup |
15+ |
Noise reduction, foot sliding, physics cleanup, retiming, gap filling |
| Quality Assessment |
15+ |
Jitter, smoothness, plausibility, MPJPE, automated QA with pass/reject |
| Format Conversion |
15+ |
13+ formats (BVH, FBX, GLB, USD, Alembic, C3D, TRC, etc.), batch conversion |
| Domain Pipelines |
40+ |
Yoga, Fitness, Dance, Martial Arts, Sports, Medical, Film/VFX, Gaming |
| Reference Video |
5+ |
Library, CLIP search, annotation, collaboration |
| AI Motion |
10+ |
Super-resolution, style transfer, motion synthesis, animation blending, IK |
| Skeleton & Avatar |
10+ |
Avatar library, preview UI, bone mapping UI, cross-platform support |
| Batch & Scale |
10+ |
Batch submission, distributed workers, video chunking, result aggregation |
| Pipeline Config |
10+ |
11 stages, per-stage config, 4 quality presets, YAML/JSON support |
| Job Management |
10+ |
Queue, status, cancel, retry, priority, filtering |
| Result Delivery |
10+ |
Polling, SSE streaming, webhooks, signed downloads |
| CLI |
10+ |
process, convert, inspect, debug, config, jobs, health |
| SDKs |
5+ |
TypeScript SDK, Python SDK with Jupyter support |
| Privacy & Security |
10+ |
Face blurring, consent management, watermarking, data retention |