Eve library

Oshun Ecosystem vs. Industry: Comprehensive Gap Analysis Report

Guide

Revision 1

On this page

Date: 2026-03-16 Scope: All AI filmmaking companies, major VFX studio proprietary tools, streaming platform technology, and the complete Oshun ecosystem Purpose: Identify gaps, feature parity opportunities, and SOTA targets across every dimension of creative production technology


Table of Contents#

  1. Executive Summary
  2. Oshun Ecosystem Overview
  3. Competitor Deep Dives
    • 3.1 AI Filmmaking Companies
    • 3.2 Major VFX Studio Proprietary Tools
    • 3.3 Streaming Platform Technology
    • 3.4 DCC Tools & Engines
  4. Master Gap Analysis Matrix
  5. Gap Analysis by Category
    • 5.1 Rendering & Real-Time Graphics
    • 5.2 AI Video Generation
    • 5.3 Post-Production AI
    • 5.4 Digital Humans & Face Technology
    • 5.5 Motion Capture & Animation
    • 5.6 Simulation & Physics
    • 5.7 Hair, Fur & Cloth
    • 5.8 Crowd Simulation
    • 5.9 Virtual Production
    • 5.10 Audio & Music AI
    • 5.11 Voice AI & Dubbing
    • 5.12 Asset Management & Pipeline
    • 5.13 Color Science & HDR
    • 5.14 Encoding & Streaming
    • 5.15 Content Security & DRM
    • 5.16 Collaboration & Review
    • 5.17 Production Management
    • 5.18 Quality Control & Validation
    • 5.19 Worldbuilding & Narrative AI
    • 5.20 Autonomous Production
  6. In-House Build Recommendations
  7. Phase 26 Completion Priority
  8. Strategic Roadmap

1. Executive Summary#

What Oshun Is#

Oshun is an interconnected ecosystem of 35 domains, 380+ libraries, and 89+ applications spanning creative production, AI generation, game engines, worldbuilding, knowledge management, motion intelligence, and autonomous production orchestration. No single company in the world has attempted anything this comprehensive.

The Competitive Landscape (120+ Tools Analyzed)#

Category Companies/Tools Analyzed
AI Video Generation OpenAI Sora 2, Google Veo 3.1, Runway Gen-4.5, Pika 2.5, Kling 2.0, Luma Ray3.14, Haiper, MiniMax Hailuo, Higgsfield, LTX Studio, Stability AI SVD
AI Post-Production InterPositive (Netflix), Topaz Labs, Flawless AI (TrueSync/DeepEditor)
AI VFX & Digital Humans Wonder Dynamics (Autodesk), Metaphysic/Brahma (DNEG), MARZ
AI Voice & Audio ElevenLabs, Respeecher, Suno, Udio, AIVA, Boomy, Google Lyria 3
AI Animation & Motion Cascadeur, DeepMotion, Move AI, Rokoko, Runway Act-One
VFX Studios (In-House Tools) ILM (20+ tools), Pixar (15+ tools), Weta FX (15+ tools), DNEG, Framestore, MPC, Sony Imageworks, DreamWorks
Disney/WDAS Hyperion, Meander, Denizen, Bonsai, Tonic, XGen, Matterhorn, Ptex
Netflix InterPositive, MPS, Content Hub, Eyeline (Scanline), VMAF, Dynamic Optimizer, Prodicle
DCC & Engines Unreal Engine 5.7, Unity/Weta Tools, Houdini 21, Maya 2026, Nuke 17, DaVinci Resolve 20, Adobe Creative Cloud
Platform Tech NVIDIA Omniverse/ACE/DLSS, Adobe Firefly, Autodesk Flame, Google SynthID
Virtual Production ILM StageCraft/Helios, Cuebric, DNEG 360, Framestore Farsight, MPC Genesis
AI Avatars & Interaction Synthesia, HeyGen, Reallusion, NVIDIA ACE
AI Editing Descript, Adobe Premiere AI

Bottom Line#

Oshun is architecturally unmatched — no competitor has an integrated ecosystem spanning autonomous production, worldbuilding, generative AI, game engines, knowledge management, motion intelligence, and cross-media franchise orchestration. However, Oshun has critical gaps in specific production technologies that ILM, Pixar, Weta, and Netflix have spent decades perfecting. This report identifies every gap and proposes in-house solutions.


2. Oshun Ecosystem Overview#

Domain Map#

Domain Purpose Libraries Status
Yemaya Creative Production Studio 35+ Production
Isis Generative AI Factory 13 Production
Hathor Worldbuilding & Narrative 13 Production
Sophia Knowledge & Research 10+ Production
Bellona Engine & Build Bridge 24 Production
Lilith Consciousness Platform 21 libs, 89 apps Production
Psyche Avatar & Screen Analysis 138+ High Activity
Aja Motion AI & Animation 38 Production
Metis Educational Platform 17 In Development
Veritas AI News Agency 30+ Production
Euterpe Music & Audio 8+ Library-Only
Uzume Live Entertainment 26 Library-Only
Calliope Storytelling & Media 20+ Library-Only
Maya Infinite Virtual Universe 40+ planned Early
Iris Universal AI Assistant 32 Planning
Nous LLM Inference Engine 1 (56 modules) Production
Kuanyin Ethics & Moderation 10 Production
@oshun/* (shared) Foundation Infrastructure 25+ Production
+ 17 more domains Various Various Various

Key Existing Capabilities (What Oshun Can Already Do)#

Production Pipeline:

  • Project management with 6 project types (Film, Game, Animation, Commercial, Music Video, Documentary)
  • 15 asset types with versioning, metadata extraction, SHA-256 checksums
  • Real-time CRDT collaboration (Yjs) with presence, cursors, voice/video
  • Engine bridges to Unreal, Unity, Godot, Blender, Houdini, DaVinci Resolve
  • Build system with Redis caching, asset interchange (GLTF/USD/FBX)
  • OpenUSD pipeline support

AI Generation:

  • Multi-provider video generation (Hunyuan via FAL, WAN via AIML)
  • 5 generation modes: Text-to-Video, Image-to-Video, Video-to-Video, Audio-to-Video, Reference-to-Video
  • 3D asset generation, Gaussian splatting
  • GPU worker management with provider routing
  • ComfyUI workflow orchestration

Motion & Animation:

  • 2D-to-3D pose lifting
  • Skeletal retargeting
  • Motion quality assessment
  • Multi-format support (BVH, FBX, GLB, USD, Alembic)
  • Human mesh recovery, IK solvers

Audio:

  • TTS integration (multiple providers)
  • AV sync with drift correction
  • Podcast generation
  • Music technology foundations (Euterpe)

Worldbuilding & Narrative:

  • World simulation (economy, politics, culture)
  • Story graphs, quest systems, dialogue
  • LLM-powered NPCs (NVIDIA ACE, Inworld AI, Convai integration)
  • Lore validation and compilation to game engines

Knowledge & Research:

  • Hybrid vector+keyword search
  • 8 chunking strategies, 4 vector DB backends
  • Knowledge graphs, citation management
  • RAG pipeline

Autonomous Production (Phase 26):

  • Goal decomposition engine (LLM-driven)
  • Multi-agent orchestration (LangGraph)
  • Autonomous creative direction
  • AAA quality benchmarking (LPIPS, FID, narrative, gameplay)
  • Cross-domain event orchestration
  • Failure recovery & dynamic replanning

3. Competitor Deep Dives#

3.1 AI Filmmaking Companies#

InterPositive (Netflix) — $600M Acquisition#

  • Core Tech: Per-production AI model trained on dailies
  • Capabilities: Relighting, color grading, shot reframing, BG replacement, VFX, missing coverage synthesis
  • Patent: US 12,511,904 B1 — Captioner model for cinematic elements (focal length, camera movement, depth of field, camera angle, occlusion, parallax)
  • Differentiator: Models trained on production's OWN footage, not internet data. Filmmaker vocabulary interface.
  • Team: 16 engineers/researchers/creatives, now at Netflix
  • Oshun Gap: CRITICAL — No equivalent capability exists in Oshun

Runway Gen-4.5#

  • Core Tech: Text/image-to-video generation, creative editing tools
  • Capabilities: Gen-4.5 video, Act-One facial performance capture, motion brushes, multi-modal editing
  • Pricing: From $12/mo
  • Differentiator: Most complete creative AI video platform with deep editing tools
  • Oshun Coverage: Partial via Isis (multi-provider video generation)

OpenAI Sora 2#

  • Core Tech: Diffusion-based text-to-video with cinematic realism
  • Capabilities: High-realism video generation, Disney character licensing (200+)
  • Differentiator: Highest realism for photorealistic scenes
  • Oshun Coverage: Accessible via provider routing in Isis

Google Veo 3.1#

  • Core Tech: Text-to-video, 1080p, 8-second extendable clips
  • Differentiator: Among strongest for detailed, realistic visuals
  • Oshun Coverage: Accessible via provider routing in Isis

Pika 2.5#

  • Core Tech: Text-to-video with strong cinematic aesthetics
  • Differentiator: Control + quality balance, neon/dramatic mood
  • Oshun Coverage: Could be added as Isis provider

Kling 2.0#

  • Core Tech: Continuous 2-minute video clips (longest in class)
  • Differentiator: Duration leadership
  • Oshun Coverage: Could be added as Isis provider

Luma AI Ray3.14#

  • Core Tech: Native 1080p, HDR pipeline (industry first), 16-bit EXR export, NeRF/Gaussian Splatting
  • Differentiator: HDR video generation, bridges 3D scanning with generative video, Unreal plugin
  • Oshun Coverage: Partial — Isis has video gen, Bellona has 3D pipeline, but no HDR video gen or NeRF integration

Wonder Dynamics (Autodesk)#

  • Core Tech: Markerless AI mocap, CG character replacement from regular video
  • Capabilities: Track actor performance, apply to CG character, automatic lighting estimation
  • Differentiator: No markers, no suits, works from regular video
  • Oshun Coverage: Partial via Aja (pose estimation, retargeting) but NOT markerless CG character replacement

Metaphysic/Brahma (DNEG)#

  • Core Tech: Photorealistic face synthesis, de-aging
  • Used On: Robert Zemeckis' Here (Tom Hanks de-aged 18-80)
  • Differentiator: Film-proven face synthesis at scale
  • Oshun Coverage: NONE — No face synthesis/de-aging capability

Cuebric#

  • Core Tech: AI-generated 2.5D backgrounds for LED wall virtual production
  • Differentiator: Real-time modification during live shooting
  • Oshun Coverage: Partial via Bellona virtual production, but no real-time AI background generation

LTX Studio#

  • Core Tech: Full script-to-screen pipeline
  • Differentiator: Writing, storyboarding, shot planning, visualization in one platform
  • Oshun Coverage: Yemaya + Hathor cover this more comprehensively

Topaz Labs#

  • Core Tech: AI upscaling to 8K, frame interpolation, denoising, restoration
  • Differentiator: Best-in-class for enhancement/restoration
  • Oshun Coverage: NONE — No AI upscaling/restoration capability

Flawless AI#

  • Core Tech: TrueSync (visual dubbing — lip movements match dubbed dialogue), DeepEditor (change dialogue in post)
  • Differentiator: First AI-dubbed theatrical release (UFO Sweden, 2025)
  • Oshun Coverage: NONE — No visual dubbing/lip-sync technology

Higgsfield Cinema Studio#

  • Core Tech: Director Panel (camera/lens simulation), Grid Mode (16 variations), multi-model integration
  • Differentiator: 20M+ users, real optical physics, model-agnostic
  • Oshun Coverage: Partial — Yemaya video gen has camera presets but not optical physics simulation

MARZ#

  • Core Tech: Vanity AI (automated de-aging), LipDub AI (automated lip-sync)
  • Used On: Wednesday, WandaVision, Moon Knight, Spider-Man: No Way Home (110+ TV productions)
  • Oshun Coverage: NONE for de-aging/lip-sync

3.2 Major VFX Studio Proprietary Tools#

ILM (Industrial Light & Magic)#

Tool Domain Oshun Equivalent Gap?
Zeno Scene graph, content creation pipeline Yemaya project/asset management PARTIAL — Zeno is a full 3D DCC framework
Helios Real-time cinema renderer (Vulkan RT) None CRITICAL — No real-time cinema renderer
Plume GPU fluid simulation (CUDA) None YES — No physics simulation
StageCraft/The Volume LED virtual production Bellona virtual-production lib PARTIAL — No physical LED wall control
BlockParty Rigging system None YES — No rigging tools
CompTime/SABRE Compositing None YES — No compositor
IMoCap Image-based motion capture Aja (pose estimation) PARTIAL — No marker-band system
FLUX Markerless facial capture None CRITICAL — No facial capture
MEDUSA/ANYMA 4D facial scanning None CRITICAL — No face scanning
PhysBAM Physics simulation None YES — No destruction/rigid body sim
Lama Layered material system None YES — No material/shading system
FaceSwap Digital de-aging (ML + CG) None CRITICAL — No de-aging
Zviz Interactive previsualization Hathor pre-production PARTIAL
MaterialX Material interchange Bellona asset interchange PARTIAL
OpenEXR HDR image format Rendering pipelines (output) PARTIAL — Consume only, don't author

Pixar#

Tool Domain Oshun Equivalent Gap?
Presto Animation system None CRITICAL — No animation system
RenderMan XPU Hybrid CPU/GPU renderer None CRITICAL — No renderer
Luna Interactive lighting None YES
USD/Hydra Scene description/rendering framework Bellona OpenUSD pipeline PARTIAL — Pipeline support, not authoring
FizT Cloth/hair simulation None YES — No cloth/hair sim
OpenSubdiv Subdivision surfaces None YES
Tractor Render farm management None YES — No render farm mgmt
Pitch Doctor Storyboarding Hathor pre-production PARTIAL
OpenTimelineIO Editorial timeline None YES — No editorial timeline
ML Denoiser Neural denoising None YES
CurveNet Spline-based rigging None YES

Disney/WDAS#

Tool Domain Oshun Equivalent Gap?
Hyperion Path-tracing renderer None CRITICAL
Meander 2D/3D hybrid animation None YES
Denizen Crowd simulation None YES
Bonsai Procedural vegetation None YES
Tonic Hair grooming None YES
XGen Hair/fur/primitive generation None YES
Matterhorn MPM physics (snow/mud) None YES
Ptex Per-face texturing None YES
Disney BRDF Material model None YES
Disney Research AI Neural rendering, diffusion, face synthesis, robotics Partial (Isis, Psyche) SIGNIFICANT GAPS

Weta FX#

Tool Domain Oshun Equivalent Gap?
Manuka Spectral path tracer None CRITICAL
Gazebo Interactive GPU renderer None CRITICAL
Lumberjack/Totara Vegetation ecosystem None YES
Barbershop Hair/fur grooming (Oscar-winning) None YES
Wig Hair dynamics None YES
Tissue Muscle/skin simulation (Oscar-winning) None YES
FACETS/APFS Facial capture/performance None CRITICAL
Deep Shapes ML facial deformation None YES
Bodyopt Neural body deformation None YES
Loki Unified multiphysics None YES
Eddy GPU volumetric fluids (Nuke plugin) None YES
PhysLight/PhysCam Physical lighting units None YES
MASSIVE Crowd simulation None YES

Other VFX Houses#

Studio Key Proprietary Tools Oshun Equivalent
DNEG Helix pipeline, Ziva VFX (muscles), Brahma AI, CLEAR AI None of these
Framestore Freak (renderer), Fibre (hair dynamics), FIRA (ML rig deformation), Farsight (VP) None of these
MPC ALICE (crowd), Furtility (fur), Tickle (lighting), Genesis (VP) None of these
Sony Imageworks OpenColorIO, Katana (originated here), Arnold (co-dev), OSL, Alembic, OpenCue, Flix, Kismet Partial via open-source adoption
DreamWorks Premo (animation), MoonRay (open-source renderer), Torch (lighting) None

3.3 Streaming Platform Technology (Netflix)#

Technology Purpose Oshun Equivalent Gap?
VMAF Perceptual video quality metric None YES
Dynamic Optimizer Per-shot encoding optimization None YES
AV1 + Film Grain Synthesis Next-gen codec with 66% bitrate reduction None YES
AVIF Next-gen image format None YES
HDR Pipeline (Dolby Vision, HDR10+) Dynamic HDR mastering None CRITICAL for distribution
Prodicle Suite On-set production tools Yemaya project mgmt PARTIAL
MPS (Media Production Suite) Dailies, VFX pulls, conform, color None YES
Content Hub/CDrive Cloud asset storage (petabytes) Yemaya asset management PARTIAL — not at Netflix scale
Photon IMF validation None YES
Eyeline Studios VFX + virtual production + AI labs Bellona + Isis PARTIAL
InterPositive AI post-production from dailies None CRITICAL
Open Connect Purpose-built CDN (8000+ servers) None YES — Not building CDN
Cosmos Encoding infrastructure None YES
Forensic Watermarking Per-session content ID None YES
DeepSpeak AI dubbing None YES

3.4 DCC Tools & Engines#

Tool Key AI Features (2026) Oshun Integration
Unreal Engine 5.7 MetaHuman, Nanite, Lumen, Motion Matching 2.0, AI Assistant Bellona bridge (9001 port)
Unity Weta Tools, Ziva, AI Assistant, MCP Bellona bridge (planned)
Houdini 21 ML Deformer, ML Volume Upresser, GenAI integration, ComfyUI Isis ComfyUI integration
Maya 2026 MotionMaker (GenAI animation), ML Deformer, Arnold Inference Bellona bridge (concept)
Nuke 17 CopyCat/BigCat ML, Gaussian Splatting, USD Not integrated
DaVinci Resolve 20 AI IntelliScript, Neural Engine (15+ features), $295 lifetime Bellona DaVinci adapter
Adobe Premiere/AE 26 Firefly, Media Intelligence, Runway partnership, Generative Extend Not integrated
NVIDIA Omniverse OpenUSD collaboration, ACE digital humans, DLSS 4.5 Partial via NVIDIA ACE in Hathor NPC
Autodesk Flame 2026 AI Upscaling, AutoMatte, ML Slow-Mo, ONNX models Not integrated

4. Master Gap Analysis Matrix#

Legend#

  • ✅ = Oshun has equivalent or better capability
  • 🟡 = Partial coverage (exists but not SOTA)
  • ❌ = No capability (gap)
  • 🔴 = Critical gap (must address for SOTA status)
# Capability Oshun Status SOTA Reference Priority
1 Autonomous production orchestration ✅ (56% Phase 26) No competitor P0 — Finish
2 Cross-domain event orchestration ✅ No competitor —
3 Multi-agent creative AI ✅ No competitor —
4 Worldbuilding & narrative AI ✅ No competitor —
5 Knowledge management & RAG ✅ No competitor at this depth —
6 Cross-media franchise management ✅ (Phase 26.5 pending) No competitor P0 — Finish
7 AI video generation (text/image-to-video) 🟡 (Hunyuan/WAN) Sora 2, Veo 3.1, Runway Gen-4.5 P1 — Add more providers
8 Per-production AI post (relighting, reframing) 🔴 InterPositive P0 — Build
9 AI upscaling / restoration / enhancement 🔴 Topaz Labs P0 — Build
10 AI face synthesis / de-aging 🔴 Metaphysic, ILM FaceSwap P1 — Build
11 Visual dubbing / lip-sync for translation 🔴 Flawless TrueSync, MARZ LipDub P1 — Build
12 Markerless AI mocap → CG character 🔴 Wonder Dynamics P1 — Extend Aja
13 AI voice cloning & synthesis 🟡 (TTS integration) ElevenLabs, Respeecher P1 — Deepen
14 AI music generation 🟡 (Euterpe foundations) Suno v5, Udio, AIVA P1 — Build
15 AI-assisted animation 🟡 (Aja motion) Cascadeur, Maya MotionMaker P2
16 Real-time rendering / cinema engine 🔴 → 📋 Neith 42.1 (150 tasks) ILM Helios, Pixar RenderMan XPU P2 — Neith Phase 42
17 Physics simulation (fluids, destruction, cloth) 🔴 → 📋 Neith 42.2 (120 tasks) PhysBAM, Loki, Houdini, Matterhorn P2 — Neith Phase 42
18 Hair/fur grooming & simulation 🔴 → 📋 Neith 44.1/42.2 Weta Barbershop/Wig, Pixar FizT, Tonic P2-P3 — Neith Phase 44
19 Facial performance capture 🔴 → 📋 Neith 44.2.4 (15 tasks) ILM FLUX/MEDUSA, Weta APFS P2 — Neith Phase 44
20 Crowd simulation 🔴 MASSIVE, Disney Denizen P2
21 Procedural vegetation ❌ → 📋 Neith 44.5 (terrain/building nodes) Weta Lumberjack/Totara, Disney Bonsai P3 — Neith Phase 44
22 Muscle/skin simulation 🔴 → 📋 Neith 44.2.1.13 (muscle system) Weta Tissue, DNEG Ziva P3 — Neith Phase 44
23 Material/shading system ❌ → 📋 Neith 42.1.3-4 (PBR + graph editor, 30 tasks) ILM Lama, Disney BRDF P2 — Neith Phase 42
24 Compositing tools 🔴 → 📋 Neith 44.4 (60 tasks) Nuke, ILM CompTime, Weta Eddy P3 — Neith Phase 44
25 Color science pipeline ❌ ACES, OpenColorIO, Disney Ptex P1 — Adopt standards
26 HDR mastering pipeline 🔴 Netflix Dolby Vision, HDR10+ P1
27 Video encoding optimization 🔴 Netflix VMAF, Dynamic Optimizer, AV1 P2
28 Perceptual quality metrics ❌ Netflix VMAF P1 — Adopt
29 Forensic watermarking ❌ Netflix, Google SynthID P2
30 Content DRM ❌ Widevine, FairPlay, PlayReady P2
31 IMF/delivery pipeline ❌ Netflix Photon, SMPTE st2067 P2
32 Virtual production (LED wall control) 🟡 (Bellona VP lib) ILM StageCraft, Cuebric P2
33 AI backgrounds for virtual production ❌ Cuebric P2
34 Render farm management ❌ Pixar Tractor, SPI OpenCue P2
35 Rigging tools ❌ → 📋 Neith 44.2.1 (15 tasks) ILM BlockParty, Pixar CurveNet Neith Phase 44
36 Editorial timeline ❌ → 📋 Neith 44.4.1 (15 tasks) Pixar OTIO, DaVinci Resolve Neith Phase 44
37 Storyboarding tools 🟡 (Hathor pre-prod) Pixar Pitch Doctor, SPI Flix P3
38 ML denoising for renders ❌ → 📋 Neith 44.1.6.14 Pixar/Disney ML Denoiser (Oscar) Neith Phase 44
39 Gaussian Splatting in compositing ❌ Nuke 17 native splats P2
40 CopyCat-style per-shot ML training ❌ Nuke CopyCat/BigCat P2
41 On-set production tools (call sheets, sides) ❌ Netflix Prodicle P3
42 Dailies review pipeline ❌ Autodesk Flow Capture, Netflix MPS P2
43 Neural style transfer ❌ Disney Research, Pixar (Elemental) P3
44 Subdivision surfaces ❌ Pixar OpenSubdiv P3 — Adopt
45 Scene description authoring (USD) 🟡 (Bellona pipeline) Pixar USD/Hydra P2 — Deepen
46 Budget/resource management (Phase 26.4) ❌ (0/30 tasks) Unique to Oshun P0 — Build
47 Canon/consistency enforcement (Phase 26.5) ❌ (0/30 tasks) Unique to Oshun P0 — Build
48 Self-improvement system (Phase 26.6) ❌ (0/30 tasks) Unique to Oshun P0 — Build
49 Human override protocols (Phase 26.7) ❌ (0/30 tasks) Unique to Oshun P0 — Build
50 End-to-end verification (Phase 26.8) ❌ (0/40 tasks) Unique to Oshun P0 — Build

5. Gap Analysis by Category#

5.1 Rendering & Real-Time Graphics#

Industry SOTA:

  • ILM Helios — Vulkan RT real-time cinema renderer for LED walls
  • Pixar RenderMan 27 XPU — Hybrid CPU/GPU, 2-10x faster, used on Toy Story 5
  • Disney Hyperion — Spectral path tracer with ML denoiser and next-gen path guiding
  • Weta Manuka — Spectral path tracer (1.248 billion render hours for Avatar: Fire and Ash)
  • NVIDIA DLSS 4.5 — 6X Dynamic Multi Frame Generation, neural rendering pipeline
  • DreamWorks MoonRay — Open-source MCRT renderer with XPU mode

Oshun Status: No rendering capability. The Maya domain (planned) would need this for the metaverse vision.

Recommendation — Build @maya/renderer (Rust + wgpu):

  • WebGPU-based renderer for cross-platform (browser, desktop, mobile)
  • Path tracing with ML denoising (adopt OIDN - Open Image Denoise)
  • Hybrid rasterization + ray tracing mode
  • Integrate with USD/Hydra as render delegate
  • Adopt open-source MoonRay concepts for production quality
  • Neural rendering pipeline for DLSS-like upscaling
  • Priority: P2 (needed for Maya domain, long-term)

5.2 AI Video Generation#

Industry SOTA:

  • Sora 2 — Highest realism for photorealistic scenes
  • Veo 3.1 — Detailed visuals, 1080p, 8-second clips
  • Runway Gen-4.5 — Most complete creative platform
  • Kling 2.0 — 2-minute continuous clips (duration leader)
  • Luma Ray3.14 — Native HDR, 16-bit EXR, fastest in class
  • Higgsfield Cinema Studio 2.0 — Optical physics simulation, multi-model aggregation

Oshun Status: Isis has Hunyuan (via FAL) and WAN (via AIML). 5 generation modes, 7 quality presets, 8 camera presets, 10 style presets.

Gaps:

  1. Missing providers: Sora, Veo, Kling, Pika, Luma, MiniMax
  2. No native HDR video generation
  3. No optical physics simulation for camera/lens
  4. No multi-model ensemble (generate with multiple models, pick best)
  5. No 16-bit EXR export from generation

Recommendation — Extend @isis/video-generation:

  • Add provider adapters for Sora 2, Veo 3.1, Runway Gen-4.5, Kling 2.0, Luma Ray3, Pika 2.5, MiniMax Hailuo
  • Implement multi-model ensemble routing (generate with N providers, score with VMAF-like metric, select best)
  • Add HDR pipeline with EXR output
  • Add camera/lens physics simulation (focal length, depth of field, bokeh, motion blur)
  • Implement Act-One-style facial performance capture integration
  • Priority: P1

5.3 Post-Production AI#

Industry SOTA:

  • InterPositive (Netflix) — Per-production model from dailies: relighting, reframing, BG replacement, VFX
  • Topaz Video AI — 8K upscaling, frame interpolation, denoising
  • Nuke CopyCat/BigCat — Train ML on production's own footage for shot-level automation
  • DaVinci Resolve Neural Engine — 15+ AI features, all local GPU
  • Autodesk Flame — AI upscaling, AutoMatte, ML Slow-Mo, ONNX model loading

Oshun Status: No post-production AI. Yemaya rendering-pipelines handles export but not AI-enhanced post.

Recommendation — Build @isis/post-production-ai (NEW LIBRARY):

This is the single most critical gap. Build an InterPositive-equivalent:

  1. Dailies Ingestion Pipeline

    • Ingest raw footage (ProRes, RAW, various codecs)
    • Extract per-frame cinematic metadata (focal length, camera movement, depth of field, angle, occlusion, parallax)
    • Build production-specific visual vocabulary
  2. Per-Production Model Training

    • Train lightweight model on production's own dailies
    • Learn visual language, lighting signatures, color palette, lens characteristics
    • Model understands editorial consistency for that specific show
  3. AI Relighting Engine

    • Modify lighting in already-shot footage
    • Match lighting between shots for continuity
    • Add/remove light sources in post
  4. AI Shot Reframing/Recomposition

    • Reframe existing shots (extract different compositions)
    • Synthesize missing coverage from existing footage
    • AI-assisted editorial decisions
  5. AI Background Replacement/Enhancement

    • Replace or enhance backgrounds using production-trained model
    • Maintain lighting and perspective consistency
  6. AI Continuity Correction

    • Detect continuity errors across shots
    • Auto-correct minor inconsistencies
  7. AI Upscaling & Enhancement (Topaz-equivalent)

    • Super-resolution (2x, 4x, 8x) using ESRGAN, Real-ESRGAN, or custom models
    • Frame interpolation (24→60fps, 30→120fps)
    • AI denoising for low-light footage
    • Film grain management (synthesis/removal, like Netflix AV1 FGS)
    • Codec artifact removal
    • AI sharpening
  8. ONNX Model Runner

    • Load custom ONNX ML models (like Flame's approach)
    • Enable studios to deploy their own trained models within the pipeline
  • Priority: P0

5.4 Digital Humans & Face Technology#

Industry SOTA:

  • ILM FaceSwap — ML + CG de-aging (Indiana Jones: 100+ artists, 3 years)
  • ILM MEDUSA — Markerless 4D face scanning at pore/wrinkle level
  • ILM ANYMA — 20 shapes → 200+ facial attributes
  • ILM FLUX — Markerless on-set facial capture under theatrical lighting (IR)
  • Weta APFS — Anatomically plausible facial system, 90-95% performance fidelity
  • Weta Deep Shapes — ML-enhanced facial deformation beyond blendshapes
  • Weta Bodyopt — Neural network body deformation
  • Disney Research — Neural face rendering, Gaussian codec avatars, high-res face swapping, MoRF, ReNeRF
  • Metaphysic/Brahma (DNEG) — Photo-real de-aging (Tom Hanks in Here)
  • MARZ Vanity AI — Automated de-aging (110+ TV productions)
  • Runway Act-One — Smartphone facial performance → character animation
  • NVIDIA ACE — Audio2Face, digital human microservices

Oshun Status:

  • Psyche domain has avatar customization, expressions, lipsync
  • Hathor has NPC dialogue with NVIDIA ACE integration
  • No face synthesis, no de-aging, no facial performance capture, no 4D scanning

Recommendation — Build across Isis + Psyche:

  1. @isis/face-synthesis (NEW)

    • Face de-aging/aging using diffusion models
    • Face swapping (actor → character)
    • Emotional expression transfer
    • Volumetric 3D face reconstruction from 2D video
    • Train on actor reference footage
  2. @psyche/facial-capture (EXTEND)

    • Smartphone-based facial performance capture (like Act-One)
    • MediaPipe or custom face mesh tracking
    • FACS-based expression decomposition
    • Retarget actor performance to CG character
  3. @isis/visual-dubbing (NEW)

    • Lip-sync adaptation for dubbed dialogue (TrueSync-equivalent)
    • Preserve emotional delivery across languages
    • Volumetric face representation for natural mouth movement
    • Integrate with ElevenLabs/Respeecher for voice + Flawless-style visuals
  • Priority: P1

5.5 Motion Capture & Animation#

Industry SOTA:

  • ILM IMoCap — Image-based mocap with patterned bands, no triangulation needed
  • Weta FACETS/APFS — Full performance capture, underwater capability
  • Wonder Dynamics — Video → CG character replacement, markerless
  • Move AI — 2nd-gen spatial motion models, 8 people simultaneously
  • DeepMotion SayMotion — Text-to-3D-animation (no video needed)
  • Rokoko — Hybrid hardware + AI video mocap
  • Cascadeur — AI inbetweening + physics-based animation
  • Autodesk MotionMaker — GenAI animation in Maya
  • Pixar Presto — ML Posing, invertible rigs, CurveNet

Oshun Status:

  • Aja domain: 38 libraries for pose estimation, 3D reconstruction, retargeting, motion quality
  • BVH/FBX/GLB/USD/Alembic format support
  • IK solvers, human mesh recovery

Gaps:

  1. No text-to-animation (DeepMotion SayMotion equivalent)
  2. No AI inbetweening with physics (Cascadeur equivalent)
  3. No video-to-CG-character replacement (Wonder Dynamics equivalent)
  4. No multi-person simultaneous capture at scale
  5. No GenAI animation from prompts

Recommendation — Extend Aja + Build New:

  1. @aja/text-to-animation (NEW)

    • Text prompt → full 3D animation sequence
    • Integrate motion diffusion models
    • Physics-informed generation (gravity, momentum, balance)
  2. @aja/ai-inbetweening (NEW)

    • Given 2 keyframes, generate smooth in-between motion
    • Physics-aware trajectory calculation
    • Style transfer between motion clips
  3. @aja/cg-replacement (NEW)

    • Actor video → skeletal extraction → CG character application
    • Automatic lighting estimation from video
    • Scene decomposition (foreground/background)
    • Integrate with Isis for character generation
  • Priority: P1-P2

5.6 Simulation & Physics#

Industry SOTA:

  • ILM PhysBAM — Level sets, FLIP/PIC, destruction (Oscar-winning, Stanford collab)
  • Weta Loki — Unified multiphysics (fluid, rigid, deformable, declarative state machine)
  • Disney Matterhorn — MPM for snow, mud, sand, foam
  • Houdini 21 — Industry standard for procedural VFX/sim
  • Pixar FizT — Cloth/hair dynamics (2.3M hairs on Sully)

Oshun Status: No physics simulation capability. Maya domain (planned) calls for Rust physics (Rapier).

Recommendation — Build @maya/physics (Rust):

  • Rapier integration for rigid body dynamics
  • Cloth simulation (position-based dynamics)
  • Hair simulation (Cosserat rods)
  • Fluid simulation (SPH or FLIP/PIC)
  • GPU-accelerated via wgpu compute shaders
  • WebAssembly build for browser-based preview
  • Priority: P2 (blocked on Maya domain progress)

5.7 Hair, Fur & Cloth#

Industry SOTA:

  • Weta Barbershop — Hair/fur creation, Oscar-winning, per-strand control
  • Weta Wig — Hair dynamics simulation
  • Pixar FizT/FizT2 — Cloth and hair sim (2.3M hairs)
  • Disney Tonic — Volume-based hair grooming
  • MPC Furtility — Procedural fur
  • Framestore Fibre — Hair/fur/feather dynamics, 80% less manual fixing

Oshun Status: No hair, fur, or cloth capability anywhere in the ecosystem.

Recommendation: Defer to Maya domain Rust implementation or integrate with Houdini via Bellona bridge. Not a near-term priority unless Maya domain accelerates.

  • Priority: P3

5.8 Crowd Simulation#

Industry SOTA:

  • MASSIVE — Fuzzy-logic agent AI, motion library blending (LOTR, now commercial)
  • Disney Denizen — 16,000+ unique crowd characters
  • MPC ALICE — 30+ films
  • Houdini Crowd — Procedural crowd system

Oshun Status: No crowd simulation. Hathor has NPC systems but for interactive gameplay, not cinematic crowds.

Recommendation — Build @hathor/crowds (NEW):

  • Agent-based crowd system with behavior rules
  • Motion library integration (from Aja)
  • Procedural character variation (body, clothing, accessories)
  • Export to Unreal/Unity via Bellona
  • Priority: P2-P3

5.9 Virtual Production#

Industry SOTA:

  • ILM StageCraft — Emmy-winning LED volume, Helios real-time renderer, used on 20+ productions
  • Cuebric — AI-generated 2.5D backgrounds for LED walls, real-time modification
  • DNEG 360 — Two of world's largest LED volumes (London, Rome)
  • Framestore Farsight — VP ecosystem on Unreal (Loki, Wonka, Barbie)
  • MPC Genesis — Unity-based VP platform
  • Netflix OpenVPCal — Open-source LED wall camera calibration

Oshun Status: Bellona has virtual-production library, but focused on pipeline integration, not control.

Recommendation — Extend @bellona/virtual-production:

  1. LED wall control protocols (receive render output, send to panels)
  2. Camera tracking integration (HTC Vive, OptiTrack, STYPE)
  3. AI background generation for LED walls (integrate Isis generation → Bellona VP display)
  4. Adopt Netflix OpenVPCal for camera calibration
  5. Real-time environment modification during shooting
  • Priority: P2

5.10 Audio & Music AI#

Industry SOTA:

  • ElevenLabs — 70+ languages, v3 emotional tags, Eleven Music, $11B valuation
  • Suno v5 — Full songs with vocals/lyrics, 44.1kHz, Suno Studio (AI-native DAW)
  • Udio — Music gen with deep customization, WMG/UMG licensed
  • AIVA — 250+ styles, cinematic/orchestral focus, MIDI export
  • Boomy — Generation + distribution to 40+ platforms
  • Google Lyria 3 — Vocals + lyrics + cover art from prompts

Oshun Status:

  • Euterpe domain has music theory, DAW engine foundations, spatial audio (Dolby Atmos, ambisonics)
  • TTS integration exists
  • Podcast generator exists
  • No AI music generation
  • No AI voice cloning

Recommendation — Extend Euterpe + Isis:

  1. @euterpe/ai-music-generation (NEW)

    • Multi-provider music generation (Suno API, Udio API, AIVA API, Google Lyria API)
    • Prompt-to-score pipeline for film/game soundtracks
    • Stem separation and remix capability
    • MIDI export for DAW refinement
    • Style-consistent generation across scenes
    • Integrate with Yemaya orchestration for automated scoring
  2. @isis/voice-cloning (NEW)

    • ElevenLabs integration for professional voice cloning
    • Respeecher integration for film-grade voice synthesis
    • Voice preservation for archival/restoration
    • Multilingual voice conversion
    • Real-time voice transformation
  • Priority: P1

5.11 Voice AI & Dubbing#

Industry SOTA:

  • Flawless TrueSync — Visual dubbing (lip movements match dubbed audio)
  • Flawless DeepEditor — Change dialogue in post while maintaining lip sync
  • Netflix DeepSpeak — AI dubbing matching original actor performance
  • HeyGen — 175+ language video translation with lip sync
  • Respeecher — Voice cloning for Star Wars (Luke, Vader)

Oshun Status: TTS integration exists. No visual dubbing, no voice cloning, no DeepEditor equivalent.

Recommendation: See 5.4 (@isis/visual-dubbing) and 5.10 (@isis/voice-cloning).

  • Priority: P1

5.12 Asset Management & Pipeline#

Industry SOTA:

  • Autodesk Flow Production Tracking (formerly ShotGrid) — Industry standard
  • Netflix Content Hub/CDrive — Petabyte-scale cloud asset management
  • MPC Packaging — Hundreds of thousands of assets across global facilities
  • OpenUSD — Scene description standard
  • Alembic — Animated scene interchange
  • MaterialX — Material interchange

Oshun Status:

  • Yemaya: 15 asset types, versioning, metadata extraction, SHA-256, thumbnails
  • Bellona: GLTF/USD/FBX interchange
  • Good foundations but not at studio scale

Gaps:

  1. No ShotGrid-like task/shot tracking (task → artist → review → approve)
  2. No Alembic support (animated cache interchange)
  3. No MaterialX authoring (material interchange)
  4. No editorial conform pipeline

Recommendation — Extend existing:

  1. Add shot/task tracking to Yemaya projects (ShotGrid equivalent)
  2. Add Alembic import/export to Bellona
  3. Add MaterialX import/export to Bellona
  4. Build editorial conform pipeline in Yemaya
  • Priority: P2

5.13 Color Science & HDR#

Industry SOTA:

  • ACES — Academy Color Encoding System (industry standard linear workflow)
  • OpenColorIO — Color management standard (Sony Imageworks created, ASWF governed)
  • Netflix — ACES mandated, Dolby Vision hero grade, P3-D65 monitors
  • Disney — Principled BRDF, Ptex, ACES adoption
  • HDR10+ — Dynamic frame-by-frame metadata

Oshun Status: No color science pipeline. Rendering pipelines have basic export but no color management.

Recommendation — Build @oshun/color-science (NEW shared library):

  1. ACES integration (ACEScg working space, ACES output transforms)
  2. OpenColorIO integration (color transform management)
  3. Dolby Vision metadata handling
  4. HDR10/HDR10+ pipeline
  5. AMF (ACES Metadata File) support
  6. Monitor calibration profiles
  • Priority: P1

5.14 Encoding & Streaming#

Industry SOTA:

  • Netflix — VMAF, Dynamic Optimizer, AV1 + Film Grain Synthesis, per-shot encoding
  • Disney+ — HEVC 4K HDR, MV-HEVC for Vision Pro, Akamai CDN
  • AV1 — 30% of Netflix viewing, 66% bitrate reduction with FGS

Oshun Status: Yemaya has codec support (H264, H265, VP9, AV1, ProRes) for export but no encoding optimization.

Recommendation — Build @oshun/encoding (NEW shared library):

  1. VMAF integration for perceptual quality measurement
  2. Shot-based encoding (scene detection → per-shot optimization)
  3. AV1 encoding with film grain synthesis
  4. AVIF for image assets
  5. Adaptive bitrate ladder generation
  6. Dolby Vision/HDR10+ encoding pipeline
  • Priority: P2

5.15 Content Security & DRM#

Industry SOTA:

  • Multi-DRM — Widevine + FairPlay + PlayReady
  • Forensic watermarking — Per-session, survives transcoding
  • Google SynthID — AI content watermarking (text, image, audio, video)
  • Netflix — Forensic watermarks + DRM + Prodicle dynamic watermarking

Oshun Status: Yemaya safety library has content filtering but no DRM or watermarking.

Recommendation — Build @oshun/content-security (NEW shared library):

  1. Forensic watermarking (invisible, per-session)
  2. AI content provenance (C2PA/SynthID integration)
  3. DRM integration (Widevine, FairPlay, PlayReady)
  4. Document watermarking (scripts, storyboards)
  5. Access-controlled asset distribution
  • Priority: P2

5.16 Collaboration & Review#

Industry SOTA:

  • Autodesk Flow Capture (Moxion) — Cloud dailies review
  • SyncSketch (Unity) — Real-time visual review
  • SPI Flix — Collaborative storyboarding
  • Netflix Prodicle — On-set communication
  • Frame.io (Adobe) — Video review and approval

Oshun Status: Yemaya has CRDT collaboration (Yjs), presence tracking, WebSocket sync, voice/video integration. This is strong.

Gaps:

  1. No visual review with frame-accurate annotation (like Frame.io)
  2. No dailies review pipeline
  3. No version comparison (side-by-side, wipe, overlay)

Recommendation — Extend @yemaya/collaboration:

  1. Frame-accurate video annotation
  2. Dailies review workflow (ingest → review → notes → distribute)
  3. Version comparison tools (side-by-side, wipe, overlay, onion skin)
  4. Integration with DaVinci Resolve for color review
  • Priority: P2

5.17 Production Management#

Industry SOTA:

  • Autodesk Flow Production Tracking — Shots, tasks, artists, reviews, schedules
  • Netflix Prodicle — 7 apps for on-set management
  • MPC Pronto — Bidding, scheduling, global resource planning
  • Framestore — Integrated production tracking

Oshun Status: Yemaya has project management (6 types), role-based access, activity feeds. Phase 26.4 adds budget/timeline management (not started).

Gaps:

  1. Shot-level tracking (not just project/asset level)
  2. Call sheet generation
  3. On-set communication tools
  4. Crew scheduling
  5. Budget tracking with actuals vs. estimates

Recommendation — Extend @yemaya/projects + Complete Phase 26.4:

  1. Add shot entity with status tracking (layout → anim → lighting → comp → final)
  2. Add crew management module
  3. Add call sheet generation
  4. Complete Phase 26.4 (30 tasks) for autonomous budget/timeline
  • Priority: P1 (Phase 26.4), P2 (production tools)

5.18 Quality Control & Validation#

Industry SOTA:

  • Netflix Automated QC — ML-powered pass/fail prediction
  • Netflix VMAF — Perceptual video quality metric
  • Pixar/Disney ML Denoiser — Oscar-winning quality from noisy renders
  • Phase 26.2 (Oshun's own) — AAA quality benchmarking (LPIPS, FID, narrative, gameplay)

Oshun Status: Phase 26.2 is COMPLETE (75/75 tasks). This is a major strength.

Gaps:

  1. No video QC for delivery (encoding artifacts, audio sync, subtitle issues)
  2. No VMAF integration
  3. No IMF validation

Recommendation:

  1. Adopt VMAF for video quality assessment
  2. Build delivery QC pipeline (encoding artifacts, A/V sync, subtitles, IMF compliance)
  3. Integrate Netflix Photon (open-source) for IMF validation
  • Priority: P2

5.19 Worldbuilding & Narrative AI#

Industry SOTA: No competitor matches Oshun here. The closest are:

  • Hathor is already SOTA for AI worldbuilding
  • Framestore + iwot Studios are building an AI-enabled platform (announced Feb 2026, early stage)

Oshun Status: Hathor with 13 libraries covering world simulation, story graphs, quests, dialogue, LLM NPCs, lore validation, economy/politics/culture simulation. Sophia provides knowledge/research backbone.

This is a clear competitive advantage. No gaps.

5.20 Autonomous Production#

Industry SOTA: No competitor has anything like this. Oshun is defining the category.

Oshun Status: Phase 26 — 200/360 tasks complete (56%).

Critical remaining work: | Section | Tasks Left | Criticality | |---------|-----------|-------------| | 26.4 Budget & Resource Management | 30 | CRITICAL — Can't produce without budgets | | 26.5 Canon & Consistency | 30 | CRITICAL — Can't do cross-media without canon | | 26.6 Self-Improvement | 30 | HIGH — Learning improves every production | | 26.7 Human Override | 30 | CRITICAL — Safety requirement | | 26.8 End-to-End Verification | 40 | CRITICAL — Must verify before sign-off |

This is the highest priority work in the entire ecosystem.


6. In-House Build Recommendations#

Tier 0: Complete Phase 26 (160 tasks remaining)#

These are already planned and partially built. Finish them.

Phase Tasks Estimated Effort
26.4 Budget & Resource Management 30 Large
26.5 Canon & Consistency Enforcement 30 Large
26.6 Self-Improvement System 30 Large
26.7 Human Override & Emergency 30 Large
26.8 End-to-End Verification 40 Very Large

Tier 1: Critical New Capabilities (Build In-House)#

New Library/Extension What It Does Models Competitor Domain
@isis/post-production-ai Per-production model training, relighting, reframing, BG replacement, upscaling, denoising InterPositive + Topaz Isis
@isis/face-synthesis De-aging, face swap, expression transfer, volumetric 3D face Metaphysic, ILM FaceSwap, MARZ Isis
@isis/visual-dubbing Lip-sync adaptation for dubbed dialogue Flawless TrueSync, MARZ LipDub Isis
@isis/voice-cloning Professional voice cloning & synthesis ElevenLabs, Respeecher Isis
@euterpe/ai-music-generation Multi-provider AI music for film/game Suno, Udio, AIVA, Lyria Euterpe
@oshun/color-science ACES, OCIO, HDR, Dolby Vision pipeline Netflix, Disney, industry standard Shared
@isis/video-generation (extend) Add 5+ providers, HDR, multi-model ensemble All AI video companies Isis

Tier 2: Important Capabilities (Build or Integrate)#

Library What It Does Priority
@aja/text-to-animation Text prompt → 3D animation sequence P2
@aja/ai-inbetweening Physics-aware motion interpolation P2
@aja/cg-replacement Video → CG character replacement (Wonder Dynamics equivalent) P2
@oshun/encoding VMAF, shot-based encoding, AV1+FGS, AVIF P2
@oshun/content-security Forensic watermarking, DRM, C2PA/SynthID P2
@bellona/virtual-production (extend) LED wall control, camera tracking, AI backgrounds P2
@yemaya/dailies-review Ingest → review → annotate → distribute P2
@yemaya/editorial OTIO integration, editorial timeline, conform pipeline P2
@bellona/alembic Animated scene interchange P2
@bellona/materialx Material interchange standard P2
@oshun/render-farm Job distribution, farm management (adopt OpenCue) P2

Tier 3: Long-Term / Deferred#

Library What It Does When
@maya/renderer Rust/wgpu real-time renderer When Maya domain accelerates
@maya/physics Rapier-based rigid/fluid/cloth sim When Maya domain accelerates
@hathor/crowds Agent-based crowd simulation When needed for production
@isis/neural-style-transfer Stylize footage/renders P3
Rigging tools Character rigging system P3 — Use DCC tools instead
Compositing Node-based compositor P3 — Use Nuke/DaVinci instead
Hair/fur grooming Hair creation tools P3 — Use Houdini/Maya instead

Neith Sovereign Stack (Phases 41-46) — Already Planned In-House#

The following capabilities are already represented in TODOS.md under the Neith Sovereign Stack (Phases 41-46), Oshun's plan to build sovereign creative technology from the ground up in Rust:

Phase Library What It Builds Industry Equivalent Tasks
41.1 @neith/core Runtime kernel, ECS, memory, threading Custom engine foundation 90
41.2 @neith/hal Hardware abstraction (Vulkan, Metal, DX12, WebGPU) Engine HAL 105
41.3 @neith/crypto Cryptography & security — 75
41.4 @neith/net Networking stack — 60
41.5 @neith/ui UI toolkit Qt, Dear ImGui 90
42.1 @neith/renderer Real-time rendering (Nanite-class virtualized geometry, GI, PBR, ray tracing) ILM Helios, Unreal Nanite/Lumen, Pixar RenderMan XPU 150
42.2 @neith/physics Multi-physics (rigid, soft, fluid, destruction, cloth) ILM PhysBAM, Weta Loki, Disney Matterhorn, Houdini 120
42.3 @neith/audio Spatial audio engine (Dolby Atmos, ambisonics, HRTF) Wwise, FMOD, Weta audio pipeline 90
42.4 @neith/ecs Entity component system Unreal/Unity ECS 60
42.5 @neith/scripting Scripting integration Unreal Blueprint, Unity C# 45
42.6 @neith/animation Full animation system (skeletal, blend trees, state machines, IK, ragdoll) Pixar Presto, DreamWorks Premo 75
43.1 @neith/ai-runtime Game AI & inference (behavior trees, GOAP, ML inference) NVIDIA ACE, custom AI 90
43.2 @neith/assets Asset pipeline (custom binary format, streaming, LOD) Studio asset pipelines 75
43.3 @neith/scene Scene management (scene graph, streaming, instancing) USD/Hydra 45
43.4 @neith/particles Particle system (GPU, VFX graph, ribbons) Unreal Niagara, Houdini 30
44.1 @neith/sculptor 3D modeling & sculpting (mesh editing, sculpting, UV, materials, rendering) Blender, Maya, ZBrush 90
44.2 @neith/animator Rigging, animation timeline, motion capture, facial animation (FACS) Maya, MotionBuilder, Pixar Presto 60
44.3 @neith/composer Full DAW (multi-track, mixing, MIDI, synthesis, spatial audio authoring) Pro Tools, Logic Pro, Ableton 75
44.4 @neith/cutter Video editing, color grading, compositing, export DaVinci Resolve, Premiere, Nuke 60
44.5 @neith/weaver Procedural content (node graph, geometry, terrain, buildings) Houdini, World Machine 60
45 @neith/linux Custom NixOS-based creative workstation OS — 200+
46 @neith/browser Servo-based web browser Chrome, Firefox 200+
TOTAL NEITH ~1,900+

This means the gap analysis "What NOT to Build" list is actually empty for creative tools — Oshun's roadmap already plans to build everything in-house. The Neith stack would make Oshun the first organization to own a fully sovereign creative technology stack from OS to browser to engine to DCC tools.

What NOT to Build (True Exceptions)#

Capability Reason Use Instead
CDN Netflix spent $1B+, infrastructure business AWS CloudFront, Cloudflare
Hardware mocap suits Physical product, different business model Rokoko, Xsens (consume their data)
Camera/lens hardware Physical optics manufacturing ARRI, RED (integrate their footage)

7. Phase 26 Completion Priority#

Phase 26 is the crown jewel of the Oshun ecosystem — the capability that no competitor has. Completing it should be the #1 priority.

  1. 26.7 Human Override & Emergency Protocols (30 tasks)

    • Why first: Safety requirement. Cannot run autonomous production without stop/override capability.
    • Emergency stop, checkpoint preservation, human takeover, decision injection
  2. 26.4 Budget & Resource Management (30 tasks)

    • Why second: Cannot produce without budgets. Token tracking, GPU compute, cost optimization.
    • Schedule generation, critical path, resource leveling
  3. 26.5 Canon & Consistency Enforcement (30 tasks)

    • Why third: Essential for cross-media franchise (Phase 27 Project Obsidian needs this).
    • Canonical truth database, style enforcement, character consistency
  4. 26.8 End-to-End Verification (40 tasks)

    • Why fourth: Verification is the final gate before autonomous sign-off.
    • Completeness checking, AAA benchmarking, franchise coherence
  5. 26.6 Self-Improvement System (30 tasks)

    • Why last: Improvement is optimization, not critical path.
    • Learning from outcomes, quality feedback, cross-project learning

8. Strategic Roadmap#

Near-Term (Next 3 Months)#

Priority Work Item Tasks
P0 Complete Phase 26.7 (Human Override) 30
P0 Complete Phase 26.4 (Budget/Resource) 30
P0 Complete Phase 26.5 (Canon/Consistency) 30
P0 Build @isis/post-production-ai (core) New library
P1 Extend Isis video gen (add 5+ providers) Extension
P1 Build @oshun/color-science New library

Medium-Term (3-6 Months)#

Priority Work Item Tasks
P0 Complete Phase 26.8 (Verification) 40
P0 Complete Phase 26.6 (Self-Improvement) 30
P1 Build @isis/face-synthesis New library
P1 Build @isis/visual-dubbing New library
P1 Build @isis/voice-cloning New library
P1 Build @euterpe/ai-music-generation New library
P1 Extend Isis post-production-ai (upscaling, enhancement) Extension

Long-Term (6-12 Months)#

Priority Work Item Notes
P2 Build @aja/text-to-animation Motion diffusion models
P2 Build @aja/cg-replacement Wonder Dynamics equivalent
P2 Build @oshun/encoding VMAF, AV1, shot-based
P2 Build @oshun/content-security Watermarking, DRM
P2 Extend Bellona virtual production LED walls, camera tracking
P2 Build @yemaya/dailies-review Review pipeline
P2 Build @yemaya/editorial OTIO, conform
P2 Begin Phase 27 (Project Obsidian) Requires Phase 26 complete

Aspirational (12+ Months)#

Priority Work Item Notes
P2-P3 @maya/renderer (Rust + wgpu) Real-time rendering
P2-P3 @maya/physics (Rapier) Simulation
P3 @hathor/crowds Crowd simulation
P3 @isis/neural-style-transfer Style transfer
P3 Render farm management Adopt OpenCue

Appendix A: Complete Competitor Tool Inventory#

ILM Tools (20+)#

Zeno, Helios, Plume, CompTime, SABRE, BlockParty, StageCraft, IMoCap, FLUX, MEDUSA, ANYMA, PhysBAM, Lama, FaceSwap, Zviz, MARS, MaterialX, OpenEXR, Alembic, custom lighting tools

Pixar Tools (15+)#

Presto, RenderMan (XPU), Luna, USD, Hydra, OpenSubdiv, FizT/FizT2, Tractor, Pitch Doctor, Geppetto, CurveNet, OpenTimelineIO, OpenEXR (contributor), ML Denoiser, crowd pipeline

Disney/WDAS Tools (15+)#

Hyperion, Meander, Denizen, Bonsai, Tonic, XGen, Matterhorn, Ptex, SeExpr, Partio, Disney BRDF, Splash, Quicksilver, Dynamic Wires, CAPS (historical), Command Center

Weta FX Tools (15+)#

Manuka, Gazebo, Lumberjack, Totara, Barbershop, Wig, Tissue, FACETS, APFS, Deep Shapes, Bodyopt, Loki, Eddy, PhysLight/PhysCam, MASSIVE

Netflix Tools (20+)#

InterPositive, VMAF, Dynamic Optimizer, Cosmos, MPS, Content Hub, CDrive, Backlot, Prodicle (7 apps), Photon, Eyeline (Studios/VFX/Labs), Open Connect, OpenVPCal, ALab, Maestro, DeepSpeak

DNEG Tools#

Helix pipeline, Ziva VFX, Brahma AI, CLEAR/CLEAR AI, proprietary Nuke automation, deep compositing pipeline

Framestore Tools#

Freak (renderer), Fibre (dynamics), FIRA (ML deformation), Farsight (VP), layered shading system

MPC Tools#

ALICE (crowd), Furtility (fur), PAPI (dynamics), Tickle (lighting), Packaging (assets), Pronto (production), Genesis (VP)

Sony Imageworks Tools (Open Source)#

OpenColorIO, Katana (originated), Arnold (co-dev), OSL, Alembic (co-dev), OpenCue, Field3D, Flix, Sprout, Kismet

DreamWorks Tools#

Premo (animation), MoonRay (renderer, open-source), Torch (lighting), Apollo platform

AI Filmmaking Companies (30+)#

InterPositive, Runway, OpenAI Sora, Google Veo, Pika, Kling, Luma, Wonder Dynamics, Metaphysic, Cuebric, LTX Studio, Topaz, Flawless AI, Higgsfield, MARZ, Stability AI, ElevenLabs, Respeecher, Suno, Udio, AIVA, Boomy, Google Lyria, Synthesia, HeyGen, Descript, DeepMotion, Move AI, Rokoko, Cascadeur, Reallusion, Haiper, MiniMax

Platform Technology#

NVIDIA (Omniverse, ACE, DLSS, RTX Kit), Adobe (Firefly, Premiere/AE AI), Autodesk (Maya MotionMaker, Flame AI), Foundry (Nuke CopyCat/BigCat, Gaussian Splatting), Blackmagic (DaVinci Neural Engine), Epic (MetaHuman, Nanite, Lumen), Unity (Weta Tools, Ziva, AI Assistant)


Appendix B: Oshun Ecosystem Complete Domain Inventory#

# Domain Deity Libraries Apps Status
1 Yemaya Yoruba ocean goddess 35+ 5 Production
2 Isis Egyptian goddess of magic 13 5 Production
3 Hathor Egyptian goddess of love/music 13 4 Production
4 Sophia Greek goddess of wisdom 10+ 4 Production
5 Bellona Roman goddess of war 24 8 Production
6 Lilith Mesopotamian spirit 21 libs, 89 apps 89 Production
7 Psyche Greek goddess of soul 138+ Various High Activity
8 Aja Yoruba goddess of forest 38 4 Production
9 Metis Greek goddess of wisdom 17 Planned In Dev
10 Veritas Roman goddess of truth 30+ Various Production
11 Euterpe Greek muse of music 8+ Planned Library-Only
12 Uzume Japanese goddess of dawn 26 Planned Library-Only
13 Calliope Greek muse of epic poetry 20+ Planned Library-Only
14 Maya Hindu goddess of illusion 40+ planned Planned Early
15 Iris Greek goddess of rainbow 32 Planned Planning
16 Nous Greek concept of mind 1 (56 modules) 1 Production
17 Kuanyin Buddhist bodhisattva 10 Planned Production
18 Aphrodite Greek goddess of love 120+ Various Production
19 Aje Yoruba orisha of wealth 30 Various Production
20 Shakti Hindu divine feminine 28 Various Production
21 Themis Greek goddess of justice 32 Planned Library-Only
22 Nisaba Sumerian goddess of writing 18 Planned Library-Only
23 Galatea Greek myth (Pygmalion) 20 Planned Library-Only
24 Demeter Greek goddess of harvest 16 Planned Library-Only
25 Hestia Greek goddess of hearth 16 Planned Library-Only
26 Nyx Greek goddess of night 18+ Various Library-Only
27 Tara Buddhist/Hindu deity 7 3 Production
28 Arete Greek concept of excellence 11 Planned Library-Only
29 Maat Egyptian goddess of truth Various Various Library-Only
Total 380+ 89+

Appendix C: Phase 70 Gap Closure Tasks#

All gaps identified in this report have been added to TODOS.md as Phase 70: Production & Filmmaking SOTA Gap Closure — 298 tasks across 14 sections. See TODOS.md Phase 70.


Report generated 2026-03-16. Based on comprehensive research across 120+ tools, 30+ companies, 7 major VFX studios, and the complete Oshun ecosystem of 380+ libraries across 35 domains.