# Yemaya — Feature Reference > **Yemaya** is the Creative Production Studio Platform for film, games, > animation, and interactive entertainment. Named after the Yoruba goddess of > the ocean, rivers, and motherhood, Yemaya unifies professional production > tools, AI-assisted workflows, and real-time collaboration into a single > cohesive ecosystem — from AAA blockbusters to solo creator projects. Yemaya solves the core problem of creative production: fragmentation. Studios juggle dozens of disconnected tools for scripting, storyboarding, asset management, 3D production, rendering, budgeting, and team communication. Yemaya consolidates all of these into one platform with three operating modes: fully manual (human drives everything), AI-assisted (humans approve AI suggestions), and fully autonomous (AI agents execute complete production pipelines end-to-end with configurable human checkpoints). The platform spans 47 libraries under `libs/yemaya/` and 5 applications across web, desktop, CLI, and SDK surfaces. --- ## Platform Overview ### Operating Modes Yemaya supports three distinct levels of AI involvement. A studio can choose the right mode per production, or even per pipeline stage, depending on how much creative control they want to retain versus delegate. | Mode | Description | | -------------------- | ---------------------------------------------------------------------------------------------- | | **Human-Driven** | Traditional manual workflows with professional tools — humans control every decision | | **AI-Assisted** | AI suggestions appear at every stage, but humans approve every action before execution | | **Fully Autonomous** | AI agents execute complete production pipelines end-to-end with configurable human checkpoints | ### Target Users Yemaya is designed to scale from solo creators through enterprise AAA production houses, with plan-gated features that match each user type's capacity needs. | User Type | Primary Use Cases | | ------------------------ | ---------------------------------------------------------------------------------- | | AAA Studios | Large-scale blockbuster game and film production with hundreds of team members | | Indie Developers | Small-team projects with professional output and limited budget | | Solo Creators | Individual artists leveraging AI assistance to produce beyond their usual capacity | | Film Production Houses | Feature films, commercials, and virtual production pipelines | | Educational Institutions | Teaching game and film development with professional tooling | ### Library Ecosystem (47 Libraries) The full `libs/yemaya/` library set is listed below. Each library is an independently versioned TypeScript package with its own `package.json` and `@yemaya/*` package name. | Library | Package | Description | | ------------------------- | --------------------------------- | ------------------------------------------------------------ | | `core` | `@yemaya/core` | Monitoring, resilience, cost management, config, encryption | | `types` | `@yemaya/types` | Branded ID primitives and `Result` helpers | | `database` | `@yemaya/database` | Prisma ORM client and schema | | `auth` | `@yemaya/auth` | JWT authentication, session management | | `rbac` | `@yemaya/rbac` | Role-based access control | | `activity` | `@yemaya/activity` | Activity logging, audit trails | | `projects` | `@yemaya/projects` | Project management, workspaces, versioning | | `assets` | `@yemaya/assets` | Digital asset management and pipelines | | `asset-library` | `@yemaya/asset-library` | Curated asset views and distribution | | `asset-generation` | `@yemaya/asset-generation` | Coordinated multi-modal asset generation workflows | | `organizations` | `@yemaya/organizations` | Organization and team management | | `collaboration` | `@yemaya/collaboration` | Real-time sync, presence, cursors (Yjs CRDT) | | `av-sync` | `@yemaya/av-sync` | Audio-video synchronization and drift correction | | `budget-management` | `@yemaya/budget-management` | Autonomous compute budget and resource management | | `canon-enforcement` | `@yemaya/canon-enforcement` | Canon, style, and character consistency enforcement | | `comfyui-integration` | `@yemaya/comfyui-integration` | ComfyUI workflow execution and integration | | `d3-visualizations` | `@yemaya/d3-visualizations` | D3-based production analytics charts | | `diagram-renderer` | `@yemaya/diagram-renderer` | Technical diagram rendering for production docs | | `enterprise` | `@yemaya/enterprise` | Enterprise deployment and analytics | | `event-handlers` | `@yemaya/event-handlers` | Cross-domain event handling | | `event-publisher` | `@yemaya/event-publisher` | Domain event publishing | | `human-override` | `@yemaya/human-override` | Human override, emergency stop, checkpoint system | | `marketplace` | `@yemaya/marketplace` | Asset, plugin, and AI-model marketplace | | `orchestration` | `@yemaya/orchestration` | Pipeline execution and coordination | | `autonomous-pipelines` | `@yemaya/autonomous-pipelines` | Film/game production automation | | `podcast-generator` | `@yemaya/podcast-generator` | Educational podcast episode generation | | `presentation-compiler` | `@yemaya/presentation-compiler` | Slide presentation compilation and export | | `production-verification` | `@yemaya/production-verification` | Completeness, AAA benchmark, and sign-off verification | | `project-obsidian` | `@yemaya/project-obsidian` | Franchise config for Project Obsidian (reference AAA title) | | `rendering-pipelines` | `@yemaya/rendering-pipelines` | Content rendering and visual compositing pipelines | | `safety` | `@yemaya/safety` | Content moderation and safety policy enforcement | | `sdk` | `@yemaya/sdk` | TypeScript/JavaScript SDK | | `sdk-cpp` | `@yemaya/sdk-cpp` | C++ SDK for engine integrations | | `sdk-python` | `@yemaya/sdk-python` | Python SDK for scripting and ML | | `self-improvement` | `@yemaya/self-improvement` | Production outcome tracking and agent performance learning | | `style-transfer` | `@yemaya/style-transfer` | AI-powered visual style transfer and brand management | | `tts-integration` | `@yemaya/tts-integration` | Text-to-speech integration for voiceover and narration | | `ui` | `@yemaya/ui` | Shared React UI components, theming, a11y, i18n, PWA | | `users` | `@yemaya/users` | User profile and preference management | | `video-generation` | `@yemaya/video-generation` | Hunyuan/WAN video generation workflows | | `agents` | `@yemaya/agents` | Multi-agent orchestration using LangGraph/CrewAI | | `community` | `@yemaya/community` | Forums, showcase, tutorials | | `blend-kernel` | `@yemaya/blend-kernel` | Living Scenes blend kernel — typed scene transitions | | `living-scenes-runtime` | `@yemaya/living-scenes-runtime` | Living Scenes runtime — score schema, conductor, determinism | | `pre-production` | `@yemaya/pre-production` | Screenplay/script parsing and export utilities | | `remote-film-capture` | `@yemaya/remote-film-capture` | Remote actor home-capture infrastructure | | `remote-film-mannequin` | `@yemaya/remote-film-mannequin` | Remote-film mannequin hardware and motion integration | --- ## Project Management ### Project Types Users create projects organized by production type, each with appropriate default settings and pipeline configurations: - **Film** — Feature films, short films, documentaries, and commercials; includes crew management, shooting schedules, and call sheet generation - **Game** — Video games across all genres and platforms; includes gameplay design tools, NPC behavior editors, and engine integration workflows - **Animation** — 2D and 3D animated content and motion graphics; includes rigging and animation review tools - **Commercial** — Advertising and promotional content with deliverable tracking - **Music Video** — Music video production with beat-sync timeline support - **Documentary** — Documentary filmmaking with interview management and archive asset organization ### Project Workspaces Each project provides a complete production workspace that consolidates all work into a single interface: - **Project Dashboard** — At-a-glance view of progress, recent activity, team members, active pipeline jobs, and key milestones; surfaces blockers and overdue items automatically - **Version Control** — Full version history for all project assets and configuration with one-click rollback to any prior point in the project's history - **Activity Feeds** — Real-time activity stream showing all changes, uploads, comments, status updates, and pipeline events across the entire team - **Project Settings** — Configure project metadata, visibility, target engine, export formats, and cross-domain connections to Isis, Hathor, Sophia, and Bellona ### Team and Access Management - **Member Invitations** — Invite team members via email or organization directory with role assignment at invitation time - **Role-Based Access** — Five project roles: Viewer (read-only), Contributor (add assets), Editor (full edit), Admin (manage team), Owner (full control including deletion) - **Visibility Control** — Private, Team-visible, Organization-wide, or Public; controls who can discover and view project information - **Activity Logging** — Complete audit trail of every action taken in the project for compliance and debugging --- ## Asset Management ### 15 Asset Types Yemaya manages every type of creative asset across five categories. The `AssetType` enum covers the full spectrum from raw images through fully animated VFX layers: | Category | Asset Types | | -------------- | ----------------------------------------------- | | **Visual** | Image, Video, 3D Model, Texture, Material, Font | | **Audio** | Audio (music, SFX, dialogue, ambient) | | **Narrative** | Document, Script, Storyboard | | **Design** | Character Design, Environment Design | | **Production** | Animation, VFX | ### Asset Pipeline (`@yemaya/assets`) The asset pipeline handles every stage of a file's lifecycle from upload through approval. Key behaviors: - **Upload and Ingest** — Drag-and-drop upload with automatic format detection and validation; supports bulk upload with background processing - **Automatic Metadata Extraction** — On ingest, extracts dimensions, duration, format, codec, color space, bit depth, frame rate, sample rate, and other technical metadata without manual data entry - **Thumbnail Generation** — Automatic preview thumbnail generation for all visual asset types including 3D model previews - **Checksum Verification** — SHA-256 integrity verification for every uploaded asset; detects corruption during transfer and prevents duplicate storage - **Asset Versioning** — Upload new versions while maintaining complete version history and per-version changelogs; switch between versions without losing prior work - **Usage Tracking** — Track precisely where each asset is used across projects, scenes, and compositions so refactoring is safe and impact analysis is instant - **Status Lifecycle** — Assets progress through Draft → Processing → Review → Approved → Archived states with configurable approval workflows ### Security and Scanning (`@yemaya/asset-library`) All content is scanned before it is visible to the production team. This applies to both uploaded and AI-generated assets: - **Virus Scanning** — All uploaded files scanned for malware before entering the asset library; infected files are rejected with detailed reports - **Content Safety Scan** — Automatic safety scoring of generated and uploaded content before it becomes visible to the production team - **Transcoding** — Format conversion for engine compatibility (e.g., PNG → basis-compressed textures, WAV → OGG, FBX → glTF) triggered automatically or on demand ### Organization and Discovery - **Hierarchical Folders** — Organize assets into nested folder structures with custom colors and icons for visual navigation in large libraries - **Tagging** — Apply custom and AI-suggested tags for flexible categorization across multiple organizational axes simultaneously - **Full-Text Search** — Search across asset names, descriptions, tags, and all extracted technical metadata in a single query - **Advanced Filtering** — Filter by type, format, date range, file size, status, assigned tags, and any custom metadata field - **Asset Library Views** — Curated shared library views for reusable assets surfaced across multiple projects within an organization ### Comments and Review - **Threaded Comments** — Leave comments on any asset with timestamp-linked threaded replies for contextual feedback - **Comment Resolution** — Mark comment threads as resolved when addressed; track outstanding feedback in a single view - **Pinned Comments** — Pin critical notes for team visibility so important feedback stays surfaced in busy projects --- ## Asset Generation ### Coordinated Multi-Modal Generation (`@yemaya/asset-generation`) The `asset-generation` library orchestrates comprehensive, coordinated asset creation workflows across all media types rather than treating each generation request in isolation: - **Course Asset Generation** — Batch generate educational and reference assets (thumbnails, illustrations, media bundles) that maintain a consistent visual style across an entire course or production handbook - **Style Guide Enforcement** — Attach a style guide definition to a generation run; all generated assets are validated against that style guide before being stored in the library, rejecting outputs that deviate from brand standards - **Asset Variants** — Generate multiple controlled variations of an asset from a single specification (different color palettes, lighting moods, crop ratios) for A/B testing and localization requirements - **Batch Processing** — Queue large asset generation runs as a single batch job with configurable concurrency and automatic retry on individual asset failures; the batch succeeds partially rather than failing entirely on a single error - **Template-Driven Generation** — Define a reusable asset template with parameterized slots; production teams fill in slot values and generate consistent assets without rewriting prompts for each variation --- ## Script and Storyboard Management ### Screenplay Tools - **Script Editor** — Full-featured screenplay editor with industry-standard formatting (Final Draft-compatible scene headings, action lines, dialogue, transitions) built into the production studio interface - **Script Versioning** — Every script edit creates a versioned snapshot with author attribution and timestamp; compare any two drafts side-by-side - **Collaborative Editing** — Multiple writers edit scripts simultaneously via Yjs CRDT-based sync (Conflict-free Replicated Data Types that allow concurrent edits with mathematically guaranteed conflict resolution) with character-level cursor sharing - **AI Script Analysis** — AI automatically extracts characters, locations, scenes, props, and dialogue from a screenplay and feeds the results into downstream production pipelines, eliminating manual script breakdown ### Storyboard Creation - **Visual Storyboarding** — Create storyboard sequences with shot compositions, camera angle annotations, and per-frame timing information in an integrated panel editor - **Shot Planning** — Define shots with type (wide, medium, close-up, extreme close-up), movement (static, pan, tilt, dolly, crane, handheld), framing notes, and target duration - **Storyboard-to-Pipeline** — Feed storyboard panel data directly into autonomous production pipelines so asset generation and scene assembly know exactly which shots must exist before production begins --- ## Character and World Design ### Character Design System - **Character Profiles** — Rich profiles with physical appearance, backstory, personality traits, motivations, relationships, and reference image galleries; all fields used to maintain consistency across the production - **AI Character Generation** — Generate character concept designs from text descriptions via the Isis generation pipeline; iterate with prompt refinement until the design matches the creative vision - **Character Sheet Export** — Export production-ready character reference sheets as PDFs or image packages for distribution to asset teams and engine artists ### World and Location Design - **World Builder** — Design and document fictional or real worlds with geography, culture, history, time period, climate, visual style, and narrative rules; integrates with Hathor for deep simulation and lore - **Location Management** — Define locations with rich descriptions, reference images, technical requirements (lighting rig, camera restrictions), and production logistics notes - **Location Scouting** — Digital location scouting tools for virtual and physical production; browse reference imagery, 3D scans, and satellite data - **Hathor Integration** — Deep worldbuilding via the Hathor domain for narrative design, faction systems, quest systems, economy simulation, and compiled lore exported back to Yemaya projects --- ## Engine Integration Engine integration in Yemaya is bidirectional: assets flow from the Yemaya library into the DCC tool, and render results, scene data, and build outputs flow back. Each engine bridge is specialized to that engine's native workflows. ### Blender Integration Full bidirectional integration with Blender for 3D production: - **Scene Import/Export** — Import and export complete Blender scenes with materials, lighting setups, and camera configurations preserved - **Material Sync** — Synchronize materials and shaders between the Yemaya asset library and Blender projects; changes propagate bidirectionally - **Render Job Submission** — Submit Blender render jobs from Yemaya and receive real-time progress updates via GPU worker infrastructure; supports both Cycles and EEVEE render engines - **Asset Pipeline Push** — Push any asset directly from the Yemaya library to a Blender project directory with automatic format conversion ### Godot Integration - **Project Sync** — Bidirectional synchronization of Godot project files and assets; keep the Yemaya library and Godot project in sync automatically - **Scene Management** — Create and manage Godot scenes from within Yemaya; export scene definitions that Godot can open directly - **Multi-Platform Export** — Configure and trigger Godot platform exports (PC, Mobile, Web, Console) from Yemaya with progress tracking ### Unreal Engine Integration - **Level Streaming** — Manage Unreal Engine level streaming configurations from Yemaya; define which levels load based on player location or scene context - **Asset Import** — Push assets from the Yemaya library directly into the Unreal Engine content browser with automatic conversion to engine-native formats - **Niagara VFX** — Configure and manage Niagara particle effect systems; define emitter parameters and link them to scene events from within Yemaya - **Build Pipeline** — Trigger and monitor Unreal Engine builds with real-time progress tracking and build log streaming ### Houdini Integration - **Procedural Generation** — Submit Houdini procedural generation jobs from Yemaya and receive results directly in the asset library; supports PDG (Procedural Dependency Graph) job networks - **Simulation Export** — Export Houdini simulation data (cloth, fluid, rigid body) for downstream engines with automatic format conversion ### DaVinci Resolve Integration (via SDK) - **Color Grading** — Synchronize color grading projects between Yemaya and DaVinci Resolve; share LUTs, grades, and color decisions bidirectionally - **Editing Timeline Sync** — Bidirectional editing timeline synchronization so editorial decisions made in Resolve are reflected in Yemaya project state - **Media Management** — Manage assets shared between Yemaya and Resolve with automatic proxy generation and offline media detection --- ## AI-Assisted and Autonomous Production ### Multi-Agent Orchestration (`@yemaya/agents`) Yemaya's AI system uses a multi-agent architecture based on LangGraph and CrewAI patterns where specialized agents collaborate on complex production tasks: - **Agent Dashboard** — Deploy, configure, and monitor AI agents from the studio interface; see what each agent is working on in real time - **Specialized Agent Types** — Purpose-built agents for scripting, storyboarding, asset generation, visual review, narrative consistency, and quality assurance; each agent has defined capabilities and declared tools - **Human-in-the-Loop** — Every AI action can require human approval before execution; configurable per pipeline step from fully autonomous to fully supervised - **Agent Collaboration** — Agents pass work products between each other in multi-agent pipelines; a script analysis agent feeds a character design agent which feeds a 3D generation agent automatically ### AI Content Generation - **Text Generation** — Script drafts, dialogue lines, scene descriptions, and marketing copy generated from high-level prompts and story context - **Image Generation** — Concept art, character designs, environment art, and texture generation via the Isis generation pipeline; all results land in the Yemaya asset library automatically - **3D Model Generation** — AI-generated 3D models from text or image descriptions via Isis; models pass through quality gates before entering the pipeline - **Gaussian Splatting** — NeRF and Gaussian splatting (techniques that reconstruct photo-realistic 3D representations from photo collections as millions of colored ellipsoids rather than polygons) for virtual location scouting and environment creation - **Audio Generation** — Sound effects, ambient audio, and music generation via Isis with mood, genre, and tempo control - **NPC Behavior Generation** — AI-generated NPC behavior trees and dialogue systems that reflect character personality profiles ### Autonomous Film Pipeline (`@yemaya/autonomous-pipelines`) `createFilmPipeline` builds an end-to-end film production pipeline. Its `FilmPipelineStage` model spans eight stages — pre-production, world building, asset generation, scene assembly, animation, rendering, post-production, and export — realized as the following ordered steps, each delegating to the appropriate specialist domain: 1. **Script Analysis** — AI parses the screenplay to extract characters, locations, scenes, and props using Sophia's entity extraction capabilities 2. **Visual Research** — AI researches visual references, mood boards, and style references for the chosen aesthetic direction using Sophia 3. **Character Creation** — AI generates character designs using Hathor for personality and backstory and Isis for visual asset generation 4. **Location Creation** — AI generates environment designs from script location descriptions using Hathor for world context and Isis for asset generation 5. **3D Asset Generation** — AI generates production-ready 3D characters and environment assets via Isis with automatic quality gate validation 6. **Audio Generation** — AI generates sound effects, ambient audio, and score elements via Isis with reference to scene emotional beats 7. **Scene Assembly, Camera, Lighting, Render, Composite** — Generated assets are assembled into scenes and the picture is rendered and composited through `mcp:execute_dcc` steps driving Blender and DaVinci 8. **Export** — Final export to target delivery formats and platforms through an `mcp:execute_dcc` export step with configurable quality presets 9. **Review** — A closing `yemaya:review` step performs automated quality review with an optional human approval gate (auto-approved when the quality preset is `draft`) ### Autonomous Game Pipeline Autonomous game production uses game-specific stages: game design document analysis, procedural world generation via Hathor, gameplay system definition, asset generation via Isis, level assembly via Bellona, and automated playtesting. Each stage is coordinated by Yemaya's orchestration layer. ### Pipeline Configuration (`@yemaya/orchestration`) - **Quality Presets** — Draft (fast, low-res), Preview (medium quality for stakeholder review), Production (high quality), and Final (delivery quality) presets with appropriate settings per preset - **Step Dependencies** — Explicit dependencies between pipeline steps so dependent steps only start when their prerequisites complete successfully - **Conditional Execution** — Steps execute conditionally based on runtime variables and the results of prior steps (e.g., skip audio generation if an existing approved score is available) - **Retry Logic** — Configurable retry count and exponential backoff for failed pipeline steps so transient failures do not halt the pipeline - **Timeout Control** — Per-step timeout configuration; long-running generation steps can have generous timeouts while fast validation steps have tight ones - **Variable Injection** — Pass arbitrary variables into pipeline execution at runtime to customize behavior without editing the pipeline definition ### Pipeline Templates - **Built-In Templates** — Pre-configured templates for common production workflows usable immediately without configuration - **Custom Templates** — Create, parameterize, and share custom pipeline templates with configurable variable schemas and documentation - **Template Marketplace** — Discover and adopt community-created and organization-vetted pipeline templates via the marketplace ### Pipeline Step Types (40 total) Every pipeline step has a `type` from the `StepType` Zod enum defined in `libs/yemaya/orchestration/src/schemas/pipeline.ts`. The 40 values are organized by the domain that executes each step: | Domain | Step Types | | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Isis | generate_image, generate_3d, generate_audio, generate_video, inpaint, upscale, style_transfer | | Sophia | research, analyze, summarize, embed | | Hathor | create_character, create_location, compile_lore, generate_dialogue | | Bellona | export, build, sync, convert | | MCP | execute_dcc, cross_dcc_transfer, material_consistency, animation_round_trip, production_status, resource_schedule, voice_trigger, voice_steer, voice_review, voice_progress | | Yemaya | review, approve, publish, notify | | Utility | transform, filter, merge, split, conditional, loop, parallel, wait | The MCP group drives DCC tools (Blender, DaVinci, etc.) and voice-steerable production orchestration through the Model Context Protocol. --- ## Autonomous Orchestration Infrastructure Phase 26 adds advanced autonomous capabilities that transform Yemaya from a production assistance platform into one that can direct entire productions without continuous human direction. ### Goal Decomposition Engine The goal decomposition engine translates high-level creative intentions — a sentence or paragraph describing a production — into hundreds of concrete, ordered, measurable tasks: - **LLM-Driven Goal Decomposition** — Accepts high-level franchise goals in natural language ("create a 6-episode animated series in the style of Studio Ghibli") and decomposes them into hundreds of concrete, ordered, measurable sub-tasks using large language models - **Hierarchical Task Networks (HTN)** — Generates structured task hierarchies (HTN is a planning formalism where complex tasks decompose recursively into primitive executable actions) from natural language goal descriptions - **Automatic Scope Inference** — Infers the full scope of work from a brief goal description; identifies implicit requirements the user did not state - **Constraint Extraction** — Parses constraints (budget limits, target release dates, quality thresholds) from goal descriptions and enforces them throughout the production lifecycle - **Milestone Auto-Generation** — Generates project milestones appropriate to the project type (different milestones for a game vs. a film) - **Monte Carlo Progress Estimation** — Uses Monte Carlo simulation (a statistical technique that runs thousands of random scenarios) to estimate completion probability and expected delivery date for each milestone - **Goal Conflict Detection** — Identifies when two stated goals contradict each other (e.g., maximum realism + minimum generation time) and proposes structured resolutions ### Intelligent Task Orchestration Once the goal is decomposed, the intelligent task orchestrator decides which tasks to run next, in what order, and on which agents: - **LLM-Driven Task Selection** — Replaces simple priority queues with an LLM that evaluates current project state and selects the next most valuable task - **Agent Bidding System** — Agents propose capability scores for available tasks and the best-matched agent is assigned; task specialization emerges dynamically - **Dynamic Role Assignment** — Roles are assigned based on task requirements analysis rather than static configuration; an agent can take different roles across different productions - **Critical Path Analysis** — Identifies the sequence of dependent tasks that determines the earliest possible completion date and optimizes it continuously - **Task Decomposition Triggers** — Automatically splits tasks that are too complex for reliable single-agent execution based on complexity estimates - **Parallel Execution Optimizer** — Identifies tasks that can run simultaneously and schedules them in parallel to minimize total wall-clock time - **Task Rollback** — When a failed task breaks a dependency chain, the system rolls back completed downstream tasks and retries from a safe checkpoint ### Autonomous Decision-Making Framework Yemaya's autonomous system must make creative decisions continuously. These decisions are documented, scored for confidence, and available for human review at any time: - **Creative Decision Framework** — AI makes style, tone, and creative direction decisions using multi-criteria decision analysis (MCDA); documents rationale for every decision for human review - **Decision Confidence Scoring** — Every autonomous decision carries a confidence score; low-confidence decisions automatically escalate to human review before execution - **Decision Audit Trail** — Complete record of every decision made, the reasoning behind it, alternatives considered, and the observed outcome for post-hoc analysis and learning - **Decision Learning** — The system updates its decision priors over time based on which decision patterns produce better production outcomes ### Autonomous Creative Direction System - **AI Creative Director Agent** — A dedicated agent with authority over visual consistency, narrative consistency, tone, pacing, and thematic coherence across the entire production - **Style Guide Generation and Enforcement** — Automatically generates a style guide from initial creative direction inputs and validates all generated content against it throughout production, rejecting outputs that deviate - **Visual Consistency Validation** — Uses CLIP embeddings (a neural network that represents images and text in a shared vector space, enabling semantic visual comparison) to verify that all generated visual assets are stylistically coherent - **Narrative Consistency Validation** — Semantic similarity checking ensures characters, plot points, and world rules remain consistent across all generated content - **Character Voice Consistency** — Each character has a defined voice and speech pattern; all generated dialogue is validated for authenticity to that voice before entering the production ### Failure Recovery and Dynamic Replanning Long autonomous productions will encounter failures. The recovery system classifies each failure and responds with the least-disruptive corrective action that keeps the production moving: - **Failure Classification** — Distinguishes recoverable failures (retriable provider timeout) from fatal failures (contradiction in core world rules requiring human input) and responds appropriately to each category - **Cascading Failure Prevention** — Detects when a failed task will cascade to block downstream tasks and proactively isolates the failure before it spreads - **Dynamic Replanning** — When the current plan becomes blocked, generates an alternative plan that achieves the same goals via different means without requiring human intervention - **Scope Reduction Optimizer** — When a goal cannot be fully achieved, identifies which elements to defer or simplify to deliver the highest-value subset on schedule (graceful degradation) - **Checkpoint-Based Recovery** — Production state is checkpointed at regular intervals so recovery after failure starts from a recent known-good state rather than the beginning --- ## Canon and Style Enforcement ### Canon Management (`@yemaya/canon-enforcement`) In long-running productions — especially franchises with multiple titles — creative facts accumulate and can contradict each other. The canon enforcement system maintains a typed registry of established truths and validates generated content against them: - **Canon Entry Registry** — Register canon facts (established truths about the world, characters, or plot) with a tier classification (`primary`, `secondary`, `tertiary`, `non-canon`) and the source document that established the fact; entries may also be scoped by media (`game`, `tv`, `marketing`, `shared`) - **Canon Status Workflow** — Each fact carries a status of `verified`, `pending`, `conflict`, or `retconned` - **Conflict Detection** — Automatically detects when two registered canon entries contradict each other and surfaces the conflict for resolution before it propagates into generated content - **Versioned Canon** — Each canon entry carries a numeric version so prior productions can be reproduced against their original canon state - **Style Rule Enforcement** — Register visual and narrative style rules (color palette constraints, vocabulary restrictions, tone requirements) tied to a style domain (`visual`, `writing`, `dialogue`, `audio`, `ui`, `animation`, `cinematography`) - **Consistency Scoring** — Character and style consistency is graded on a `perfect` / `high` / `moderate` / `low` / `inconsistent` scale, constraining downstream generation against established appearance, speech, and behavioral rules --- ## Autonomous Quality Assurance ### AAA Quality Benchmark System The quality benchmark system maintains objective standards based on AAA game and film production benchmarks. Rather than relying on subjective human review alone, these automated checks flag quality regressions as soon as they occur: - **Visual Fidelity Scoring** — Automated scoring using perceptual quality metrics; LPIPS (Learned Perceptual Image Patch Similarity — a metric that measures image similarity as humans perceive it) and FID (Fréchet Inception Distance — a measure of how similar the distribution of generated images is to a reference distribution) against AAA reference benchmarks - **Polygon Count and Topology Validation** — Verifies 3D assets meet polygon budget and topology standards for the target engine and platform - **Audio Quality Analysis** — Measures dynamic range, clarity, noise floor, and stereo imaging of generated audio against professional broadcast standards - **Narrative Quality Analysis** — Evaluates coherence, engagement potential, and pacing of generated narrative content against reference works - **AAA Readiness Score** — Synthesizes all quality dimensions into a single readiness score for stakeholder communication ### Automated Visual Quality Validation - **AI Anomaly Detection** — Detects visual artifacts, texture seams, lighting inconsistencies, and anatomical errors in generated visual assets automatically - **Color Grading Consistency** — Validates that color grading is consistent across scenes and matches the established visual style of the production - **Screenshot Regression Testing** — Compares rendered frames against golden baselines with configurable perceptual diff tolerances to catch visual regressions between pipeline runs - **Visual Polish Suggestions** — Automatically identifies areas where additional polish would most improve perceived quality and queues targeted improvement tasks ### Automated Narrative Quality Validation - **Plot Hole Detection** — Identifies narrative contradictions, unresolved story threads, and logical impossibilities in generated story content - **Character Arc Consistency** — Validates that character growth across the narrative follows logical trajectories given their established traits and motivations - **Cliche and Trope Detection** — Identifies overused story elements with an originality score and suggests more distinctive alternatives - **Tension Curve Analysis** — Plots the emotional tension across the narrative arc and identifies sections where pacing falls flat or peaks too early --- ## Production Verification and Sign-Off ### Completeness and Benchmark Verification (`@yemaya/production-verification`) The production verification library provides systematic pre-delivery checks that confirm a production meets both completeness requirements and quality standards before sign-off. These checks run as a structured gate before any deliverable leaves the platform: - **Asset Completeness Checker** — Verifies that every required asset category (characters, environments, props, audio, UI, cinematics) has been delivered at the required completeness percentage; assets in Partial or Missing state block sign-off - **AAA Benchmark Validation** — Runs every deliverable asset against defined AAA benchmark thresholds for its category; a pass/fail result with a numeric score is recorded for each asset - **Coherence Validation** — Checks that all assets from the same production are visually and narratively coherent with each other; flags outlier assets that are stylistically inconsistent with the production as a whole - **Multi-Stage Sign-Off** — Structured sign-off workflow where different stakeholders (art director, producer, QA lead) must each approve their relevant deliverable categories; a production cannot advance to delivery until all required sign-offs are recorded - **Sign-Off Reports** — Generates a formal sign-off report document summarizing the production's completeness status, benchmark scores, coherence results, and signatory list for legal and archival purposes --- ## Human Override and Safety Controls ### Override and Emergency Stop System (`@yemaya/human-override`) The human override system provides absolute human authority over all autonomous production activity at any point in the pipeline. No matter how far an autonomous pipeline has progressed, a human can pause, redirect, or stop it: - **Emergency Stop** — A single action immediately halts all autonomous agents, suspends all in-progress generation jobs, and freezes the production state; the reason for the stop is recorded with the identity of who triggered it - **Selective Override** — Apply an override to a specific pipeline step, agent, or asset category without stopping the entire production; a director can pause the character design stage while allowing audio generation to continue - **Escalation Triggers** — Define conditions under which the system should automatically escalate to a human before proceeding; escalation categories are quality failure, budget overrun, timeline slip, repeated failure, creative deadlock, ethical concern, legal compliance, security concern, and system health - **Checkpoint System** — Define named checkpoints in the production pipeline where autonomous execution pauses and waits for human review before continuing, regardless of the current operating mode - **Recovery Plans** — When an emergency stop is resolved, attach a recovery plan (what was changed, what needs to be re-run) and resume from the checkpoint preceding the stop rather than from the beginning - **Override Audit Trail** — Every override applied, modified, and lifted is recorded with full context; provides a human decision log alongside the autonomous decision log for complete accountability --- ## Self-Improving Production System ### Production Outcome Learning (`@yemaya/self-improvement`) The self-improvement library enables the production platform to learn from each completed production and improve agent performance on subsequent ones: - **Production Outcome Tracking** — Record the outcome of every completed production task: success, failure, partial success, timeout, and cancellation; outcomes are linked to the agent, task type, and quality parameters used - **Agent Performance Analytics** — Aggregate per-agent performance metrics (success rate, average quality score, time-to-complete) across productions to identify which agents perform best on which task types - **Quality Trend Analysis** — Track quality score trends over time per asset category; rising trends confirm that parameter improvements are working, falling trends trigger automatic investigation and re-calibration - **Transferable Pattern Extraction** — Identify production strategies that worked well and codify them as transferable patterns that can be applied to future productions of the same type or genre - **Lesson Registry** — Maintain a searchable registry of lessons learned from past productions; new production setups query the registry to avoid repeating known failure modes and to inherit proven techniques - **Improvement Priority Scoring** — Score which improvements would deliver the greatest quality gain per unit of compute cost; prioritizes improvement effort automatically --- ## Real-Time Collaboration ### Multi-User Presence (`@yemaya/collaboration`) - **User Presence** — See who is currently active in the project, what document or asset they are viewing, and their current activity state in real time - **Activity States** — Idle, active, typing, selecting, dragging, scrolling detected automatically and shown to all collaborators - **Focus Context** — See exactly which document, asset, scene, or timeline each collaborator is focused on at any moment ### CRDT-Based Document Sync Conflict-free collaborative editing is powered by Yjs CRDTs (Conflict-free Replicated Data Types — data structures that enable concurrent edits by multiple users with mathematically guaranteed conflict-free resolution, eliminating merge conflicts entirely): - **Real-Time Co-Editing** — Multiple users edit the same document simultaneously with automatic conflict resolution; no merge conflicts, no lost work - **CRDT Text** — Collaborative text editing with character-level granularity supporting any number of simultaneous authors - **CRDT Arrays and Maps** — Collaborative data structures for complex document types beyond plain text (scene graphs, asset metadata, pipeline configurations) - **Offline Support** — Continue editing documents while offline; changes synchronize and merge automatically when the connection is restored - **Selective Sync** — Subscribe to specific regions of a document to reduce bandwidth when working with very large asset files ### Cursor and Selection Sharing - **Remote Cursors** — See other users' cursor positions in real time with color-coded identifiers so you always know exactly who is editing where - **Selection Sharing** — See what text or regions other users have selected; prevents two people from unknowingly editing the same passage concurrently - **Typing Indicators** — Real-time indicators showing when collaborators are actively editing so you know to wait before making adjacent changes - **Name Tags** — User name tags displayed at cursor positions for identification in busy multi-author documents ### Voice and Video Communication - **Voice Calls** — Peer-to-peer voice calls within the studio environment without leaving the production interface - **Video Calls** — WebRTC-based video conferencing for remote teams with multiple simultaneous participants - **Screen Sharing** — Share your screen during production review sessions so remote collaborators see exactly what you see - **Call Recording** — Record voice/video sessions for review, documentation, and asynchronous playback by team members in different time zones ### Advanced Sync Features - **Undo/Redo Manager** — Collaborative undo/redo with per-user undo stacks and scoped operations that respect other users' concurrent changes - **Binary Protocol Optimizer** — Optimized binary sync protocol with configurable compression algorithms for high-performance real-time collaboration on large projects with limited bandwidth - **Bandwidth Management** — Adaptive sync that reduces update frequency and payload size when bandwidth is constrained, with configurable limits per session ### External Collaboration Tool Integrations Yemaya connects to popular external review and collaboration services so teams are not forced to leave their existing workflows. These integrations complement the built-in real-time features rather than replace them: | Tool | Purpose | | -------- | ----------------------------------------------------------------------- | | Evercast | Real-time video collaboration for color grading and editing sessions | | Frame.io | Video review and approval workflows with frame-accurate comments | | Louper | VFX review and feedback with annotation tools | | SyncDNA | Audio/video synchronization for remote review sessions | | ShotGrid | Autodesk ShotGrid integration for VFX pipeline management | | Miro | Visual collaboration boards for pre-production planning and pitch decks | | Notion | Documentation and knowledge management linked to project context | | Slack | Team messaging and event notification integration | | Discord | Community and team communication server integration | --- ## Audio-Video Synchronization ### AV Sync Engine (`@yemaya/av-sync`) The `av-sync` library provides precision audio-video synchronization for productions with multiple simultaneously captured streams. This is especially important for remote filmmaking scenarios where actors capture from different devices and rooms, each with its own clock: - **Sync Session Management** — Create and manage named synchronization sessions that associate multiple audio and video tracks captured across different devices, rooms, or time zones - **Drift Detection and Correction** — Continuously measures the drift (gradual timing divergence) between audio and video tracks and applies frame-accurate correction to keep tracks aligned; common in distributed capture scenarios where different devices have different clock rates - **Waveform Analysis** — Analyzes audio waveforms to find common transients (sharp audio peaks like claps, dialogue starts, or music beats) across tracks and uses them as automatic sync points - **Common Timebase Synchronization** — Converts all tracks to a common timebase (the fundamental unit of time measurement for a video project, e.g., 24fps film or 29.97fps broadcast NTSC) eliminating frame-rate conversion artifacts - **Sync Presets** — Pre-configured sync profiles for common scenarios (cinema 24fps, broadcast 29.97fps, web 60fps, high-speed 120fps) with all timebase and drift correction parameters pre-tuned --- ## Production Management ### Scheduling - **Production Schedules** — Create multi-phase production schedules with milestones, deadlines, and task dependencies visualized in Gantt-style views - **AI-Optimized Scheduling** — AI analyzes task dependencies, team availability, and historical velocity to suggest optimal task ordering and resource allocation - **Schedule Sharing** — Share production schedules with team members and external stakeholders at configurable permission levels - **Call Sheet Generation** — Automated call sheet creation for film and commercial production including crew assignments, location details, and shooting order ### Budget Management (`@yemaya/budget-management`) - **Budget Creation** — Create project budgets with line items organized by department and category using industry-standard production budget structures - **Multi-Currency Support** — Budget tracking in multiple currencies with live exchange rate conversion; essential for international co-productions - **Budget Templates** — Pre-built templates for each project type (film, game, commercial) with appropriate default line-item structures - **Granular Line-Item Tracking** — Track spending at the individual line-item level with actual-vs-budgeted comparison at every level of the hierarchy - **AI Cost Prediction** — AI-powered cost prediction based on historical project data and current scope; surfaces cost risk before it materializes - **Variance Tracking** — Track budget variance (planned vs. actual) across all categories with automated alerts when variance exceeds configurable thresholds - **Autonomous Resource Management** — Phase 26 budget management integrates with the autonomous pipeline to enforce compute budgets in real time; stops generation jobs that would exceed approved spending limits and reroutes them to more cost-efficient providers ### Crew Management - **Crew Database** — Searchable database of crew members with roles, skills, availability calendars, and negotiated rate cards - **Crew Assignment** — Assign crew members to specific production days, roles, and tasks with automatic conflict detection for double-bookings - **Availability Management** — Track crew availability and detect scheduling conflicts automatically before they cause production delays ### Location Management - **Location Database** — Digital location library with photos, GPS coordinates, availability windows, permit requirements, and logistics notes - **Virtual Location Scouting** — Browse and scout locations virtually using reference imagery, 3D scans, and satellite data without physical travel --- ## Gameplay and NPC Systems ### Gameplay Design - **Mechanics Documentation** — Define and document game mechanics, rules, and systems in structured formats that feed directly into engine integration workflows - **AI Balance Analysis** — AI-assisted game balance analysis that evaluates player progression curves, economy health, and difficulty fairness with tuning recommendations ### NPC Systems - **Behavior Trees** — Create and manage NPC behavior trees with visual node editors; define patrol patterns, combat responses, dialogue triggers, and complex multi-step behaviors - **AI Dialogue Generation** — Generate contextually appropriate NPC dialogue using AI with full awareness of character personality, faction allegiance, current game state, and player history - **NPC Personality Profiles** — Define detailed NPC personalities (using the Big Five personality model or custom trait systems) that influence dialogue, behavior, and reactions consistently across the entire production --- ## Virtual and Extended Reality ### Virtual Production - **Virtual Production Stage** — Digital virtual production stage for real-time filmmaking with LED wall integration concepts; preview composited actors against virtual environments before committing to final renders - **Camera Tracking** — Virtual camera tracking for mixed-reality production setups where physical cameras drive virtual camera positions ### XR Support - **VR Previewing** — Preview assembled scenes in virtual reality directly from the studio interface using a connected VR headset - **AR Asset Placement** — Place and preview assets in augmented reality for location scouting and virtual set design --- ## Remote Filmmaking Platform Remote filmmaking is a paradigm where actors perform from their homes, drones serve as autonomous mobile cameras, and the majority of the visual environment is CGI-rendered in real time. This surface coordinates Aphrodite, Aja, Maya, Isis, Bellona, and Oya into a unified remote production system. Two libraries are implemented under the `@yemaya/remote-film-*` prefix: `@yemaya/remote-film-capture` (the home-capture, performer-tracking, relighting, drone-integration, and AI-director surface) and `@yemaya/remote-film-mannequin` (mannequin hardware, motion driving, and safety filtering). The capture, mocap, relighting, drone-orchestration, and AI-director feature groups below all ship inside `@yemaya/remote-film-capture` — they are feature areas of that one library, not separate packages. ### Remote Actor Home Capture Infrastructure (`@yemaya/remote-film-capture`) - **Home Studio Setup Wizard** — Guided setup that walks actors through optimal camera placement, lighting, and backdrop configuration using their existing household equipment; includes hardware compatibility checking - **Room Acoustics Analyzer** — Analyzes the actor's room using microphone input to detect echo, reverb, and ambient noise with concrete remediation suggestions - **Automatic Camera Calibration** — Calibrates intrinsic camera parameters, corrects lens distortion, and establishes a consistent color profile for each actor's home setup - **Network Quality Analyzer** — Measures bandwidth, latency, jitter, and packet loss with minimum threshold enforcement; warns before a session begins if the connection is insufficient - **AI Background Segmentation** — Real-time deep-learning person extraction without requiring a green screen, running at 30+ FPS at 1080p with temporal coherence to prevent edge shimmer between frames - **Hair and Fine Detail Matting** — Specialized alpha matting networks for natural hair boundary extraction that preserves fine strands and wispy edges - **Hybrid Chroma Key + AI Segmentation** — For actors with professional backdrops, combines traditional green screen chroma key with AI refinement for maximum extraction quality ### Performer Tracking and Body Analysis (`@yemaya/remote-film-capture`) - **Real-Time 2D Pose Estimation** — Body landmark tracking from webcam at 30+ FPS using MediaPipe/MoveNet; no specialized motion capture hardware required - **3D Pose Lifting** — Converts 2D landmark positions to full 3D body poses using learned depth priors; enables spatial body tracking from a single camera - **Facial Mesh Capture** — 468+ landmark facial mesh capture for expression retargeting onto digital characters - **Hand and Finger Tracking** — 21-landmark hand mesh capture for detailed gestural performance - **Gaze Estimation** — Tracks where the actor is looking for eye-line continuity in composited scenes where the actor cannot physically see their scene partners ### Virtual Relighting and Light Matching - **Real-Time Lighting Analysis** — Analyzes the actual lighting hitting the actor (direction, color temperature, intensity, hardness) from the camera feed - **Virtual Relighting** — Removes the actor's real lighting and applies the CGI scene's lighting conditions to the actor footage in real time for convincing scene integration - **Shadow Synthesis** — Generates cast shadows on the actor from CGI scene geometry (e.g., a virtual window frame casting a shadow across the actor) - **Lighting Continuity Tracking** — Ensures consistent lighting treatment across takes shot at different times of day or across different actor home setups ### Drone Cinematography Orchestration (`@yemaya/remote-film-capture`) - **Autonomous Cinematographic Flight** — Drones execute planned shot trajectories autonomously while maintaining safe flight parameters - **Multi-Drone Fleet Coordination** — Coordinate multiple drones as a camera fleet for multi-angle simultaneous capture with collision avoidance - **3D Scene Scanning** — Drones perform structured photogrammetry flights to create 3D scans of real-world locations for virtual set creation - **AI-Driven Framing Decisions** — AI cinematographer agent makes real-time framing decisions (rule of thirds, leading lines, subject tracking) ### Multi-Camera Director Console (`@yemaya/remote-film-capture`) - **Remote Director Interface** — Unified console for switching between actor feeds, drone cameras, and virtual cameras with single-frame latency - **Real-Time Composited Preview** — Director sees the final composited output (actor + CGI environment) in real time during capture, not just raw feeds - **Actor Direction Tools** — Send real-time direction notes to individual actors while the session is live without interrupting the capture flow - **Take Management** — Mark, name, and rate takes in real time for editor review ### AI Director Agent System (`@yemaya/remote-film-capture`) - **AI Director Agent** — An AI agent that can direct scenes autonomously, coordinating camera choices, actor direction cues, and scene pacing decisions without a human director present - **Multi-Modal Direction** — AI analyzes audio (dialogue delivery, emotional intensity) and video (blocking, facial expression, energy level) simultaneously to assess performance quality and decide on additional takes - **Shot List Execution** — AI works through a pre-defined shot list, instructing drones and switching between cameras to capture every planned shot - **Improvisation Response** — When actors deviate from the script, AI detects the deviation and decides whether to embrace or redirect based on production goals ### Remote-Film Mannequin (`@yemaya/remote-film-mannequin`) A physical mannequin surface for remote performance, driven by captured actor motion: - **Mannequin Hardware and Portable Design** — Hardware abstraction and portable-rig configuration for the physical mannequin - **Motion Driving** — Drives mannequin pose from streamed performer motion with a low-latency motion-streaming path - **Motion Safety Filter** — Filters incoming motion against safe articulation limits before the mannequin actuates - **Body Adjustment and Pose Presets** — Body-proportion adjustment and reusable pose presets - **Facial Display and Clothing Surface** — Facial-display rendering and clothing-surface treatment on the mannequin - **Force Feedback** — Force-feedback control for tactile performance response ### Remote Audio Capture and Spatial Integration (planned, Phase 69.9) Remote audio parity with the video capture path: calibrated home microphone capture with room-tone profiling and noise policy, synchronized multi-source audio (performer, ambient, reference), spatial/ambisonic capture where the rig supports it, and integration of remote audio into the production mix with the same timecode discipline as the camera feeds. ### Real-Time Compositing Pipeline (planned, Phase 69.5) A live compositing path that keys, color-matches, and composites remote performers into virtual sets or reference plates in real time for the director console preview — low-latency GPU keying, per-source color pipeline, and composite quality that is decision-grade even though final compositing happens in post. ### Remote Direction and Communication (planned, Phase 69.7) The human communication layer of a distributed set: multi-party intercom and talkback routing (director → performer, director → crew, private channels), cue and countdown distribution, on-screen direction annotations pushed to performer displays, and session recording of direction for continuity. --- ## Immersive VR Production Studio (Phase 73) A director's workspace in VR — the studio interface rebuilt as an immersive spatial environment where a director orchestrates AI-driven game and film production from inside the work. Yemaya owns the studio shell (planned on the Yemaya side); Maya's `libs/maya/vr-studio` and `vr-studio-web` packages carry the world-runtime side. Isis supplies generation, Bellona engine bridges, and Psyche presence/avatars. - **XR Runtime and Platform Abstraction** — one runtime across Quest, Vision Pro, and PC VR (OpenXR), with hand/eye/controller input normalized into studio interactions. - **Spatial UI Framework** — 3D panels, volumetric viewports, and gaze/hand interaction primitives the rest of the studio composes. - **Director's Workspace Shell** — the spatial equivalent of studio-web: project spaces arranged around the director, with semantic zoom from franchise → production → scene → shot → asset. - **AI Agent Orchestration in VR** — the agent fleet visualized and directed spatially: agents report progress on boards, proposals appear as previews in the room, and approval/steering happens by direct manipulation. - **Virtual Camera and Cinematography** — physical camera operation in VR (handheld virtual cameras, dolly/crane rigs, lens/DOF control) producing real camera moves for previz and final rendering. - **Multi-User Collaboration** — several creators in the same spatial session with presence avatars (Psyche), role-scoped tools, and shared review. - **Spatial Timeline and Sequencer** — the production timeline as a walkable, editable spatial object; scene beats and shot cards manipulated by hand. - **Per-Discipline Toolsets** — VR-native authoring surfaces for worldbuilding review, set dressing, lighting, animation blocking, audio placement, and edit review, each delegating to the owning domain's pipeline. --- ## Post-Production AI (planned) Phase 70 closes the gap between Yemaya and the AI post-production capabilities of studios like ILM, Pixar, Weta, and Netflix. The centerpiece is `@isis/post-production-ai` — a per-production AI model training system. ### Per-Production Model Training - **Dailies Ingestion Pipeline** — Ingests production dailies in all professional camera formats (ProRes, ARRIRAW, R3D, BRAW, CinemaDNG, XAVC, MXF) and extracts per-frame cinematic metadata for model training - **Cinematic Analysis** — Automatically classifies focal length, camera movement, depth of field, camera angle, shot type (wide through extreme close-up), and lighting characteristics from ingested dailies - **Production Visual Vocabulary** — Trains a lightweight AI model on a production's own dailies to learn that production's specific visual signature — its lighting style, lens choices, color palette, and editorial rhythm - **Actor Appearance Model** — Learns actor-specific appearance characteristics for consistency in AI-generated coverage shots and pickup shots ### AI Relighting Engine - **Inverse Rendering** — Estimates scene geometry, materials, and lighting from existing footage; enables modification of lighting without re-shooting - **Light Source Addition** — Adds synthetic lights to already-shot footage in a physically plausible way - **Time-of-Day Modification** — Converts day footage to night or golden hour without re-shooting, matching the production's established visual vocabulary - **Shot-to-Shot Lighting Match** — Automatically matches the lighting of one shot to another for editorial continuity ### AI Shot Reframing and Coverage Synthesis - **AI Shot Reframing** — Extracts different compositions from existing footage (e.g., produces a close-up from a wide shot) without a pickup shoot - **Virtual Camera Repositioning** — Synthesizes a new camera angle from existing footage using the per-production model; creates coverage that was never physically shot --- ## Video Generation ### Hunyuan / WAN Video Generation (`@yemaya/video-generation`) The `video-generation` library integrates Hunyuan and WAN (Wan-Video — a state-of-the-art open-source video generation model family) into the Yemaya production pipeline: - **Text-to-Video Generation** — Generate cinematic video clips from detailed text descriptions; control duration, resolution, frame rate, motion intensity, and aspect ratio - **Image-to-Video Animation** — Animate a static reference image into a video clip with configurable motion type, speed, and camera movement style - **Video Continuation** — Extend an existing video clip by generating seamless additional frames that maintain scene continuity, camera motion, and subject consistency - **Style-Conditioned Generation** — Apply a reference video's cinematic style (lighting, color grade, motion feel) to newly generated content - **GPU Worker Integration** — Video generation jobs are dispatched to dedicated GPU workers via the Isis generation pipeline; Yemaya tracks job status and ingests completed clips directly into the project asset library --- ## ComfyUI Integration ### Workflow Execution (`@yemaya/comfyui-integration`) ComfyUI is a node-based visual workflow editor for AI generation that supports complex multi-step generation graphs. The `comfyui-integration` library makes these workflows executable from within Yemaya's production pipelines without requiring a running ComfyUI interface: - **Workflow Core** — Execute ComfyUI workflow JSON definitions programmatically from Yemaya; the library handles serialization, submission, and result retrieval automatically - **Node Graph Validation** — Validate a ComfyUI workflow graph before execution to catch missing inputs, type mismatches, and disconnected nodes; prevents wasted GPU time on invalid workflows - **Workflow Import** — Import existing ComfyUI workflow JSON files into the Yemaya workflow registry for versioning, sharing, and pipeline integration - **RunComfy Serverless Execution** — Execute ComfyUI workflows via RunComfy (a serverless ComfyUI hosting service) for on-demand GPU access without managing infrastructure; useful for burst capacity during production peaks - **Execution Monitoring** — Track ComfyUI execution progress in real time with node-level progress reporting; identify which node in a complex workflow is currently executing --- ## TTS Integration ### Text-to-Speech for Production (`@yemaya/tts-integration`) The `tts-integration` library provides text-to-speech capabilities tailored for production contexts: narration, dialogue scratch tracks, accessibility audio, and automated voiceover for animatics: - **TTS Core Engine** — Synthesize speech from text with configurable voice, speed, pitch, and emotional inflection; supports multiple TTS providers (ElevenLabs, AWS Polly, Azure Cognitive Services) with automatic fallback - **Production Voice Profiles** — Define named voice profiles for characters and narrators with consistent voice identity across all generated lines; a voice profile stores provider choice, voice ID, and all synthesis parameters so a character always sounds the same across the production - **Batch Line Generation** — Generate hundreds of dialogue lines from a script in a single batch job; lines are organized by speaker and scene for easy integration into the editorial timeline - **Pronunciation Dictionary** — Define custom pronunciation rules for non-standard terms (invented place names, technical jargon, character names with unconventional spelling) so generated speech is consistent and correct - **Audio Export** — Export generated audio in WAV, MP3, OGG, and FLAC formats at configurable sample rates and bit depths; automatically linked to the corresponding script line in the project timeline --- ## Style Transfer ### Visual Style Transfer and Brand Management (`@yemaya/style-transfer`) The `style-transfer` library applies a reference visual style to new content, enabling consistent aesthetic treatment across an entire production. It separates style (color palette, texture character, compositional habits) from content, so a reference image's look can be applied to different subject matter: - **Style Core** — Extract a transferable style representation from one or more reference images using neural style analysis; the extracted style captures color palette, texture character, brushwork feel, and compositional habits without copying specific content - **Text-Guided Style Transfer** — Apply an extracted style to new content generated from a text prompt; the generated image inherits the reference's aesthetic while depicting new subject matter - **Image-to-Image Style Transfer** — Apply a reference style to an existing image; useful for making disparate assets (photos, renders, illustrations) visually coherent for a production - **Brand Manager** — Define a brand as a collection of visual rules (primary and secondary color palettes, typography hierarchy, logo placement zones, forbidden color combinations, required style elements); the brand manager validates generated content against brand rules and reports violations with specific corrective suggestions - **Style Consistency Scoring** — Compute a numeric style consistency score between a batch of assets; a low score surfaces assets that are stylistic outliers and need re-generation or manual adjustment - **Style Library** — Maintain a searchable library of named, reusable style definitions that can be applied across multiple productions and shared within an organization --- ## Rendering Pipelines ### Content Rendering and Visual Compositing (`@yemaya/rendering-pipelines`) The `rendering-pipelines` library provides high-level rendering orchestration that coordinates multiple rendering systems into a unified output pipeline. Rather than exposing engine-specific details, it provides a single abstraction that routes work to the appropriate renderer: - **Render Core** — Define and execute rendering jobs that combine scene data, asset references, lighting parameters, and camera configurations into a submitted render; the render core abstracts over the underlying rendering engine (Blender, Unreal, real-time web renderer) - **Content Renderer** — Specialized renderer for 2D content (images, motion graphics, text-over-video) that operates without a full 3D engine; used for lower-budget deliverables and rapid prototyping - **Visual Compositor** — Composite multiple rendered layers (background, foreground, characters, VFX, titles, color grade) into a final output frame; supports alpha channel compositing, blend modes, and temporal effects - **Export Pipeline** — Convert composited renders into delivery formats (ProRes, H.264/H.265, WebM, image sequences) at configurable quality levels; handles frame-rate conversion, color space transformation, and codec parameter tuning - **Render Factory** — Factory functions for creating pre-configured render setups for common scenarios (web delivery, theatrical DCP, broadcast HDR, social media vertical) without requiring manual parameter configuration --- ## Podcast Generator ### Educational Podcast Episode Generation (`@yemaya/podcast-generator`) The `podcast-generator` library enables structured, production-quality podcast episode creation from source material — particularly valuable for educational productions, game lore explainers, and behind-the-scenes series: - **Episode Core** — Define episode structure with segments (intro, main topic, guest interview, summary, outro), speaker assignments, target duration, and tone (educational, narrative, conversational, interview) - **Script Generation from Source** — Provide reference documents, lore compendiums, or topic briefs; AI generates a structured episode script with natural speaker transitions, clear explanations, and a defined narrative arc - **Multi-Speaker Production** — Assign different TTS voices or human speaker slots to different segment roles; the generator produces a properly attributed multi-track audio session - **Chapter Markers** — Automatically generate chapter markers for podcast platforms (Apple Podcasts, Spotify) based on episode segment boundaries - **Audio Post-Processing** — Apply EQ, compression, and normalization to generated audio segments to produce broadcast-quality masters consistent with professional podcast production standards --- ## Presentation Compiler ### Slide Presentation Compilation (`@yemaya/presentation-compiler`) The `presentation-compiler` library generates professional slide presentations from production data, research summaries, pitch decks, and structured outlines: - **Slide Builder** — Programmatically define presentations with typed slide elements: headings, body text, images, bullet lists, data tables, video embeds, and code blocks; every element has typed dimensions, alignment, and animation properties - **Theme Engine** — Apply named visual themes that control fonts, color palette, background treatment, and spacing globally; themes can be derived from the production's brand manager for automatic brand consistency - **Layout System** — Pre-defined and custom slide layouts that constrain where content elements can be placed; prevents poorly laid-out slides from auto-generated content - **Transition and Animation** — Configure slide transitions (fade, push, wipe, zoom) and element entrance/exit animations with precise timing and easing functions - **Speaker Notes** — Attach presenter notes to each slide with full formatting support; notes export alongside the presentation for presenter view - **Export Formats** — Export to PPTX (PowerPoint), PDF, HTML slide deck, and image sequence; different formats serve different presentation contexts from boardrooms to web embedding --- ## D3 Visualizations ### Production Analytics Charts (`@yemaya/d3-visualizations`) The `d3-visualizations` library provides D3.js-based interactive charts and visualizations for production analytics, pipeline dashboards, and data-driven production documents: - **Chart Engine** — Declarative chart definition API supporting bar charts, line charts, area charts, scatter plots, pie/donut charts, heatmaps, and network graphs; all charts are interactive (hover, click, zoom, filter) and responsive to container dimensions - **Series Management** — Define multiple data series on a single chart with independent scales, colors, and interaction behaviors; useful for comparing budget vs. actuals, planned vs. completed tasks, or multi-agent performance metrics on a single canvas - **Axis and Legend Control** — Fully configurable axes (linear, logarithmic, time, ordinal scales) with custom tick formatting, grid lines, and axis labels; legends are automatically generated from series definitions with configurable placement - **Theme System** — Named visual themes that inherit from the production's brand colors; all charts in a production report automatically match the visual language of pitch materials - **Animation System** — Smooth transitions when data updates (new ingestion job completes, pipeline step finishes); charts animate from old values to new values using D3's interpolation rather than snapping --- ## Diagram Renderer ### Technical Diagram Generation (`@yemaya/diagram-renderer`) The `diagram-renderer` library generates technical diagrams for production documentation, architecture overviews, pipeline graphs, and narrative structure maps: - **Node-Edge Graph Renderer** — Render directed and undirected graphs with configurable node shapes, edge styles (solid, dashed, curved, orthogonal), and automatic layout algorithms (Dagre hierarchical, force-directed, circular) - **Layout Engine** — Multiple automatic layout strategies; hierarchical layout is used for pipeline dependency diagrams, force-directed for faction relationship maps, and circular for timeline event chains - **Edge Routing** — Intelligent edge routing that avoids node overlap and minimizes crossing; produces readable diagrams even for dense graphs with many inter-node relationships - **Export Formats** — Export diagrams as SVG (lossless, web-ready), PNG (raster at configurable DPI), and PDF; SVG exports retain interactivity for web embedding --- ## Plugin Marketplace and SDKs ### Unified Three-Vertical Marketplace (`@yemaya/marketplace`) The marketplace is a single unified surface covering three distinct verticals. Creators can monetize their work across all three, and buyers can search across all three simultaneously. #### Asset Marketplace - **Browse and Purchase** — 3D models, textures, audio assets, and creative resources with high-quality preview renders and technical specifications - **Flexible Licensing** — Personal, commercial, and enterprise license types with usage tracking and license verification built into the asset pipeline - **Seller Analytics** — Revenue tracking, payout management, and sales analytics for asset creators with monthly payout processing - **Community Reviews** — Ratings and reviews with verified-purchase flags; reported reviews moderated by the platform team - **Full E-Commerce** — Shopping cart, payment processing, refund management, and transaction history for asset purchases #### Plugin Marketplace - **Plugin Categories** — Tool plugins, agent plugins, importer/exporter plugins, renderer plugins, and integration plugins for third-party services - **Sandboxed Execution** — Plugins run in isolated sandbox environments with declared permission sets; plugins cannot access data outside their granted scope - **Dependency Management** — Automatic dependency resolution and version compatibility checking; prevents incompatible plugin combinations - **Revenue Sharing** — Plugin developers earn configurable revenue share from marketplace sales with transparent payout reporting #### AI Model Marketplace - **Model Discovery** — Browse and download AI model checkpoints, LoRA adapters (Low-Rank Adaptation — lightweight fine-tuning adapters that shift model behavior for specific styles or characters), embeddings, and VAEs (Variational Autoencoders — the encoding/decoding components of diffusion models) - **Standardized Model Cards** — Documentation with architecture description, training data sources, benchmark metrics, usage instructions, and known limitations for every listed model - **Model Versioning** — Track model versions with changelogs and comparative benchmark metrics so users make informed upgrade decisions - **Fine-Tune Sharing** — Upload and share models fine-tuned on custom datasets; community-contributed models broaden the available style palette - **Interactive Demos** — Test models directly in the browser with sample prompts before downloading - **Verified Publishers** — Publisher verification system with full model portfolio display and reputation metrics ### Cross-Marketplace Search Search across all three marketplaces simultaneously with faceted filtering by vertical, price type, rating, tags, and publisher. Trending and featured content is surfaced across all verticals for discovery. ### Developer SDKs All three SDKs expose the full Yemaya API surface with typed method signatures, typed error hierarchies, automatic retry with exponential backoff, and configurable authentication (API key or JWT): | SDK | Language | Use Cases | | -------------------- | ---------- | --------------------------------------------------- | | `@yemaya/sdk` | TypeScript | Web apps, Node.js automation, CI/CD scripts | | `@yemaya/sdk-cpp` | C++ | Engine plugins for Unreal Engine and custom engines | | `@yemaya/sdk-python` | Python | ML pipelines, DCC tool integration, data science | --- ## Organizations and Enterprise ### Organization Management (`@yemaya/organizations`) - **Organization Creation** — Create organizations with name, description, logo, website URL, and domain verification for SSO configuration - **Team Management** — Create named teams within organizations with specific role sets and project access scoping - **Member Management** — Invite, remove, and update organization members with role assignments and access level management - **Organization Settings** — Centralized billing configuration, security policies, default AI generation quotas, and global compliance settings ### Enterprise Plans Feature availability is gated by plan tier. The table below shows what each plan unlocks across the key platform dimensions: | Feature | Free | Starter | Professional | Enterprise | | ----------------------- | ------ | ------- | ------------ | ---------- | | Projects | 3 | 10 | Unlimited | Unlimited | | Team members | 5 | 25 | 100 | Unlimited | | Storage | 5 GB | 50 GB | 500 GB | Custom | | GPU rendering hours | 0 | 10/mo | 100/mo | Custom | | AI generation credits | 100 | 1,000 | 10,000 | Custom | | Engine integrations | 1 | 3 | All | All | | Real-time collaboration | Basic | Full | Full | Full | | Marketplace access | Browse | Full | Full | Full | | Plugin development | No | Yes | Yes | Yes | | API access | No | Read | Full | Full | | GDPR compliance | No | No | Yes | Yes | | SOC2 audit logging | No | No | Yes | Yes | | Custom SSO | No | No | No | Yes | | Dedicated support | No | No | Priority | Dedicated | ### Role-Based Access Control (RBAC) (`@yemaya/rbac`) RBAC has three levels that compose to determine effective permissions. A user's access on any resource is the intersection of their organization role, their team membership, and their project-level role: - **Organization Level** — Owner (full admin), Admin (manage members and billing), Member (standard access), Viewer (read-only) - **Team Level** — Team-specific roles and project access scoping; teams can be granted access to specific projects at specific permission levels - **Project Level** — Owner, Admin, Editor, Contributor, Viewer with fine-grained capability flags per resource type ### Compliance and Audit - **GDPR Compliance** — Personal data export, right-to-erasure deletion workflows, and consent management per EU data protection requirements - **SOC2 Audit Logging** — Complete, tamper-evident audit trail of all user actions for compliance reporting; includes who, what, when, and from where - **API Key Management** — Scoped API keys with configurable permission sets for programmatic access; keys can be rotated and revoked independently ### Enterprise Analytics - **Usage Analytics** — Track platform usage across the organization with breakdowns by team, project, and user - **Resource Consumption** — Monitor GPU rendering hours, storage usage, and API call volumes against organizational quotas - **Cost Attribution** — Attribute AI generation and compute costs to specific teams, projects, or users for accurate internal chargeback --- ## Content Safety and Compliance (`@yemaya/safety`) All AI-generated and user-uploaded content passes through safety checks before it becomes visible to the production team. Safety is enforced as part of the asset pipeline, not as an optional step: - **Content Filtering** — Automatic detection and filtering of unsafe or inappropriate generated content before it enters the asset library; runs on every AI-generated asset without requiring manual triggering - **Safety Scoring** — Every AI-generated asset receives a multi-dimensional safety score (violence, sexual content, hate speech, self-harm) before becoming available to the production team - **Custom Safety Policies** — Configure safety thresholds and category-specific policies per organization to match the studio's content standards and the production's target rating (G, PG, PG-13, R, etc.) --- ## Community and Showcase (`@yemaya/community`) - **Discussion Forums** — Structured forums for creators to share knowledge, discuss techniques, ask questions, and showcase work-in-progress - **Showcase Gallery** — Public gallery for creators to exhibit completed work from Yemaya projects; supports curated collections and featured spotlights - **Tutorial Library** — Community-created and official tutorials covering all Yemaya features and production workflows with difficulty ratings and completion tracking --- ## Platform Availability Yemaya is accessible from eight distinct surfaces spanning browser, desktop, API, CLI, and three SDK languages: | Platform | Technology | Description | | -------------- | ------------------ | ---------------------------------------------- | | Studio Web | Vite + React (PWA) | Full-featured browser-based studio | | Studio Desktop | Electron | Native desktop with local engine integration | | CLI | Commander.js | Command-line project management and automation | | REST API | Hono (30 routers) | Complete API with OpenAPI documentation | | WebSocket | WSS | Real-time collaboration and presence | | TypeScript SDK | npm package | SDK for TypeScript/JavaScript clients | | C++ SDK | Native library | SDK for engine plugin development | | Python SDK | PyPI package | SDK for ML and scripting workflows | --- ## Access Tiers The three operating modes (Human-Driven, AI-Assisted, and Fully Autonomous) affect which tasks are manual, which are AI-suggested, and which are fully automated. The table maps each major feature area to its behavior in each mode: | Feature | Human-Driven | AI-Assisted | Fully Autonomous | | ----------------------------- | -------------- | ----------- | ---------------- | | Project management | Full | Full | Full | | Asset management | Full | Full | Full | | Script and storyboard editing | Full | AI suggests | AI generates | | Character and world design | Full | AI suggests | AI generates | | Engine integration | Full | Full | Full | | Scene assembly | Manual | AI assists | AI assembles | | Rendering | Manual trigger | Auto-queue | Auto-pipeline | | Audio production | Manual | AI assists | AI generates | | Quality review | Human review | AI + human | AI auto-approve | | Canon enforcement | Manual | Automated | Automated | | Export and distribution | Manual | Scheduled | Automated | | Real-time collaboration | Full | Full | N/A | --- ## Cross-Domain Integration Yemaya delegates specialized work to other Oshun domains rather than owning every capability itself. The boundary design matters: Yemaya stores only a lightweight ID pointing into the other domain's database, never a replicated copy of the data. This keeps each domain's data model sovereign and prevents Yemaya migrations from requiring coordination with Isis, Hathor, Sophia, or Bellona. Resolution happens at query time through internal API calls or the `/v1/capabilities/{domain}` reverse proxy. Communication is asynchronous by default. Yemaya emits events when its state changes (project created, asset uploaded, member joined) and the capability domains react. The capability domains emit completion events (asset generated, world published, build completed) and Yemaya reacts by updating the project state visible to users. | Domain | Integration Purpose | Reference Field | | ----------- | ------------------------------------------------------ | ------------------------------ | | **Isis** | AI generation jobs (3D, images, video, audio, text) | `isisWorkflowIds[]` on Project | | **Hathor** | Worldbuilding (characters, locations, lore, narrative) | `hathorWorldId` on Project | | **Sophia** | Research materials and knowledge bases | `sophiaPackIds[]` on Project | | **Bellona** | Engine builds, rendering, export, optimization, sync | `bellonaAssetId` on Asset | ### Events Published Yemaya publishes 12 `yemaya.*` events (`YemayaEventTypes` in `@oshun/contracts`): `project.created`, `project.updated`, `project.archived`, `member.joined`, `member.left`, `asset.uploaded`, `asset.processed`, `asset.approved`, `asset.rejected`, `comment.created`, `comment.resolved`, and `session.joined`. ### Events Subscribed The following 6 inbound events are defined by `YEMAYA_SUBSCRIPTIONS` in `@yemaya/event-handlers`. Each handler updates the Yemaya project state in response to a completed action in a capability domain: | Event | Source | Yemaya Action | | -------------------------- | ------- | ------------------------------------------------- | | `isis.asset.generated` | Isis | Ingest generated asset into project asset library | | `isis.job.failed` | Isis | Surface failure alert to project dashboard | | `sophia.document.ingested` | Sophia | Link research document to project knowledge | | `hathor.world.published` | Hathor | Sync world state to project world design section | | `bellona.build.completed` | Bellona | Update build status in project dashboard | | `bellona.export.ready` | Bellona | Notify project team that deliverable is ready | --- ## Neith Production Toolchain Dependencies (planned — Phases 132, 135, 141, 147, 153, 158, 163) Yemaya owns creative production orchestration, collaboration, review, delivery, and autonomous studio workflows. Neith owns the sovereign workstation tools that Yemaya schedules, automates, wraps in production policy, and connects to teams. - **Audio runtime and production sound (Phase 132)**: Yemaya's production audio workflows (ADR, dailies audio, mix review, delivery loudness) run on the `@neith/audio-runtime` / `@neith/audio-graph` stack and its plugin hosts; Yemaya owns the production policy around them (session templates, review loudness targets, deliverable specs). > **Status:** planned phase backlog. The `@neith/foundry`, `@neith/anim2d-*`, > `@neith/colorist-*`, `@neith/matchmove-*`, `@neith/vse-*`, and `@neith/vp-*` > packages named below are not yet present under `libs/neith/`; the integrations > are scoped for the indicated future phases. - **VFX compositor and delivery (Phase 135)**: Yemaya integrates `@neith/foundry`, `@neith/motion`, `@neith/silhouette`, `@neith/matchmove`, and `@neith/conform` for node compositing, motion graphics, roto, planar tracking, paint/cleanup, 3D compositing, deep EXR, OCIO/ACES, editorial conform, OTIO/AAF/EDL/XML/FCPXML round trips, dailies, review, QC, IMF/DCP/ProRes/DNx/EXR output, HDR/SDR trim, captions, subtitles, archival packages, and render-farm execution. - **2D animation (Phase 141)**: Yemaya consumes `@neith/anim2d-*` scene, timing, exposure-sheet, frame-by-frame drawing, onion-skin, cleanup, ink/paint, rigged puppet, lip-sync, palette, multiplane camera, compositing, render, review, and delivery features for animation productions and story-driven video pipelines. - **Colorist suite (Phase 147)**: Yemaya uses `@neith/colorist-*` for color-managed timelines, camera RAW debayer, grade node graphs, Baselight / Nucoda-style layer stacks, ACES/OCIO, HDR wheels, qualifiers, keyers, windows, tracking, film emulation, restoration, denoise, HDR mastering, trims, metadata, control surfaces, livegrade, review, and delivery exports. - **Matchmove and camera tracking (Phase 153)**: Yemaya wraps `@neith/matchmove-*` project setup, media/camera metadata, 2D feature tracking, lens distortion, camera/object/body solve, set reconstruction, dense 3D, stabilization, retime, exports, viewer interaction, ML-assisted solve tools, and Python/Rust scripting into VFX production workflows. - **DCC video sequence editor (Phase 158)**: Yemaya consumes `@neith/vse-*` timeline, strip model, ingest, proxy, transition, effects, audio, playback, render queue, review, dailies, conform, roundtrip, and animatic tools as a sovereign editorial surface. - **Virtual production (Phase 163)**: Yemaya coordinates `@neith/vp-*` render-cluster, LED volume, camera tracking, inner/outer frustum, color, virtual scene, ICVFX, mixed-reality, operator-control, broadcast studio, capture, playback, editorial handoff, and VFX handoff features for remote and distributed productions. - **Niagara-class GPU VFX authoring (Phase 161)**: Yemaya schedules and reviews `@neith/vfx-*` node-based GPU particle/VFX authoring as a production effects surface, wrapping it in production policy (asset review, shot assignment) the same way it wraps the compositor and VSE.