Eve library

Yemaya Domain - Developer Guide

Guide

Revision 1

On this page

Yemaya is the Creative Studio domain. It serves as the central orchestration layer and user-facing platform, providing the API gateway, background workers, web studio, desktop application, and CLI. Yemaya integrates with all other domains (Isis for generation, Hathor for worldbuilding, Sophia for research, Bellona for builds) and manages users, projects, organizations, assets, collaboration, and subscriptions.


Prerequisites#

Required Software#

Software Minimum Version Purpose
Node.js >= 20.0.0 Runtime for API, workers, and CLI
pnpm >= 9.0.0 Package manager (monorepo workspace)
Docker >= 24.0 Infrastructure (PostgreSQL, Redis, MinIO)
TypeScript ^6.0.3 Type checking and compilation

For Desktop Development#

Software Purpose
Electron Desktop app framework (studio-desktop)

Project Structure#

text
apps/yemaya/
  api/               # API Gateway - RESTful + gRPC (Hono, OpenAPI, Scalar docs)
  workers/           # Background task processors (BullMQ)
  studio-web/        # React SPA for the creative studio
  studio-desktop/    # Electron desktop application
  cli/               # Command-line interface

libs/yemaya/
  sdk/               # TypeScript SDK with React hooks
  sdk-cpp/           # C++ SDK (CMake)
  sdk-python/        # Python SDK
  database/          # Prisma schema and generated client
  core/              # Core domain logic
  auth/              # Authentication library
  agents/            # AI agent orchestration
  assets/            # Asset management
  asset-library/     # Asset library browsing
  collaboration/     # Real-time collaboration
  community/         # Community features
  enterprise/        # Enterprise features
  event-handlers/    # Cross-domain event handling
  event-publisher/   # Cross-domain event publishing
  marketplace/       # Plugin marketplace
  orchestration/     # Multi-domain orchestration
  organizations/     # Organization management
  projects/          # Project management
  rbac/              # Role-based access control
  safety/            # Content safety and moderation
  ui/                # Shared UI components
  users/             # User management
  autonomous-pipelines/ # Autonomous production pipelines
  activity/          # Activity tracking

API Dependencies#

The API integrates with every domain via their client libraries:

  • @isis/client - Generation services
  • @sophia/client - Search and knowledge services
  • @hathor/client - Worldbuilding services
  • @oshun/bellona-client - Build and engine services

Local Development Setup#

1. Start Infrastructure#

Yemaya requires PostgreSQL, Redis, and MinIO (S3-compatible storage).

bash
# Start all core infrastructure
docker compose -f docker/docker-compose.dev.yml up -d

This starts:

  • PostgreSQL (port 5432) - with yemaya database
  • Redis (port 6379) - for caching and job queues
  • MinIO (ports 9000/9001) - for asset storage

2. Set Environment Variables#

Variable Default Used By
YEMAYA_DATABASE_URL postgresql://oshun:oshun_dev@localhost:5432/yemaya database
DATABASE_URL Same as above workers
REDIS_HOST localhost api, workers
REDIS_PORT 6379 api, workers
REDIS_PASSWORD (none for local) api, workers
MINIO_ENDPOINT localhost workers
MINIO_PORT 9000 workers
MINIO_ACCESS_KEY minioadmin workers
MINIO_SECRET_KEY minioadmin workers
ANTHROPIC_API_KEY (optional) AI agents
OPENAI_API_KEY (optional) AI agents
GOOGLE_API_KEY (optional) AI agents
LOG_LEVEL info all services
LOG_PRETTY true (dev) / false (prod) all services
WORKER_CONCURRENCY 5 workers
WORKER_JOB_TIMEOUT_MS 300000 (5 min) workers

3. Database Setup (Prisma)#

bash
cd libs/yemaya/database

# Generate the Prisma client
npx prisma generate

# Apply schema to local database
npx prisma db push

# Or run migrations if available
npx prisma migrate dev

4. Install Dependencies#

bash
pnpm install

Running Services#

API Gateway#

The main API service built with Hono, providing REST and gRPC endpoints with OpenAPI documentation (via Scalar).

bash
# Development (hot reload)
pnpm nx serve @yemaya/api

# Or directly
cd apps/yemaya/api
pnpm dev   # tsx watch src/index.ts

Features:

  • OpenAPI schema generation: pnpm nx openapi:generate @yemaya/api
  • OpenAPI validation: pnpm nx openapi:validate @yemaya/api
  • Code generation for SDKs: pnpm nx codegen @yemaya/api (TypeScript, Python, Go)
  • Docker build: pnpm nx docker-build @yemaya/api
  • Type checking: pnpm nx typecheck @yemaya/api

API architecture:

  • Hono routes with @hono/zod-openapi for typed endpoints
  • Swagger UI via @hono/swagger-ui
  • Scalar API reference via @scalar/hono-api-reference
  • JWT authentication via jose
  • Password hashing via bcrypt
  • Structured logging via pino
  • Redis caching via ioredis

Background Workers#

Process background tasks using BullMQ queues. Workers can be started individually or all at once.

bash
# Start all workers
pnpm nx serve @yemaya/workers

# Start specific worker types
cd apps/yemaya/workers
pnpm start:orchestration   # --worker=orchestration
pnpm start:rendering       # --worker=rendering
pnpm start:export          # --worker=export
pnpm start:pipeline        # --worker=pipeline
pnpm start:notification    # --worker=notification

Worker types:

Worker File Purpose
Agent Worker agent-worker.ts AI agent task execution
Asset Worker asset-worker.ts Asset processing and thumbnails
Media Worker media-worker.ts Media transcoding and processing
3D Generation Worker 3d-generation-worker.ts 3D model generation
Notification Worker notification-worker.ts Push/email notifications

Worker configuration is loaded from environment variables via Zod schema validation (src/config.ts). Invalid configuration causes the worker to exit with a descriptive error.

Studio Web (React SPA)#

bash
pnpm nx serve @yemaya/studio-web

Features:

  • Vite dev server
  • i18n support (English, Spanish, French)
  • Storybook stories for UI components
  • Zustand stores for state management (auth, projects, assets, AI, collaboration, themes)
  • PWA support with service worker (src/sw.ts)
  • Route-based code splitting with lazy loading (src/lib/lazy.tsx)
  • Accessibility testing (src/test/a11y.test.tsx)
  • Deep linking support (src/lib/deepLink.ts)
  • Performance monitoring (src/lib/performance.ts)

Studio Desktop (Electron)#

bash
pnpm nx serve @yemaya/studio-desktop

CLI#

bash
pnpm nx serve @yemaya/cli

Building#

Build All Yemaya Projects#

bash
pnpm nx run-many --target=build --projects=tag:scope:yemaya

Build Individual Projects#

bash
# API (tsup bundler)
pnpm nx build @yemaya/api

# Workers (tsup bundler)
pnpm nx build @yemaya/workers

# Studio Web (Vite)
pnpm nx build @yemaya/studio-web

# Desktop (Electron)
pnpm nx build @yemaya/studio-desktop

# SDK (tsup)
pnpm nx build @yemaya/sdk

# C++ SDK (CMake)
cd libs/yemaya/sdk-cpp
mkdir build && cd build
cmake ..
make

# Python SDK
cd libs/yemaya/sdk-python
pip install -e .

Docker Builds#

bash
# API Docker image
pnpm nx docker-build @yemaya/api

# Workers Docker image
pnpm nx docker-build @yemaya/workers

Testing#

Test Runner Configuration#

All Yemaya apps use Vitest via the @nx/vite:test executor.

Running Tests#

bash
# All Yemaya projects
pnpm nx run-many --target=test --projects=tag:scope:yemaya

# Individual projects
pnpm nx test @yemaya/api
pnpm nx test @yemaya/workers
pnpm nx test @yemaya/studio-web

# With coverage
cd apps/yemaya/api
pnpm test:coverage   # vitest run --coverage

# Watch mode
cd apps/yemaya/api
pnpm test:watch      # vitest

Linting & Formatting#

bash
# Lint all
pnpm nx run-many --target=lint --projects=tag:scope:yemaya

# Individual
pnpm nx lint @yemaya/api

# Format check
cd apps/yemaya/api
pnpm format:check    # prettier --check src/

# Auto-fix
cd apps/yemaya/api
pnpm lint:fix        # eslint src/ --fix
pnpm format          # prettier --write src/

Type Checking#

bash
pnpm nx typecheck @yemaya/api
pnpm nx typecheck @yemaya/workers

Database Schema#

The Yemaya database (libs/yemaya/database/prisma/schema.prisma) uses PostgreSQL and is the largest schema in the monorepo. It manages:

  • Users & Authentication - Users, accounts (OAuth), sessions, API keys, preferences
  • Organizations & Teams - Organizations, members, roles, invitations
  • Projects & Workspaces - Projects, project members, permissions
  • Asset Library - Assets with metadata, tags, thumbnails, versions
  • Activity & Notifications - Activity feed, in-app and email notifications
  • Billing & Subscriptions - Plans, subscriptions, invoices, payment methods
  • Plugin Marketplace - Plugins, versions, reviews, installations

The database URL is YEMAYA_DATABASE_URL.

Domain-specific data (generation jobs, world state, research documents, build pipelines) lives in separate domain databases accessed via domain client libraries.

Prisma Workflow#

bash
cd libs/yemaya/database

# Generate client after schema changes
npx prisma generate

# Create new migration
npx prisma migrate dev --name your_change_description

# Apply migrations in production
npx prisma migrate deploy

# View database in browser
npx prisma studio

Common Development Tasks#

Adding a New Production Pipeline#

Yemaya's autonomous pipelines (libs/yemaya/autonomous-pipelines/) orchestrate multi-step creative workflows across domains.

  1. Define the pipeline steps and their domain dependencies:

    typescript
    export const myPipeline = {
      name: 'character-sheet-pipeline',
      steps: [
        { domain: 'sophia', action: 'research-character-archetype' },
        { domain: 'isis', action: 'generate-character-art' },
        { domain: 'hathor', action: 'create-world-entity' },
        { domain: 'bellona', action: 'export-to-engine' },
      ],
    };
    
  2. Implement the pipeline in libs/yemaya/autonomous-pipelines/.

  3. Add a BullMQ queue for the pipeline in apps/yemaya/workers/src/queues.ts.

  4. Create or extend a worker in apps/yemaya/workers/src/workers/ to process the pipeline jobs.

  5. Add an API endpoint in apps/yemaya/api/ for triggering the pipeline.

  6. The worker uses domain client libraries (@isis/client, @sophia/client, @hathor/client, @oshun/bellona-client) to interact with each domain.

Adding a New API Endpoint#

  1. Create the route file in apps/yemaya/api/src/routes/:

    typescript
    import { createRoute, z } from '@hono/zod-openapi';
    import type { AppType } from '../app.js';
    
    const myRoute = createRoute({
      method: 'get',
      path: '/my-resource',
      responses: {
        200: {
          description: 'Success',
          content: {
            'application/json': {
              schema: z.object({
                /* ... */
              }),
            },
          },
        },
      },
    });
    
    export function registerMyRoutes(app: AppType): void {
      app.openapi(myRoute, async (c) => {
        // Handler implementation
        return c.json({
          /* ... */
        });
      });
    }
    
  2. Register the routes in the app setup.

  3. Run pnpm nx openapi:generate @yemaya/api to update the OpenAPI spec.

  4. Run pnpm nx openapi:validate @yemaya/api to verify the spec.

Adding a New Worker Type#

  1. Create the worker file in apps/yemaya/workers/src/workers/:

    typescript
    import { Worker, type Job } from 'bullmq';
    
    export function createMyWorker(connection: RedisConnection) {
      return new Worker(
        'my-queue',
        async (job: Job) => {
          // Process the job
        },
        { connection, concurrency: 5 }
      );
    }
    
  2. Add the queue definition in src/queues.ts.

  3. Register the worker in src/workers/index.ts.

  4. Add the worker type to the config schema in src/config.ts:

    typescript
    type: z.enum(['all', 'agent', 'media', 'asset', 'notification',
                   'generation-3d', 'my-worker']).default('all'),
    
  5. Add a start script in package.json:

    json
    "start:my-worker": "node dist/index.js --worker=my-worker"
    

Adding SDK React Hooks#

The TypeScript SDK (libs/yemaya/sdk/) includes React hooks:

  1. Add a new hook in libs/yemaya/sdk/src/hooks/:

    typescript
    import { useQuery, useMutation } from '@tanstack/react-query';
    import { useYemayaClient } from './useYemaya.js';
    
    export function useMyResource(id: string) {
      const client = useYemayaClient();
      return useQuery({
        queryKey: ['my-resource', id],
        queryFn: () => client.myResource.get(id),
      });
    }
    
  2. Export from libs/yemaya/sdk/src/react.ts.


Debugging Tips#

API Not Starting#

  • Verify PostgreSQL is running and YEMAYA_DATABASE_URL is set
  • Check Redis connectivity (required for caching)
  • Ensure Prisma client is generated
  • The API uses jose for JWT; check JWT_SECRET environment variable
  • Review Hono middleware chain for error middleware

Worker Not Processing Jobs#

  • Workers connect to Redis via REDIS_HOST/REDIS_PORT
  • Check which worker type is running (see --worker= argument)
  • Verify queue names match between API job submission and worker registration
  • Worker config validation is strict (Zod schema); check console for validation errors on startup
  • Stalled jobs are detected at WORKER_STALLED_INTERVAL (default 30s)
  • Jobs timeout at WORKER_JOB_TIMEOUT_MS (default 5 minutes)

Studio Web Issues#

  • The studio uses Zustand for state management with devtools
  • Check store middleware (src/stores/middleware.ts) for persistence issues
  • i18n locale files are in src/i18n/locales/{lang}/
  • Route prefetching is in src/lib/routePrefetch.ts
  • Service worker issues: check src/sw.ts

Cross-Domain Integration Issues#

  • The API imports domain clients (@isis/client, etc.)
  • Event handlers (@yemaya/event-handlers) process events from other domains
  • Event publishing (@yemaya/event-publisher) sends events to other domains
  • Check the event bus connection (Redis Streams)

Database Issues#

  • The Yemaya schema is large; prisma generate may take a few seconds
  • Use npx prisma studio to browse data visually
  • For migration conflicts: npx prisma migrate reset (destructive)
  • The generated Prisma client is at libs/yemaya/database/src/generated/client/

Performance#

  • API uses Redis caching via @oshun/cache
  • Workers have configurable concurrency (default 5 per type)
  • The studio web uses lazy loading and route-based code splitting
  • Performance monitoring is available via src/lib/performance.ts
  • Optimistic updates are in src/stores/optimistic.ts