Yemaya is the Creative Studio domain. It serves as the central orchestration layer and user-facing platform, providing the API gateway, background workers, web studio, desktop application, and CLI. Yemaya integrates with all other domains (Isis for generation, Hathor for worldbuilding, Sophia for research, Bellona for builds) and manages users, projects, organizations, assets, collaboration, and subscriptions.
Prerequisites#
Required Software#
| Software | Minimum Version | Purpose |
|---|---|---|
| Node.js | >= 20.0.0 | Runtime for API, workers, and CLI |
| pnpm | >= 9.0.0 | Package manager (monorepo workspace) |
| Docker | >= 24.0 | Infrastructure (PostgreSQL, Redis, MinIO) |
| TypeScript | ^6.0.3 | Type checking and compilation |
For Desktop Development#
| Software | Purpose |
|---|---|
| Electron | Desktop app framework (studio-desktop) |
Project Structure#
apps/yemaya/
api/ # API Gateway - RESTful + gRPC (Hono, OpenAPI, Scalar docs)
workers/ # Background task processors (BullMQ)
studio-web/ # React SPA for the creative studio
studio-desktop/ # Electron desktop application
cli/ # Command-line interface
libs/yemaya/
sdk/ # TypeScript SDK with React hooks
sdk-cpp/ # C++ SDK (CMake)
sdk-python/ # Python SDK
database/ # Prisma schema and generated client
core/ # Core domain logic
auth/ # Authentication library
agents/ # AI agent orchestration
assets/ # Asset management
asset-library/ # Asset library browsing
collaboration/ # Real-time collaboration
community/ # Community features
enterprise/ # Enterprise features
event-handlers/ # Cross-domain event handling
event-publisher/ # Cross-domain event publishing
marketplace/ # Plugin marketplace
orchestration/ # Multi-domain orchestration
organizations/ # Organization management
projects/ # Project management
rbac/ # Role-based access control
safety/ # Content safety and moderation
ui/ # Shared UI components
users/ # User management
autonomous-pipelines/ # Autonomous production pipelines
activity/ # Activity tracking
API Dependencies#
The API integrates with every domain via their client libraries:
@isis/client- Generation services@sophia/client- Search and knowledge services@hathor/client- Worldbuilding services@oshun/bellona-client- Build and engine services
Local Development Setup#
1. Start Infrastructure#
Yemaya requires PostgreSQL, Redis, and MinIO (S3-compatible storage).
# Start all core infrastructure
docker compose -f docker/docker-compose.dev.yml up -d
This starts:
- PostgreSQL (port 5432) - with
yemayadatabase - Redis (port 6379) - for caching and job queues
- MinIO (ports 9000/9001) - for asset storage
2. Set Environment Variables#
| Variable | Default | Used By |
|---|---|---|
YEMAYA_DATABASE_URL |
postgresql://oshun:oshun_dev@localhost:5432/yemaya |
database |
DATABASE_URL |
Same as above | workers |
REDIS_HOST |
localhost |
api, workers |
REDIS_PORT |
6379 |
api, workers |
REDIS_PASSWORD |
(none for local) | api, workers |
MINIO_ENDPOINT |
localhost |
workers |
MINIO_PORT |
9000 |
workers |
MINIO_ACCESS_KEY |
minioadmin |
workers |
MINIO_SECRET_KEY |
minioadmin |
workers |
ANTHROPIC_API_KEY |
(optional) | AI agents |
OPENAI_API_KEY |
(optional) | AI agents |
GOOGLE_API_KEY |
(optional) | AI agents |
LOG_LEVEL |
info |
all services |
LOG_PRETTY |
true (dev) / false (prod) |
all services |
WORKER_CONCURRENCY |
5 |
workers |
WORKER_JOB_TIMEOUT_MS |
300000 (5 min) |
workers |
3. Database Setup (Prisma)#
cd libs/yemaya/database
# Generate the Prisma client
npx prisma generate
# Apply schema to local database
npx prisma db push
# Or run migrations if available
npx prisma migrate dev
4. Install Dependencies#
pnpm install
Running Services#
API Gateway#
The main API service built with Hono, providing REST and gRPC endpoints with OpenAPI documentation (via Scalar).
# Development (hot reload)
pnpm nx serve @yemaya/api
# Or directly
cd apps/yemaya/api
pnpm dev # tsx watch src/index.ts
Features:
- OpenAPI schema generation:
pnpm nx openapi:generate @yemaya/api - OpenAPI validation:
pnpm nx openapi:validate @yemaya/api - Code generation for SDKs:
pnpm nx codegen @yemaya/api(TypeScript, Python, Go) - Docker build:
pnpm nx docker-build @yemaya/api - Type checking:
pnpm nx typecheck @yemaya/api
API architecture:
- Hono routes with
@hono/zod-openapifor typed endpoints - Swagger UI via
@hono/swagger-ui - Scalar API reference via
@scalar/hono-api-reference - JWT authentication via
jose - Password hashing via
bcrypt - Structured logging via
pino - Redis caching via
ioredis
Background Workers#
Process background tasks using BullMQ queues. Workers can be started individually or all at once.
# Start all workers
pnpm nx serve @yemaya/workers
# Start specific worker types
cd apps/yemaya/workers
pnpm start:orchestration # --worker=orchestration
pnpm start:rendering # --worker=rendering
pnpm start:export # --worker=export
pnpm start:pipeline # --worker=pipeline
pnpm start:notification # --worker=notification
Worker types:
| Worker | File | Purpose |
|---|---|---|
| Agent Worker | agent-worker.ts |
AI agent task execution |
| Asset Worker | asset-worker.ts |
Asset processing and thumbnails |
| Media Worker | media-worker.ts |
Media transcoding and processing |
| 3D Generation Worker | 3d-generation-worker.ts |
3D model generation |
| Notification Worker | notification-worker.ts |
Push/email notifications |
Worker configuration is loaded from environment variables via Zod schema
validation (src/config.ts). Invalid configuration causes the worker to exit
with a descriptive error.
Studio Web (React SPA)#
pnpm nx serve @yemaya/studio-web
Features:
- Vite dev server
- i18n support (English, Spanish, French)
- Storybook stories for UI components
- Zustand stores for state management (auth, projects, assets, AI, collaboration, themes)
- PWA support with service worker (
src/sw.ts) - Route-based code splitting with lazy loading (
src/lib/lazy.tsx) - Accessibility testing (
src/test/a11y.test.tsx) - Deep linking support (
src/lib/deepLink.ts) - Performance monitoring (
src/lib/performance.ts)
Studio Desktop (Electron)#
pnpm nx serve @yemaya/studio-desktop
CLI#
pnpm nx serve @yemaya/cli
Building#
Build All Yemaya Projects#
pnpm nx run-many --target=build --projects=tag:scope:yemaya
Build Individual Projects#
# API (tsup bundler)
pnpm nx build @yemaya/api
# Workers (tsup bundler)
pnpm nx build @yemaya/workers
# Studio Web (Vite)
pnpm nx build @yemaya/studio-web
# Desktop (Electron)
pnpm nx build @yemaya/studio-desktop
# SDK (tsup)
pnpm nx build @yemaya/sdk
# C++ SDK (CMake)
cd libs/yemaya/sdk-cpp
mkdir build && cd build
cmake ..
make
# Python SDK
cd libs/yemaya/sdk-python
pip install -e .
Docker Builds#
# API Docker image
pnpm nx docker-build @yemaya/api
# Workers Docker image
pnpm nx docker-build @yemaya/workers
Testing#
Test Runner Configuration#
All Yemaya apps use Vitest via the @nx/vite:test executor.
Running Tests#
# All Yemaya projects
pnpm nx run-many --target=test --projects=tag:scope:yemaya
# Individual projects
pnpm nx test @yemaya/api
pnpm nx test @yemaya/workers
pnpm nx test @yemaya/studio-web
# With coverage
cd apps/yemaya/api
pnpm test:coverage # vitest run --coverage
# Watch mode
cd apps/yemaya/api
pnpm test:watch # vitest
Linting & Formatting#
# Lint all
pnpm nx run-many --target=lint --projects=tag:scope:yemaya
# Individual
pnpm nx lint @yemaya/api
# Format check
cd apps/yemaya/api
pnpm format:check # prettier --check src/
# Auto-fix
cd apps/yemaya/api
pnpm lint:fix # eslint src/ --fix
pnpm format # prettier --write src/
Type Checking#
pnpm nx typecheck @yemaya/api
pnpm nx typecheck @yemaya/workers
Database Schema#
The Yemaya database (libs/yemaya/database/prisma/schema.prisma) uses
PostgreSQL and is the largest schema in the monorepo. It manages:
- Users & Authentication - Users, accounts (OAuth), sessions, API keys, preferences
- Organizations & Teams - Organizations, members, roles, invitations
- Projects & Workspaces - Projects, project members, permissions
- Asset Library - Assets with metadata, tags, thumbnails, versions
- Activity & Notifications - Activity feed, in-app and email notifications
- Billing & Subscriptions - Plans, subscriptions, invoices, payment methods
- Plugin Marketplace - Plugins, versions, reviews, installations
The database URL is YEMAYA_DATABASE_URL.
Domain-specific data (generation jobs, world state, research documents, build pipelines) lives in separate domain databases accessed via domain client libraries.
Prisma Workflow#
cd libs/yemaya/database
# Generate client after schema changes
npx prisma generate
# Create new migration
npx prisma migrate dev --name your_change_description
# Apply migrations in production
npx prisma migrate deploy
# View database in browser
npx prisma studio
Common Development Tasks#
Adding a New Production Pipeline#
Yemaya's autonomous pipelines (libs/yemaya/autonomous-pipelines/) orchestrate
multi-step creative workflows across domains.
-
Define the pipeline steps and their domain dependencies:
typescriptexport const myPipeline = { name: 'character-sheet-pipeline', steps: [ { domain: 'sophia', action: 'research-character-archetype' }, { domain: 'isis', action: 'generate-character-art' }, { domain: 'hathor', action: 'create-world-entity' }, { domain: 'bellona', action: 'export-to-engine' }, ], }; -
Implement the pipeline in
libs/yemaya/autonomous-pipelines/. -
Add a BullMQ queue for the pipeline in
apps/yemaya/workers/src/queues.ts. -
Create or extend a worker in
apps/yemaya/workers/src/workers/to process the pipeline jobs. -
Add an API endpoint in
apps/yemaya/api/for triggering the pipeline. -
The worker uses domain client libraries (
@isis/client,@sophia/client,@hathor/client,@oshun/bellona-client) to interact with each domain.
Adding a New API Endpoint#
-
Create the route file in
apps/yemaya/api/src/routes/:typescriptimport { createRoute, z } from '@hono/zod-openapi'; import type { AppType } from '../app.js'; const myRoute = createRoute({ method: 'get', path: '/my-resource', responses: { 200: { description: 'Success', content: { 'application/json': { schema: z.object({ /* ... */ }), }, }, }, }, }); export function registerMyRoutes(app: AppType): void { app.openapi(myRoute, async (c) => { // Handler implementation return c.json({ /* ... */ }); }); } -
Register the routes in the app setup.
-
Run
pnpm nx openapi:generate @yemaya/apito update the OpenAPI spec. -
Run
pnpm nx openapi:validate @yemaya/apito verify the spec.
Adding a New Worker Type#
-
Create the worker file in
apps/yemaya/workers/src/workers/:typescriptimport { Worker, type Job } from 'bullmq'; export function createMyWorker(connection: RedisConnection) { return new Worker( 'my-queue', async (job: Job) => { // Process the job }, { connection, concurrency: 5 } ); } -
Add the queue definition in
src/queues.ts. -
Register the worker in
src/workers/index.ts. -
Add the worker type to the config schema in
src/config.ts:typescripttype: z.enum(['all', 'agent', 'media', 'asset', 'notification', 'generation-3d', 'my-worker']).default('all'), -
Add a start script in
package.json:json"start:my-worker": "node dist/index.js --worker=my-worker"
Adding SDK React Hooks#
The TypeScript SDK (libs/yemaya/sdk/) includes React hooks:
-
Add a new hook in
libs/yemaya/sdk/src/hooks/:typescriptimport { useQuery, useMutation } from '@tanstack/react-query'; import { useYemayaClient } from './useYemaya.js'; export function useMyResource(id: string) { const client = useYemayaClient(); return useQuery({ queryKey: ['my-resource', id], queryFn: () => client.myResource.get(id), }); } -
Export from
libs/yemaya/sdk/src/react.ts.
Debugging Tips#
API Not Starting#
- Verify PostgreSQL is running and
YEMAYA_DATABASE_URLis set - Check Redis connectivity (required for caching)
- Ensure Prisma client is generated
- The API uses
josefor JWT; checkJWT_SECRETenvironment variable - Review Hono middleware chain for error middleware
Worker Not Processing Jobs#
- Workers connect to Redis via
REDIS_HOST/REDIS_PORT - Check which worker type is running (see
--worker=argument) - Verify queue names match between API job submission and worker registration
- Worker config validation is strict (Zod schema); check console for validation errors on startup
- Stalled jobs are detected at
WORKER_STALLED_INTERVAL(default 30s) - Jobs timeout at
WORKER_JOB_TIMEOUT_MS(default 5 minutes)
Studio Web Issues#
- The studio uses Zustand for state management with devtools
- Check store middleware (
src/stores/middleware.ts) for persistence issues - i18n locale files are in
src/i18n/locales/{lang}/ - Route prefetching is in
src/lib/routePrefetch.ts - Service worker issues: check
src/sw.ts
Cross-Domain Integration Issues#
- The API imports domain clients (
@isis/client, etc.) - Event handlers (
@yemaya/event-handlers) process events from other domains - Event publishing (
@yemaya/event-publisher) sends events to other domains - Check the event bus connection (Redis Streams)
Database Issues#
- The Yemaya schema is large;
prisma generatemay take a few seconds - Use
npx prisma studioto browse data visually - For migration conflicts:
npx prisma migrate reset(destructive) - The generated Prisma client is at
libs/yemaya/database/src/generated/client/
Performance#
- API uses Redis caching via
@oshun/cache - Workers have configurable concurrency (default 5 per type)
- The studio web uses lazy loading and route-based code splitting
- Performance monitoring is available via
src/lib/performance.ts - Optimistic updates are in
src/stores/optimistic.ts