Eve library

Yemaya Architecture Documentation

Guide

Revision 1

On this page

This document provides a comprehensive overview of the Yemaya platform architecture.

System Overview#

Yemaya is a modular, microservices-oriented platform built with TypeScript. The architecture emphasizes:

  • Modularity: Each package has a single responsibility
  • Type Safety: Full TypeScript throughout the stack
  • Scalability: Horizontal scaling with Kubernetes
  • Extensibility: Plugin architecture for custom integrations
  • Real-time: WebSocket-based collaboration

Architecture Diagram#

text
┌─────────────────────────────────────────────────────────────────────────┐
│                           Client Applications                            │
├─────────────────────────────────────────────────────────────────────────┤
│  ┌───────────────┐   ┌───────────────┐   ┌───────────────┐              │
│  │   Web App     │   │  Desktop App  │   │    CLI Tool   │              │
│  │  (React/Vite) │   │  (Electron)   │   │  (Commander)  │              │
│  └───────────────┘   └───────────────┘   └───────────────┘              │
└─────────────────────────────────────────────────────────────────────────┘
                                │
                                │ HTTPS / WSS
                                ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                            API Gateway                                   │
│                    (nginx / Kong / Traefik)                             │
│  ┌─────────────────────────────────────────────────────────────────┐   │
│  │ • Rate Limiting  • Authentication  • Load Balancing  • SSL      │   │
│  └─────────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────┘
                                │
                                ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                          Backend Services                                │
├─────────────────────────────────────────────────────────────────────────┤
│  ┌───────────────┐   ┌───────────────┐   ┌───────────────┐              │
│  │    API        │   │   WebSocket   │   │   Workers     │              │
│  │   (Hono)      │   │   Server      │   │  (Bull/BullMQ)│              │
│  └───────────────┘   └───────────────┘   └───────────────┘              │
│          │                   │                   │                       │
│          └───────────────────┴───────────────────┘                       │
│                              │                                           │
│                              ▼                                           │
│  ┌─────────────────────────────────────────────────────────────────┐   │
│  │                    Service Layer                                 │   │
│  │  ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐   │   │
│  │  │Projects │ │ Assets  │ │  AI     │ │ Render  │ │ Collab  │   │   │
│  │  │Service  │ │Service  │ │Service  │ │Service  │ │Service  │   │   │
│  │  └─────────┘ └─────────┘ └─────────┘ └─────────┘ └─────────┘   │   │
│  └─────────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────┘
                                │
                                ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                         Data Layer                                       │
├─────────────────────────────────────────────────────────────────────────┤
│  ┌───────────────┐   ┌───────────────┐   ┌───────────────┐              │
│  │  PostgreSQL   │   │     Redis     │   │    MinIO      │              │
│  │  (Database)   │   │   (Cache)     │   │  (Storage)    │              │
│  └───────────────┘   └───────────────┘   └───────────────┘              │
└─────────────────────────────────────────────────────────────────────────┘
                                │
                                ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                      External Integrations                               │
├─────────────────────────────────────────────────────────────────────────┤
│  ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐           │
│  │ Blender │ │  Godot  │ │ Unreal  │ │ AI APIs │ │ DaVinci │           │
│  │  Bridge │ │  Bridge │ │  Bridge │ │(Claude, │ │ Resolve │           │
│  │         │ │         │ │         │ │  GPT)   │ │         │           │
│  └─────────┘ └─────────┘ └─────────┘ └─────────┘ └─────────┘           │
└─────────────────────────────────────────────────────────────────────────┘

Component Deep Dive#

API Layer (@yemaya/api)#

The API is built with Hono, a lightweight web framework:

  • OpenAPI-first: All routes are documented with Zod-OpenAPI
  • Type-safe: Full TypeScript with compile-time validation
  • Middleware: Authentication, rate limiting, logging, CORS
  • Versioned: API versioning (v1, v2, etc.)
typescript
// Example route definition
app.openapi(
  createRoute({
    method: 'get',
    path: '/v1/projects',
    request: { query: ProjectQuerySchema },
    responses: { 200: ProjectListResponseSchema },
  }),
  async (c) => {
    const projects = await projectService.list(c.get('user').id);
    return c.json({ data: projects });
  }
);

Database Layer (@yemaya/database)#

PostgreSQL with Prisma ORM:

  • Migrations: Version-controlled schema changes
  • Connection Pooling: PgBouncer for connection management
  • Soft Deletes: Data archival instead of hard deletes
  • Audit Logging: Track all data changes

Caching Strategy (@yemaya/core)#

Multi-level caching with Redis:

  1. L1 Cache: In-memory (process-local)
  2. L2 Cache: Redis (shared)
  3. CDN Cache: For static assets
typescript
// Cache decorator pattern
@Cacheable({ ttl: 300, key: 'project:{id}' })
async getProject(id: string): Promise<Project> {
  return this.db.project.findUnique({ where: { id } });
}

Object Storage (@yemaya/assets)#

MinIO-compatible S3 storage:

  • Buckets: Organized by type (assets, uploads, renders)
  • Presigned URLs: Secure direct uploads
  • CDN Integration: Cloudflare/CloudFront delivery
  • Versioning: Asset version history

AI Integration (@yemaya/ai)#

Multi-provider AI support:

typescript
// Provider abstraction
const ai = createAIClient({
  provider: 'anthropic', // or 'openai', 'google'
  model: 'claude-3-opus',
});

const response = await ai.generate({
  prompt: 'Generate a character backstory',
  maxTokens: 1000,
});

Agent System (@yemaya/agents)#

LangGraph-based multi-agent orchestration:

  • Director Agent: Coordinates other agents
  • Specialized Agents: Domain-specific tasks
  • Human-in-the-loop: Approval workflows
  • Memory: Persistent context across sessions

Real-time Collaboration (@yemaya/collaboration)#

Yjs CRDTs with WebSocket transport:

  • Conflict-free: Automatic merge of concurrent changes
  • Offline Support: Sync when reconnected
  • Presence: User cursors and selections
  • Undo/Redo: Collaborative history

Engine Bridges#

Blender (@yemaya/blender)#

Python-to-TypeScript bridge via WebSocket:

python
# Blender addon
yemaya.export_mesh("character", {
    "format": "gltf",
    "include_animations": True
})

Godot (@yemaya/godot)#

GDExtension with C++ bindings:

gdscript
# Godot GDScript
var asset = Yemaya.load_asset("res://characters/hero.tres")
asset.apply_to_node($Character)

Unreal (@yemaya/unreal)#

Unreal Engine plugin:

cpp
// Unreal C++
UYemayaAsset* Asset = UYemayaSubsystem::Get()->LoadAsset(AssetId);
Asset->ApplyToActor(TargetActor);

Data Flow#

Asset Upload Flow#

text
Client                API              Storage         Database
  │                    │                  │               │
  │ 1. Request upload  │                  │               │
  │ ─────────────────> │                  │               │
  │                    │                  │               │
  │ 2. Presigned URL   │                  │               │
  │ <───────────────── │                  │               │
  │                    │                  │               │
  │ 3. Upload to S3    │                  │               │
  │ ────────────────────────────────────> │               │
  │                    │                  │               │
  │ 4. Confirm upload  │                  │               │
  │ ─────────────────> │                  │               │
  │                    │                  │               │
  │                    │ 5. Verify file   │               │
  │                    │ ─────────────────>               │
  │                    │                  │               │
  │                    │ 6. Create record │               │
  │                    │ ────────────────────────────────>│
  │                    │                  │               │
  │ 7. Asset metadata  │                  │               │
  │ <───────────────── │                  │               │

Real-time Collaboration Flow#

text
User A              Server             User B
  │                    │                  │
  │ 1. Edit document   │                  │
  │ ─────────────────> │                  │
  │                    │ 2. Broadcast     │
  │                    │ ─────────────────>
  │                    │                  │
  │                    │ 3. Edit document │
  │                    │ <─────────────────
  │ 4. Receive update  │                  │
  │ <───────────────── │                  │
  │                    │                  │
  │ 5. CRDT merge      │ 6. CRDT merge   │
  │ (automatic)        │ (automatic)      │

Security Architecture#

Authentication#

  • JWT Tokens: Short-lived access tokens (15 min)
  • Refresh Tokens: Long-lived, rotatable (7 days)
  • API Keys: For service-to-service communication

Authorization#

Role-Based Access Control (RBAC):

Role Permissions
Owner Full access, billing management
Admin Project settings, member management
Editor Create/edit content
Viewer Read-only access

Data Protection#

  • Encryption at rest: AES-256 for stored data
  • Encryption in transit: TLS 1.3 for all connections
  • Field-level encryption: For sensitive data (API keys)

Deployment Architecture#

Kubernetes Resources#

yaml
Namespace: yemaya
├── Deployments
│   ├── yemaya-api (3 replicas, HPA 3-20)
│   ├── yemaya-web (2 replicas, HPA 2-10)
│   └── yemaya-worker (2 replicas)
├── StatefulSets
│   ├── yemaya-postgres (1 replica + read replicas)
│   └── yemaya-redis (3 replicas, Sentinel)
├── Services
│   ├── yemaya-api (ClusterIP)
│   ├── yemaya-web (ClusterIP)
│   └── yemaya-postgres (ClusterIP)
├── Ingress
│   └── yemaya-ingress (nginx)
├── ConfigMaps
│   └── yemaya-config
├── Secrets
│   └── yemaya-secrets
└── PodDisruptionBudgets
    ├── yemaya-api-pdb
    └── yemaya-web-pdb

Scaling Strategy#

  • API: Horizontal scaling based on CPU/memory
  • Database: Vertical scaling + read replicas
  • Cache: Redis Cluster for high availability
  • Storage: S3-compatible with CDN

Monitoring & Observability#

Metrics (Prometheus)#

  • Request rate, latency, errors (RED)
  • Resource utilization (CPU, memory, disk)
  • Business metrics (users, projects, renders)

Logging (structured JSON)#

json
{
  "timestamp": "2024-01-15T10:30:00Z",
  "level": "info",
  "service": "api",
  "requestId": "abc123",
  "userId": "user_456",
  "message": "Project created",
  "duration": 45
}

Tracing (OpenTelemetry)#

Distributed tracing across all services for debugging and performance analysis.

Further Reading#