This document provides a comprehensive overview of the Yemaya platform architecture.
System Overview#
Yemaya is a modular, microservices-oriented platform built with TypeScript. The architecture emphasizes:
- Modularity: Each package has a single responsibility
- Type Safety: Full TypeScript throughout the stack
- Scalability: Horizontal scaling with Kubernetes
- Extensibility: Plugin architecture for custom integrations
- Real-time: WebSocket-based collaboration
Architecture Diagram#
text
┌─────────────────────────────────────────────────────────────────────────┐
│ Client Applications │
├─────────────────────────────────────────────────────────────────────────┤
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │
│ │ Web App │ │ Desktop App │ │ CLI Tool │ │
│ │ (React/Vite) │ │ (Electron) │ │ (Commander) │ │
│ └───────────────┘ └───────────────┘ └───────────────┘ │
└─────────────────────────────────────────────────────────────────────────┘
│
│ HTTPS / WSS
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ API Gateway │
│ (nginx / Kong / Traefik) │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ • Rate Limiting • Authentication • Load Balancing • SSL │ │
│ └─────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ Backend Services │
├─────────────────────────────────────────────────────────────────────────┤
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │
│ │ API │ │ WebSocket │ │ Workers │ │
│ │ (Hono) │ │ Server │ │ (Bull/BullMQ)│ │
│ └───────────────┘ └───────────────┘ └───────────────┘ │
│ │ │ │ │
│ └───────────────────┴───────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ Service Layer │ │
│ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │
│ │ │Projects │ │ Assets │ │ AI │ │ Render │ │ Collab │ │ │
│ │ │Service │ │Service │ │Service │ │Service │ │Service │ │ │
│ │ └─────────┘ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │ │
│ └─────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ Data Layer │
├─────────────────────────────────────────────────────────────────────────┤
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │
│ │ PostgreSQL │ │ Redis │ │ MinIO │ │
│ │ (Database) │ │ (Cache) │ │ (Storage) │ │
│ └───────────────┘ └───────────────┘ └───────────────┘ │
└─────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ External Integrations │
├─────────────────────────────────────────────────────────────────────────┤
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
│ │ Blender │ │ Godot │ │ Unreal │ │ AI APIs │ │ DaVinci │ │
│ │ Bridge │ │ Bridge │ │ Bridge │ │(Claude, │ │ Resolve │ │
│ │ │ │ │ │ │ │ GPT) │ │ │ │
│ └─────────┘ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │
└─────────────────────────────────────────────────────────────────────────┘
Component Deep Dive#
API Layer (@yemaya/api)#
The API is built with Hono, a lightweight web framework:
- OpenAPI-first: All routes are documented with Zod-OpenAPI
- Type-safe: Full TypeScript with compile-time validation
- Middleware: Authentication, rate limiting, logging, CORS
- Versioned: API versioning (v1, v2, etc.)
typescript
// Example route definition
app.openapi(
createRoute({
method: 'get',
path: '/v1/projects',
request: { query: ProjectQuerySchema },
responses: { 200: ProjectListResponseSchema },
}),
async (c) => {
const projects = await projectService.list(c.get('user').id);
return c.json({ data: projects });
}
);
Database Layer (@yemaya/database)#
PostgreSQL with Prisma ORM:
- Migrations: Version-controlled schema changes
- Connection Pooling: PgBouncer for connection management
- Soft Deletes: Data archival instead of hard deletes
- Audit Logging: Track all data changes
Caching Strategy (@yemaya/core)#
Multi-level caching with Redis:
- L1 Cache: In-memory (process-local)
- L2 Cache: Redis (shared)
- CDN Cache: For static assets
typescript
// Cache decorator pattern
@Cacheable({ ttl: 300, key: 'project:{id}' })
async getProject(id: string): Promise<Project> {
return this.db.project.findUnique({ where: { id } });
}
Object Storage (@yemaya/assets)#
MinIO-compatible S3 storage:
- Buckets: Organized by type (assets, uploads, renders)
- Presigned URLs: Secure direct uploads
- CDN Integration: Cloudflare/CloudFront delivery
- Versioning: Asset version history
AI Integration (@yemaya/ai)#
Multi-provider AI support:
typescript
// Provider abstraction
const ai = createAIClient({
provider: 'anthropic', // or 'openai', 'google'
model: 'claude-3-opus',
});
const response = await ai.generate({
prompt: 'Generate a character backstory',
maxTokens: 1000,
});
Agent System (@yemaya/agents)#
LangGraph-based multi-agent orchestration:
- Director Agent: Coordinates other agents
- Specialized Agents: Domain-specific tasks
- Human-in-the-loop: Approval workflows
- Memory: Persistent context across sessions
Real-time Collaboration (@yemaya/collaboration)#
Yjs CRDTs with WebSocket transport:
- Conflict-free: Automatic merge of concurrent changes
- Offline Support: Sync when reconnected
- Presence: User cursors and selections
- Undo/Redo: Collaborative history
Engine Bridges#
Blender (@yemaya/blender)#
Python-to-TypeScript bridge via WebSocket:
python
# Blender addon
yemaya.export_mesh("character", {
"format": "gltf",
"include_animations": True
})
Godot (@yemaya/godot)#
GDExtension with C++ bindings:
gdscript
# Godot GDScript
var asset = Yemaya.load_asset("res://characters/hero.tres")
asset.apply_to_node($Character)
Unreal (@yemaya/unreal)#
Unreal Engine plugin:
cpp
// Unreal C++
UYemayaAsset* Asset = UYemayaSubsystem::Get()->LoadAsset(AssetId);
Asset->ApplyToActor(TargetActor);
Data Flow#
Asset Upload Flow#
text
Client API Storage Database
│ │ │ │
│ 1. Request upload │ │ │
│ ─────────────────> │ │ │
│ │ │ │
│ 2. Presigned URL │ │ │
│ <───────────────── │ │ │
│ │ │ │
│ 3. Upload to S3 │ │ │
│ ────────────────────────────────────> │ │
│ │ │ │
│ 4. Confirm upload │ │ │
│ ─────────────────> │ │ │
│ │ │ │
│ │ 5. Verify file │ │
│ │ ─────────────────> │
│ │ │ │
│ │ 6. Create record │ │
│ │ ────────────────────────────────>│
│ │ │ │
│ 7. Asset metadata │ │ │
│ <───────────────── │ │ │
Real-time Collaboration Flow#
text
User A Server User B
│ │ │
│ 1. Edit document │ │
│ ─────────────────> │ │
│ │ 2. Broadcast │
│ │ ─────────────────>
│ │ │
│ │ 3. Edit document │
│ │ <─────────────────
│ 4. Receive update │ │
│ <───────────────── │ │
│ │ │
│ 5. CRDT merge │ 6. CRDT merge │
│ (automatic) │ (automatic) │
Security Architecture#
Authentication#
- JWT Tokens: Short-lived access tokens (15 min)
- Refresh Tokens: Long-lived, rotatable (7 days)
- API Keys: For service-to-service communication
Authorization#
Role-Based Access Control (RBAC):
| Role | Permissions |
|---|---|
| Owner | Full access, billing management |
| Admin | Project settings, member management |
| Editor | Create/edit content |
| Viewer | Read-only access |
Data Protection#
- Encryption at rest: AES-256 for stored data
- Encryption in transit: TLS 1.3 for all connections
- Field-level encryption: For sensitive data (API keys)
Deployment Architecture#
Kubernetes Resources#
yaml
Namespace: yemaya
├── Deployments
│ ├── yemaya-api (3 replicas, HPA 3-20)
│ ├── yemaya-web (2 replicas, HPA 2-10)
│ └── yemaya-worker (2 replicas)
├── StatefulSets
│ ├── yemaya-postgres (1 replica + read replicas)
│ └── yemaya-redis (3 replicas, Sentinel)
├── Services
│ ├── yemaya-api (ClusterIP)
│ ├── yemaya-web (ClusterIP)
│ └── yemaya-postgres (ClusterIP)
├── Ingress
│ └── yemaya-ingress (nginx)
├── ConfigMaps
│ └── yemaya-config
├── Secrets
│ └── yemaya-secrets
└── PodDisruptionBudgets
├── yemaya-api-pdb
└── yemaya-web-pdb
Scaling Strategy#
- API: Horizontal scaling based on CPU/memory
- Database: Vertical scaling + read replicas
- Cache: Redis Cluster for high availability
- Storage: S3-compatible with CDN
Monitoring & Observability#
Metrics (Prometheus)#
- Request rate, latency, errors (RED)
- Resource utilization (CPU, memory, disk)
- Business metrics (users, projects, renders)
Logging (structured JSON)#
json
{
"timestamp": "2024-01-15T10:30:00Z",
"level": "info",
"service": "api",
"requestId": "abc123",
"userId": "user_456",
"message": "Project created",
"duration": 45
}
Tracing (OpenTelemetry)#
Distributed tracing across all services for debugging and performance analysis.