# Yemaya Architecture Documentation

This document provides a comprehensive overview of the Yemaya platform
architecture.

## System Overview

Yemaya is a modular, microservices-oriented platform built with TypeScript. The
architecture emphasizes:

- **Modularity**: Each package has a single responsibility
- **Type Safety**: Full TypeScript throughout the stack
- **Scalability**: Horizontal scaling with Kubernetes
- **Extensibility**: Plugin architecture for custom integrations
- **Real-time**: WebSocket-based collaboration

## Architecture Diagram

```
┌─────────────────────────────────────────────────────────────────────────┐
│                           Client Applications                            │
├─────────────────────────────────────────────────────────────────────────┤
│  ┌───────────────┐   ┌───────────────┐   ┌───────────────┐              │
│  │   Web App     │   │  Desktop App  │   │    CLI Tool   │              │
│  │  (React/Vite) │   │  (Electron)   │   │  (Commander)  │              │
│  └───────────────┘   └───────────────┘   └───────────────┘              │
└─────────────────────────────────────────────────────────────────────────┘
                                │
                                │ HTTPS / WSS
                                ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                            API Gateway                                   │
│                    (nginx / Kong / Traefik)                             │
│  ┌─────────────────────────────────────────────────────────────────┐   │
│  │ • Rate Limiting  • Authentication  • Load Balancing  • SSL      │   │
│  └─────────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────┘
                                │
                                ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                          Backend Services                                │
├─────────────────────────────────────────────────────────────────────────┤
│  ┌───────────────┐   ┌───────────────┐   ┌───────────────┐              │
│  │    API        │   │   WebSocket   │   │   Workers     │              │
│  │   (Hono)      │   │   Server      │   │  (Bull/BullMQ)│              │
│  └───────────────┘   └───────────────┘   └───────────────┘              │
│          │                   │                   │                       │
│          └───────────────────┴───────────────────┘                       │
│                              │                                           │
│                              ▼                                           │
│  ┌─────────────────────────────────────────────────────────────────┐   │
│  │                    Service Layer                                 │   │
│  │  ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐   │   │
│  │  │Projects │ │ Assets  │ │  AI     │ │ Render  │ │ Collab  │   │   │
│  │  │Service  │ │Service  │ │Service  │ │Service  │ │Service  │   │   │
│  │  └─────────┘ └─────────┘ └─────────┘ └─────────┘ └─────────┘   │   │
│  └─────────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────┘
                                │
                                ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                         Data Layer                                       │
├─────────────────────────────────────────────────────────────────────────┤
│  ┌───────────────┐   ┌───────────────┐   ┌───────────────┐              │
│  │  PostgreSQL   │   │     Redis     │   │    MinIO      │              │
│  │  (Database)   │   │   (Cache)     │   │  (Storage)    │              │
│  └───────────────┘   └───────────────┘   └───────────────┘              │
└─────────────────────────────────────────────────────────────────────────┘
                                │
                                ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                      External Integrations                               │
├─────────────────────────────────────────────────────────────────────────┤
│  ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐           │
│  │ Blender │ │  Godot  │ │ Unreal  │ │ AI APIs │ │ DaVinci │           │
│  │  Bridge │ │  Bridge │ │  Bridge │ │(Claude, │ │ Resolve │           │
│  │         │ │         │ │         │ │  GPT)   │ │         │           │
│  └─────────┘ └─────────┘ └─────────┘ └─────────┘ └─────────┘           │
└─────────────────────────────────────────────────────────────────────────┘
```

## Component Deep Dive

### API Layer (@yemaya/api)

The API is built with [Hono](https://hono.dev/), a lightweight web framework:

- **OpenAPI-first**: All routes are documented with Zod-OpenAPI
- **Type-safe**: Full TypeScript with compile-time validation
- **Middleware**: Authentication, rate limiting, logging, CORS
- **Versioned**: API versioning (v1, v2, etc.)

```typescript
// Example route definition
app.openapi(
  createRoute({
    method: 'get',
    path: '/v1/projects',
    request: { query: ProjectQuerySchema },
    responses: { 200: ProjectListResponseSchema },
  }),
  async (c) => {
    const projects = await projectService.list(c.get('user').id);
    return c.json({ data: projects });
  }
);
```

### Database Layer (@yemaya/database)

PostgreSQL with Prisma ORM:

- **Migrations**: Version-controlled schema changes
- **Connection Pooling**: PgBouncer for connection management
- **Soft Deletes**: Data archival instead of hard deletes
- **Audit Logging**: Track all data changes

### Caching Strategy (@yemaya/core)

Multi-level caching with Redis:

1. **L1 Cache**: In-memory (process-local)
2. **L2 Cache**: Redis (shared)
3. **CDN Cache**: For static assets

```typescript
// Cache decorator pattern
@Cacheable({ ttl: 300, key: 'project:{id}' })
async getProject(id: string): Promise<Project> {
  return this.db.project.findUnique({ where: { id } });
}
```

### Object Storage (@yemaya/assets)

MinIO-compatible S3 storage:

- **Buckets**: Organized by type (assets, uploads, renders)
- **Presigned URLs**: Secure direct uploads
- **CDN Integration**: Cloudflare/CloudFront delivery
- **Versioning**: Asset version history

### AI Integration (@yemaya/ai)

Multi-provider AI support:

```typescript
// Provider abstraction
const ai = createAIClient({
  provider: 'anthropic', // or 'openai', 'google'
  model: 'claude-3-opus',
});

const response = await ai.generate({
  prompt: 'Generate a character backstory',
  maxTokens: 1000,
});
```

### Agent System (@yemaya/agents)

LangGraph-based multi-agent orchestration:

- **Director Agent**: Coordinates other agents
- **Specialized Agents**: Domain-specific tasks
- **Human-in-the-loop**: Approval workflows
- **Memory**: Persistent context across sessions

### Real-time Collaboration (@yemaya/collaboration)

Yjs CRDTs with WebSocket transport:

- **Conflict-free**: Automatic merge of concurrent changes
- **Offline Support**: Sync when reconnected
- **Presence**: User cursors and selections
- **Undo/Redo**: Collaborative history

### Engine Bridges

#### Blender (@yemaya/blender)

Python-to-TypeScript bridge via WebSocket:

```python
# Blender addon
yemaya.export_mesh("character", {
    "format": "gltf",
    "include_animations": True
})
```

#### Godot (@yemaya/godot)

GDExtension with C++ bindings:

```gdscript
# Godot GDScript
var asset = Yemaya.load_asset("res://characters/hero.tres")
asset.apply_to_node($Character)
```

#### Unreal (@yemaya/unreal)

Unreal Engine plugin:

```cpp
// Unreal C++
UYemayaAsset* Asset = UYemayaSubsystem::Get()->LoadAsset(AssetId);
Asset->ApplyToActor(TargetActor);
```

## Data Flow

### Asset Upload Flow

```
Client                API              Storage         Database
  │                    │                  │               │
  │ 1. Request upload  │                  │               │
  │ ─────────────────> │                  │               │
  │                    │                  │               │
  │ 2. Presigned URL   │                  │               │
  │ <───────────────── │                  │               │
  │                    │                  │               │
  │ 3. Upload to S3    │                  │               │
  │ ────────────────────────────────────> │               │
  │                    │                  │               │
  │ 4. Confirm upload  │                  │               │
  │ ─────────────────> │                  │               │
  │                    │                  │               │
  │                    │ 5. Verify file   │               │
  │                    │ ─────────────────>               │
  │                    │                  │               │
  │                    │ 6. Create record │               │
  │                    │ ────────────────────────────────>│
  │                    │                  │               │
  │ 7. Asset metadata  │                  │               │
  │ <───────────────── │                  │               │
```

### Real-time Collaboration Flow

```
User A              Server             User B
  │                    │                  │
  │ 1. Edit document   │                  │
  │ ─────────────────> │                  │
  │                    │ 2. Broadcast     │
  │                    │ ─────────────────>
  │                    │                  │
  │                    │ 3. Edit document │
  │                    │ <─────────────────
  │ 4. Receive update  │                  │
  │ <───────────────── │                  │
  │                    │                  │
  │ 5. CRDT merge      │ 6. CRDT merge   │
  │ (automatic)        │ (automatic)      │
```

## Security Architecture

### Authentication

- **JWT Tokens**: Short-lived access tokens (15 min)
- **Refresh Tokens**: Long-lived, rotatable (7 days)
- **API Keys**: For service-to-service communication

### Authorization

Role-Based Access Control (RBAC):

| Role   | Permissions                         |
| ------ | ----------------------------------- |
| Owner  | Full access, billing management     |
| Admin  | Project settings, member management |
| Editor | Create/edit content                 |
| Viewer | Read-only access                    |

### Data Protection

- **Encryption at rest**: AES-256 for stored data
- **Encryption in transit**: TLS 1.3 for all connections
- **Field-level encryption**: For sensitive data (API keys)

## Deployment Architecture

### Kubernetes Resources

```yaml
Namespace: yemaya
├── Deployments
│   ├── yemaya-api (3 replicas, HPA 3-20)
│   ├── yemaya-web (2 replicas, HPA 2-10)
│   └── yemaya-worker (2 replicas)
├── StatefulSets
│   ├── yemaya-postgres (1 replica + read replicas)
│   └── yemaya-redis (3 replicas, Sentinel)
├── Services
│   ├── yemaya-api (ClusterIP)
│   ├── yemaya-web (ClusterIP)
│   └── yemaya-postgres (ClusterIP)
├── Ingress
│   └── yemaya-ingress (nginx)
├── ConfigMaps
│   └── yemaya-config
├── Secrets
│   └── yemaya-secrets
└── PodDisruptionBudgets
    ├── yemaya-api-pdb
    └── yemaya-web-pdb
```

### Scaling Strategy

- **API**: Horizontal scaling based on CPU/memory
- **Database**: Vertical scaling + read replicas
- **Cache**: Redis Cluster for high availability
- **Storage**: S3-compatible with CDN

## Monitoring & Observability

### Metrics (Prometheus)

- Request rate, latency, errors (RED)
- Resource utilization (CPU, memory, disk)
- Business metrics (users, projects, renders)

### Logging (structured JSON)

```json
{
  "timestamp": "2024-01-15T10:30:00Z",
  "level": "info",
  "service": "api",
  "requestId": "abc123",
  "userId": "user_456",
  "message": "Project created",
  "duration": 45
}
```

### Tracing (OpenTelemetry)

Distributed tracing across all services for debugging and performance analysis.

## Further Reading

- [API Documentation](../api/README.md)
- [Database Schema](./database-schema.md)
- [Security Guide](./security.md)
- [Performance Tuning](./performance.md)
