Canonical source: the canonical architecture document for this domain is
deep-dive/architecture.md , rendered in the
docs center. This file is its operational companion (service topology, APIs,
schemas, runtime detail) — reconciled by linking, not duplicated (§2, §13).
Comprehensive architectural documentation for the Isis Generative AI Factory.
System Overview#
Isis is the generative AI content factory within the Oshun monorepo. It
orchestrates AI-powered generation of images, video, audio, 3D models, textures,
and more through a job-based architecture. The system provides job submission
via REST API, GPU worker processing with ComfyUI and custom executors, workflow
management with versioning and templates, and output tracking with full
provenance and lineage.
Isis acts as the creative engine of the platform: other domains submit
generation requests, and Isis manages the full lifecycle from job queuing
through GPU execution to output storage and provenance recording.
Service Architecture#
Apps (5 services)#
text Copy
+------------------+
| isis CLI |
| (Commander) |
+--------+---------+
|
+----------------+----------------+
| | |
+--------v--------+ +----v----------+ +----v-----------+
| generation-api | | workflow- | | output- |
| Port 4001 | | registry | | registry |
| (Hono) | | Port 3001 | | Port 3002 |
| /api/v1/jobs | | (Hono) | | (Hono) |
| /api/v1/models | | /api/v1/ | | /api/v1/ |
+--------+--------+ | workflows | | outputs |
| | /api/v1/ | | /api/v1/ |
| | templates | | policies |
| +---------------+ +----------------+
|
+--------v--------+
| gpu-worker |
| Redis queue |
| GPU executors |
| - texture |
| - blender |
| - gaussian |
| - mesh |
+-----------------+
generation-api (Port 4001)#
The primary REST API for submitting and managing AI generation jobs. Built with
Hono framework with comprehensive middleware stack.
Framework : Hono with CORS, secure headers, rate limiting, pretty JSON
Authentication : JWT-based auth with API key support. JWT_SECRET required
in production (fail-fast). Routes /health and /ready bypass auth
Rate Limiting : 100 requests per 60 seconds on /api/* routes
Validation : Zod schemas via @hono/zod-validator for all request/response
payloads
Services : JobService (submit, list, cancel, stats), ModelService
(register, list, get, update, upload URLs), ComfyUIService (workflow
execution)
Health : /health (liveness), /ready (database connectivity check for
jobs and models with latency metrics)
Dependencies : @oshun/logging, @hono/zod-validator
gpu-worker#
Background GPU job processor that polls a Redis queue and dispatches to
specialized executors.
Queue : Redis-based job queue (QUEUE_NAME env, default gpu-jobs)
GPU Devices : Configurable via GPU_DEVICES env (comma-separated device
indices)
Executors :
texture-upscale-executor - Texture upscaling
blender-render-executor - Blender scene rendering
gaussian-splatting-executor - NeRF/Gaussian splatting
mesh-processing-executor - 3D mesh operations
Configuration : Worker ID, type (general/specialized), work directory,
model cache directory, log level
Events : Emits worker:started, worker:stopped, job:started,
job:progress, job:completed, job:failed
Logging : Pino logger with structured output
workflow-registry (Port 3001)#
Hono-based API for managing ComfyUI workflow definitions, versioning, and
templates.
Routes : /api/v1/workflows (CRUD, versioning), /api/v1/templates
(starter templates)
Storage : Currently in-memory (production would migrate to PostgreSQL)
Features : Workflow versioning with changelogs, star/favorite system,
categorization, visibility controls (PRIVATE/TEAM/ORGANIZATION/PUBLIC)
Dependencies : @oshun/logging
output-registry (Port 3002)#
Hono-based API for output file management, storage tiering, and retention
policies.
Routes : /api/v1/outputs (CRUD, search, access tracking),
/api/v1/policies (retention policy management)
Features : Storage tiering (HOT/WARM/COLD/GLACIER), retention policies with
auto-archive, provenance tracking, lineage graphs, file deduplication via
content hashing
Health : Database connectivity check with latency metrics
Dependencies : @oshun/logging
cli#
Commander-based CLI tool (isis) for AI content generation workflow automation.
Commands : generate (image/video/audio/3D), jobs (list/cancel/status),
workflows (manage), outputs (list/download), config (init/set/profile),
health (service check), auth (login/logout/token)
API URL : Defaults to http://localhost:3001, configurable via
ISIS_API_URL
Features : Progress bars, JSON output mode, API key authentication, debug
mode
Dependencies : commander, chalk, dotenv
Library Architecture#
Core Libraries (6)#
Library
Purpose
@isis/client
TypeScript SDK (IsisClient): generation API client, job submission, wait-for-completion, output download. Includes HttpTransport, RequestBuilder, typed errors (NetworkError, TimeoutError, RateLimitError, JobError, WorkflowError)
@isis/database
Prisma ORM schema and generated client for the isis PostgreSQL schema
@isis/ai-providers
Unified provider adapters: Anthropic (Claude), OpenAI (GPT/DALL-E), Google Generative AI (Gemini), ElevenLabs (voice), Stability AI (image), Replicate (open-source models)
@isis/workflows
ComfyUI workflow registry: definition storage, versioning, template management
@isis/outputs
Output manifest management: file tracking, storage tiering, retention enforcement
@isis/event-publisher
Cross-domain event publishing for generation lifecycle events
ComfyUI Integration (2)#
Library
Purpose
@isis/comfyui-sdk
ComfyUI SDK wrapper: server connection, workflow execution, node management
@isis/comfyui-nodes
Custom ComfyUI node definitions for Oshun-specific generation workflows
Specialized Generation (4)#
Library
Purpose
@isis/3d-generation
3D asset generation: Point-E (text-to-3D), NeRF, Gaussian splatting
@isis/ai-video
Video generation: Zeroscope, Runway integration
@isis/audio-generation
TTS and audio generation: ElevenLabs voice synthesis, music generation
@isis/gaussian-splatting
NeRF and Gaussian splatting: point cloud processing, splat rendering, scene reconstruction
Quality Assurance (1)#
Library
Purpose
@isis/anomaly-detection
Quality and anomaly detection for generated outputs
Data Flow#
Generation Pipeline#
text Copy
Client/CLI/Other Domain Isis Services Storage
───────────────────── ──────────────────────── ──────────────────
POST /api/v1/jobs generation-api
{ │
type: TEXT_TO_IMAGE, │ validate request
prompt: "...", │ create GenerationJob
model: "sd-xl" │ enqueue to Redis
} │
│
gpu-worker
│ poll queue
│
├─ resolve model from
│ ModelRegistry
│
├─ execute via
│ ComfyUI / Executor
│
├─ upload output to S3
│
├─ create GeneratedOutput
│ with hash, dimensions,
│ storage tier
│
├─ create Provenance
│ record (model, params,
│ seed, GPU, cost)
│
└─ update job status ──> output-registry
COMPLETED │
store manifest,
lineage edges
│
S3 bucket ──> HOT
(auto-tier
to WARM/COLD)
Workflow Execution Flow#
text Copy
User workflow-registry generation-api
──── ───────────────── ──────────────
Create workflow
with ComfyUI nodes ──> Store definition
+ version history
│
Submit job with │
workflowId ──────────────────────────────────> Resolve workflow
│
Load definition
│
Validate params
against schema
│
Execute via
ComfyUI/Executor
Output Lineage#
text Copy
Original Image Upscaled Converted
───────────── ──────── ─────────
GeneratedOutput ──UPSCALED_FROM──> GeneratedOutput
(512x512) (2048x2048)
│
CONVERTED_FROM
│
v
GeneratedOutput
(engine-native)
Database Schema#
PostgreSQL database with Prisma ORM. Connection URL:
ISIS_DATABASE_URL=postgresql://oshun:oshun_dev@localhost:5432/isis
Core Models#
Model
Purpose
Key Fields
GenerationJob
Job queue with full status tracking
type (16 generation types), status, priority, prompt, negativePrompt, modelId, workflowId, parameters, gpuTimeSeconds, costEstimate
Workflow
Workflow definitions with versioning
engine (COMFYUI/BLENDER/UNREAL/GODOT/CUSTOM), category, status, visibility, currentVersion, definition, parameters, tags[], runCount, starCount
WorkflowVersion
Version history for workflows
version, changelog, definition, deprecated
WorkflowTemplate
Starter templates for common workflows
engine, category, definition, featured, usageCount
WorkflowStar
Star/favorite tracking per user
workflowId, userId
GeneratedOutput
Output file manifest with storage management
type (IMAGE/VIDEO/AUDIO/MODEL_3D/...), mimeType, size, hashValue, storageBucket, storageKey, storageTier, width, height, duration, accessCount
LineageEdge
Derivation relationships between outputs
sourceId, targetId, type (DERIVED_FROM/COMPOSED_OF/REFINED_FROM/UPSCALED_FROM/CONVERTED_FROM)
Provenance
Full generation metadata and audit trail
workflowInfo, modelInfo, lorasUsed, parameters, seed, gpu, provider, cost
ModelRegistry
Model/checkpoint/LoRA registry
type (CHECKPOINT/LORA/CONTROLNET/VAE/EMBEDDING/UPSCALER), format (SAFETENSORS/PYTORCH/ONNX/GGUF/...), version, hashValue, triggerWords[], compatibleEngines[]
RetentionPolicy
Storage lifecycle rules
hotRetentionDays, warmRetentionDays, coldRetentionDays, totalRetentionDays, archiveAfterInactivityDays
GpuWorker
GPU worker registration
gpuType, gpuMemoryMb, gpuCount, supportedTypes[], supportedEngines[], totalGpuTimeSeconds
AuditLog
Change tracking
action, entityType, entityId, previousState, newState
Generation Types (16)#
TEXT_TO_IMAGE, IMAGE_TO_IMAGE, TEXT_TO_VIDEO, IMAGE_TO_VIDEO,
TEXT_TO_3D, IMAGE_TO_3D, TEXT_TO_AUDIO, VOICE_SYNTHESIS,
MUSIC_GENERATION, UPSCALING, INPAINTING, BLENDER_RENDER,
GAUSSIAN_SPLATTING, MESH_PROCESSING, TEXTURE_UPSCALE
API Endpoints#
Generation API (Port 4001)#
Method
Path
Description
POST
/api/v1/jobs
Submit a generation job
GET
/api/v1/jobs
List jobs (filtered, paginated)
GET
/api/v1/jobs/stats
Queue statistics
GET
/api/v1/jobs/:id
Get job status
POST
/api/v1/jobs/:id/cancel
Cancel a job
GET
/api/v1/jobs/:id/output
Get job output
GET
/api/v1/jobs/:id/audio
Get audio contract snapshot
POST
/api/v1/models
Register a model
GET
/api/v1/models
List models
GET
/api/v1/models/:id
Get model details
PATCH
/api/v1/models/:id
Update model
POST
/api/v1/models/:id/upload-url
Get pre-signed upload URL
POST
/api/v1/models/:id/upload-complete
Mark upload complete
GET
/health
Liveness check
GET
/ready
Readiness (DB health)
Workflow Registry (Port 3001)#
Method
Path
Description
POST
/api/v1/workflows
Create workflow
GET
/api/v1/workflows
List workflows
GET
/api/v1/workflows/:id
Get workflow
PATCH
/api/v1/workflows/:id
Update workflow
DELETE
/api/v1/workflows/:id
Delete workflow
GET
/api/v1/templates
List templates
GET
/api/v1/templates/:id
Get template
GET
/health
Liveness check
GET
/ready
Readiness check
Output Registry (Port 3002)#
Method
Path
Description
POST
/api/v1/outputs
Register output
GET
/api/v1/outputs
List outputs
GET
/api/v1/outputs/:id
Get output details
PATCH
/api/v1/outputs/:id
Update output
DELETE
/api/v1/outputs/:id
Delete output
GET
/api/v1/policies
List retention policies
POST
/api/v1/policies
Create retention policy
GET
/health
Liveness check
GET
/ready
Readiness (DB health)
Technology Stack#
Layer
Technology
Runtime
Node.js (TypeScript)
API Framework
Hono (all 3 HTTP services)
Validation
Zod via @hono/zod-validator
Database
PostgreSQL with Prisma ORM (schema: isis)
Job Queue
Redis-based queue
Object Storage
S3-compatible (MinIO in dev)
CLI Framework
Commander.js with chalk
AI SDKs
Anthropic, OpenAI, Google Generative AI, ElevenLabs, Stability AI, Replicate
GPU Processing
ComfyUI, custom executors (Blender, Gaussian splatting)
Logging
@oshun/logging (generation-api), Pino (gpu-worker)
Build
Nx with @nx/js:tsc
Testing
Vitest
Deployment Architecture#
text Copy
┌──────────────────────────────────────────────────────────────┐
│ ECS Cluster │
│ │
│ ┌───────────────┐ ┌────────────────┐ ┌────────────────┐ │
│ │generation-api │ │workflow-registry│ │output-registry │ │
│ │ (Fargate) │ │ (Fargate) │ │ (Fargate) │ │
│ │ Port 4001 │ │ Port 3001 │ │ Port 3002 │ │
│ └───────┬───────┘ └────────┬────────┘ └────────┬───────┘ │
│ │ │ │ │
│ ┌───────┴───────────────────┴─────────────────────┴───────┐ │
│ │ Redis │ │
│ │ (job queue + cache) │ │
│ └──────────────────────┬───────────────────────────────────┘ │
│ │ │
│ ┌──────────────────────┴───────────────────────────────────┐ │
│ │ PostgreSQL │ │
│ │ (isis schema) │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ gpu-worker(s) │ │
│ │ (EC2 GPU instances) │ │
│ │ NVIDIA A10G / A100 / T4 │ │
│ │ Multiple workers per instance │ │
│ └──────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘
│
┌─────────┴─────────┐
│ S3 (MinIO dev) │
│ Generated outputs │
│ Model storage │
└───────────────────┘
GPU Workers : Run on EC2 instances with NVIDIA GPUs. Support multiple GPU
devices per instance. Auto-scale based on queue depth
API Services : Stateless Fargate tasks behind load balancer
CI/CD : Core ci.yml pipeline, deploy-ecs.yml for deployment,
database.yml for migrations
Cross-Domain Dependencies#
Shared Oshun Libraries Used#
Library
Usage in Isis
@oshun/logging
Structured logging in generation-api, registries
Domains That Consume Isis Events#
Consumer Domain
Event
Reaction
Bellona
isis.asset.generated
Converts and imports into engine
Yemaya
(via client SDK)
Displays generated content in UI
Domains That Request Generation#
Requester Domain
Mechanism
Use Case
Yemaya
REST API / client SDK
User-initiated generation
Hathor
REST API
Worldbuilding asset generation
Bellona
@bellona/integration
Engine-specific asset generation