# Isis Domain

> **Isis** - The Generative AI Factory

Isis handles all AI-powered content generation including images, video, audio,
3D models, and text. It provides job submission via a Hono REST API with JWT
auth and rate limiting, GPU worker management with Redis queues, workflow
versioning and templating, and output tracking with lineage and provenance for
generative workflows.

## Documentation Claim Status

Capability claims are continuously reconciled against implemented artifacts and
checklist truth in:

- `docs/domains/isis/remediation-v2/ISIS_V2_DOCS_CLAIMS_RECONCILIATION.md`

Unsupported claims are explicitly marked as `[UNSUPPORTED]` in the
reconciliation report and, where applicable, in source docs.

---

## Architecture

### Apps (5)

| App                 | Port | Purpose                                                                                                                     |
| ------------------- | ---- | --------------------------------------------------------------------------------------------------------------------------- |
| `generation-api`    | 4001 | Hono REST API with CORS, secure headers, rate limiting (100 req/60s), JWT auth, Zod validation                              |
| `gpu-worker`        | -    | Redis queue-based GPU worker with 4 executors: `texture-upscale`, `blender-render`, `gaussian-splatting`, `mesh-processing` |
| `workflow-registry` | 3001 | Hono app for workflow and template CRUD, versioning, and starring                                                           |
| `output-registry`   | 3002 | Hono app for output manifest management and retention policy configuration                                                  |
| `cli`               | -    | Commander.js CLI with `generate`, `jobs`, `workflows`, `outputs`, `config`, `health`, `auth` commands                       |

### Libraries (13)

| Library                    | Key Exports                                                                                                 |
| -------------------------- | ----------------------------------------------------------------------------------------------------------- |
| `@isis/client`             | `IsisClient` SDK with `HttpTransport`, typed errors, and full API coverage for jobs, workflows, and outputs |
| `@isis/database`           | Prisma schema (752 lines, 12 models) covering generation jobs, workflows, outputs, and provenance           |
| `@isis/ai-providers`       | Unified provider adapters for Anthropic, OpenAI, Google Generative AI, ElevenLabs, Stability AI, Replicate  |
| `@isis/workflows`          | ComfyUI workflow registry with versioning and template management                                           |
| `@isis/comfyui-nodes`      | Custom ComfyUI nodes for Oshun-specific generation tasks                                                    |
| `@isis/comfyui-sdk`        | ComfyUI SDK wrapper for WebSocket-based workflow execution                                                  |
| `@isis/3d-generation`      | 3D asset generation via Point-E, NeRF, and Gaussian splatting                                               |
| `@isis/ai-video`           | Video generation via Zeroscope and Runway                                                                   |
| `@isis/audio-generation`   | TTS and audio generation pipelines                                                                          |
| `@isis/outputs`            | Output manifest management, lineage tracking, and storage tiering                                           |
| `@isis/event-publisher`    | Typed event publishing for generation lifecycle and asset events                                            |
| `@isis/gaussian-splatting` | NeRF and Gaussian splatting support with mesh processing                                                    |
| `@isis/anomaly-detection`  | Quality scoring and anomaly detection for generated outputs                                                 |

### Technology Stack

| Layer      | Technologies                                                                 |
| ---------- | ---------------------------------------------------------------------------- |
| Framework  | Hono (with CORS, secure headers, rate limiting)                              |
| Database   | PostgreSQL with Prisma ORM (schema: `isis`)                                  |
| AI SDKs    | Anthropic, OpenAI, Google Generative AI, ElevenLabs, Stability AI, Replicate |
| Job Queue  | Redis-based (poll model with configurable interval)                          |
| Storage    | S3-compatible (MinIO in dev)                                                 |
| Auth       | JWT Bearer tokens with middleware validation                                 |
| Validation | Zod schemas for request/response validation                                  |
| Logging    | `@oshun/logging` (Pino-based structured logging)                             |
| Build      | `@nx/js:tsc`                                                                 |
| Testing    | Vitest                                                                       |

---

## Architecture Decisions

Isis domain ADRs are tracked in:

- `docs/domains/isis/adr/README.md`
- `docs/domains/isis/adr/ADR-0001-queueing-model-and-job-envelope-contract.md`
- `docs/domains/isis/adr/ADR-0002-workflow-schema-canonicalization-and-compatibility.md`
- `docs/domains/isis/adr/ADR-0003-runpod-serverless-execution-strategy.md`
- `docs/domains/isis/adr/ADR-0004-runpod-endpoint-provisioning-mechanism.md`

---

## Database Schema

12 models in `libs/isis/database/prisma/schema.prisma` (752 lines):

| Model              | Table                | Purpose                                                |
| ------------------ | -------------------- | ------------------------------------------------------ |
| `GenerationJob`    | `generation_jobs`    | Job queue with 16 generation types and status tracking |
| `Workflow`         | `workflows`          | Workflow definitions with category and tags            |
| `WorkflowVersion`  | `workflow_versions`  | Versioned workflow configurations                      |
| `WorkflowTemplate` | `workflow_templates` | Reusable workflow templates                            |
| `WorkflowStar`     | `workflow_stars`     | User favorites for workflows                           |
| `GeneratedOutput`  | `generated_outputs`  | File manifest with lineage and storage tier            |
| `LineageEdge`      | `lineage_edges`      | Output derivation and transformation tracking          |
| `Provenance`       | `provenance`         | Generation metadata, inputs, and parameters            |
| `ModelRegistry`    | `model_registry`     | Checkpoint, LoRA, and upscaler management              |
| `RetentionPolicy`  | `retention_policies` | Output retention and lifecycle configuration           |
| `GpuWorker`        | `gpu_workers`        | Worker registration and GPU capabilities               |
| `AuditLog`         | `audit_log`          | Operation audit trail                                  |

---

## API Endpoints

### Generation API (Port 4001)

Base URL: `http://localhost:4001`

#### Jobs

| Method | Path                                        | Description                               |
| ------ | ------------------------------------------- | ----------------------------------------- |
| POST   | `/api/v1/jobs`                              | Submit a generation job (requires auth)   |
| GET    | `/api/v1/jobs`                              | List jobs with pagination (requires auth) |
| GET    | `/api/v1/jobs/:id`                          | Get job status                            |
| GET    | `/api/v1/jobs/:id/queue-position`           | Get queue-backed job position             |
| POST   | `/api/v1/jobs/:id/cancel`                   | Cancel a pending/queued/running job       |
| GET    | `/api/v1/jobs/:id/output`                   | Get job output and artifacts              |
| GET    | `/api/v1/jobs/:id/audio`                    | Get audio parameter/retrieval contracts   |
| GET    | `/api/v1/jobs/admin/queue/health`           | Admin queue health snapshot               |
| POST   | `/api/v1/jobs/admin/queue/drain`            | Admin drain waiting/delayed queue jobs    |
| POST   | `/api/v1/jobs/admin/legacy-pending/recover` | Admin recover legacy pending jobs         |
| POST   | `/api/v1/jobs/admin/workflow-ids/backfill`  | Admin backfill missing workflow IDs       |
| POST   | `/api/v1/jobs/admin/costs/backfill`         | Admin backfill missing historical costs   |
| GET    | `/api/v1/jobs/dead-letter`                  | List dead-letter queue entries            |
| POST   | `/api/v1/jobs/dead-letter/:outboxId/replay` | Replay a dead-letter queue entry          |

#### Models

| Method | Path             | Description                      |
| ------ | ---------------- | -------------------------------- |
| GET    | `/api/v1/models` | List available models            |
| POST   | `/api/v1/models` | Register a model (requires auth) |

#### Health

| Method | Path      | Description     |
| ------ | --------- | --------------- |
| GET    | `/health` | Liveness check  |
| GET    | `/ready`  | Readiness check |

### Workflow Registry (Port 3001)

| Method | Path                    | Description          |
| ------ | ----------------------- | -------------------- |
| POST   | `/api/v1/workflows`     | Create workflow      |
| GET    | `/api/v1/workflows`     | List workflows       |
| GET    | `/api/v1/workflows/:id` | Get workflow details |
| POST   | `/api/v1/templates`     | Create template      |
| GET    | `/api/v1/templates`     | List templates       |

### Output Registry (Port 3002)

| Method | Path                                        | Description                       |
| ------ | ------------------------------------------- | --------------------------------- |
| GET    | `/api/v1/outputs`                           | List generated outputs            |
| GET    | `/api/v1/outputs/:id`                       | Get output metadata               |
| POST   | `/api/v1/outputs/admin/provenance/backfill` | Admin backfill missing provenance |
| POST   | `/api/v1/policies`                          | Create retention policy           |
| GET    | `/api/v1/policies`                          | List retention policies           |

---

## Generation Types

Isis supports 16 generation modes across 5 categories:

**Image:**

- `TEXT_TO_IMAGE`, `IMAGE_TO_IMAGE`, `UPSCALING`, `INPAINTING`

**Video:**

- `TEXT_TO_VIDEO`, `IMAGE_TO_VIDEO`

**3D:**

- `TEXT_TO_3D`, `IMAGE_TO_3D`, `GAUSSIAN_SPLATTING`, `MESH_PROCESSING`,
  `TEXTURE_UPSCALE`, `BLENDER_RENDER`

**Audio:**

- `TEXT_TO_AUDIO`, `VOICE_SYNTHESIS`, `MUSIC_GENERATION`

---

## AI Provider Adapters

`@isis/ai-providers` provides a unified interface across:

| Provider                 | Capabilities                        |
| ------------------------ | ----------------------------------- |
| **Anthropic**            | Claude models for text generation   |
| **OpenAI**               | GPT for text, DALL-E for images     |
| **Google Generative AI** | Gemini models                       |
| **ElevenLabs**           | Voice synthesis                     |
| **Stability AI**         | Image generation (Stable Diffusion) |
| **Replicate**            | Open-source model hosting           |

---

## GPU Worker Architecture

The GPU worker (`apps/isis/gpu-worker`) operates on a poll-based model:

1. Polls Redis queue for pending jobs at configurable intervals
2. Routes jobs to appropriate executor based on generation type
3. Four built-in executors: `texture-upscale`, `blender-render`,
   `gaussian-splatting`, `mesh-processing`
4. Reports progress and completion status back to the queue
5. Handles graceful shutdown on SIGINT/SIGTERM

---

## Cross-Domain Integration

### Events Published

- `isis.job.submitted` / `isis.job.completed` / `isis.job.failed` - Generation
  lifecycle
- `isis.asset.generated` - New asset available (consumed by Bellona for engine
  import)
- `isis.output.stored` - Output file stored in S3

### Events Consumed

| Source Event                  | Purpose                             |
| ----------------------------- | ----------------------------------- |
| `yemaya.generation.requested` | Creative studio generation requests |
| `hathor.asset.requested`      | Worldbuilding asset generation      |

---

## Environment Variables

```bash
ISIS_DATABASE_URL=postgresql://oshun:oshun_dev@localhost:5432/isis
REDIS_URL=redis://localhost:6379
ISIS_QUEUE_NAME=isis:jobs:submit
# Optional in dev. In production, set ISIS_QUEUE_REDIS_URL or REDIS_URL.
ISIS_QUEUE_REDIS_URL=redis://localhost:6379
# Optional rollout switch. Set to false to pause outbox->queue dispatch.
ISIS_QUEUE_PRODUCER_ENABLED=true
# Optional canary rollout when ISIS_QUEUE_PRODUCER_ENABLED=false
ISIS_QUEUE_PRODUCER_CANARY_ENABLED=false
ISIS_QUEUE_PRODUCER_CANARY_PERCENTAGE=0
# Optional comma-separated project allowlist for queue canary routing
ISIS_QUEUE_PRODUCER_CANARY_PROJECT_IDS=
S3_ENDPOINT=http://localhost:9000
S3_ACCESS_KEY=minioadmin
S3_SECRET_KEY=minioadmin

# AI Providers
ISIS_OPENAI_API_KEY=...
ISIS_ANTHROPIC_API_KEY=...
ISIS_REPLICATE_API_TOKEN=...
ISIS_ELEVENLABS_API_KEY=...
ISIS_STABILITY_API_KEY=...

# Service Ports
ISIS_API_PORT=4001
ISIS_WORKFLOW_PORT=3001
ISIS_OUTPUT_PORT=3002
```

---

## Development

```bash
# Start Isis local stack with GPU profile
docker compose -f docker/docker-compose.yml --profile gpu up -d \
  postgres redis qdrant isis-api isis-generator

# Validate compose path wiring
bash scripts/isis/validate_compose_app_paths.sh

# Run compose smoke checks (/health + /ready)
CONFIG_ONLY=true scripts/isis/smoke_compose_health.sh

# Verify canary SLO/cost release gate from job history
ISIS_DATABASE_URL=postgres://... scripts/isis/verify_canary_slo_cost.sh

# Advance canary rollout with promote/rollback decision output
ISIS_DATABASE_URL=postgres://... CURRENT_PERCENTAGE=10 \
  scripts/isis/advance_queue_canary_rollout.sh

# Run pre-cutover game-day harness and emit structured report
ISIS_DATABASE_URL=postgres://... ISIS_API_BASE_URL=https://<isis-api-host> \
  CURRENT_PERCENTAGE=10 STRICT=true scripts/isis/run_release_gameday.sh

# Run post-cutover validation harness and emit sign-off evidence report
ISIS_DATABASE_URL=postgres://... ISIS_API_BASE_URL=https://<isis-api-host> \
  STRICT=true scripts/isis/run_post_cutover_validation.sh

# Run the generation API
pnpm nx dev @isis/generation-api

# Run the workflow registry
pnpm nx dev @isis/workflow-registry

# Run the output registry
pnpm nx dev @isis/output-registry

# Run tests
pnpm nx test @isis/client
pnpm nx test @isis/ai-providers
pnpm nx test @isis/workflows

# Build all Isis libraries
pnpm nx run-many --target=build --projects=tag:scope:isis

# Run database migrations
pnpm nx run @isis/database:prisma:migrate
pnpm nx run @isis/database:prisma:generate
```

For full local startup details (GPU and non-GPU paths), see
`docs/domains/isis/developer-guide.md`.

---

## CI/CD

Isis uses the core CI pipeline (`ci.yml`) and the general ECS deployment
pipeline (`deploy-ecs.yml`). Database migrations run via `database.yml`.

## Related Documentation

- [Architecture](./architecture.md) - Detailed Isis architecture
- [Monorepo Architecture](../../architecture.md) - Monorepo overview
- [Cross-Domain Integration](../../cross-domain-integration.md)
- [Isis RunPod On-Call Runbook](../../runbooks/isis-runpod-oncall.md)
- [Isis Security Incident Response Runbook](../../runbooks/isis-security-incident-response.md)
- [Isis Compliance Incident Response Runbook](../../runbooks/isis-compliance-incident-response.md)
