# ADR-0001: Queueing Model and Job Envelope Contract

**Status**: Accepted  
**Date**: 2026-02-21  
**Authors**: Isis Platform, Isis API, Oshun Platform Engineering  
**Reviewers**: `@oshun/core-team`, `@oshun/isis-team`, `@oshun/platform`, `@oshun/devops`

## Context and Problem Statement

Isis currently persists submitted jobs directly to PostgreSQL and marks them `pending` at submit time. Queue state is inferred from database rows and rough wait-time estimation, but no dedicated producer/consumer queue exists in the generation API path.

Current code evidence:
- `apps/isis/generation-api/src/services/job.service.ts` writes jobs directly and computes queue position from `status = 'pending'`.
- `apps/isis/generation-api/src/app.ts` readiness includes placeholder checks (`queue: 'ok'`, `storage: 'ok'`) instead of dependency-truthful queue health.
- `libs/isis/client/src/resources/generation.ts` exposes queue-oriented operations that require a real queue backend contract.

This creates an execution-critical risk: jobs can be accepted without deterministic execution guarantees.

## Decision

Adopt **Redis + BullMQ** as the canonical execution queue for Isis generation jobs, with a **transactional outbox pattern** and a shared, versioned job envelope contract.

### Normative Queue Topology

- `isis:jobs:submit` (primary queue)
- `isis:jobs:dlq` (dead-letter queue)
- `isis:jobs:events` (lifecycle event stream/topic)
- `isis:jobs:outbox` (database-backed outbox relay source)

### Canonical Job Envelope v1

Every enqueued message must follow this minimum shape:

```json
{
  "jobId": "uuid",
  "type": "text-to-image",
  "input": {},
  "priority": "normal",
  "metadata": {},
  "idempotencyKey": "string"
}
```

Required semantics:
- `jobId`: immutable primary correlation key.
- `idempotencyKey`: required for dedupe at producer and consumer boundaries.
- `metadata`: must carry `userId`, `projectId`, and `correlationId` when available.

### State Machine

`pending` is removed from production submission flow and replaced with:

`queued -> running -> completed | failed | cancelled`

`pending` remains only for legacy migration/backfill compatibility.

## Decision Drivers

- Deterministic execution from API submit to worker consumption.
- Safe retry and idempotency guarantees.
- Clear cancellation semantics.
- Operational visibility (depth, lag, throughput, failure classes).
- Compatibility with existing TypeScript/Node services and Redis footprint.

## Considered Options

### Option A: Continue DB-only queue emulation (Rejected)

Pros:
- Minimal short-term code change.

Cons:
- No durable producer/consumer execution contract.
- No native retry/DLQ primitives.
- Queue position and latency are estimations, not queue truth.

### Option B: In-process memory queue per service (Rejected)

Pros:
- Fast local implementation.

Cons:
- Not horizontally safe.
- Loses in-flight work on process restart.
- Cannot support controlled replays or DLQ operations.

### Option C: BullMQ + Redis + Outbox (Chosen)

Pros:
- Production-grade queue semantics (retry, backoff, priorities, delayed jobs).
- Strong operational tooling and clear queue introspection.
- Natural fit for Node/TypeScript and existing platform patterns.

Cons:
- Requires Redis dependency hardening and runtime configuration.
- Requires disciplined outbox and replay operations.

## Consequences

### Positive

- Submit path and execution path are explicitly connected and auditable.
- Retry/DLQ behavior is standardized.
- Queue position and throughput come from runtime queue state.
- Production readiness checks become dependency-truthful.

### Tradeoffs

- Additional operational dependency (Redis/BullMQ) and alerting surface.
- Requires migration of legacy `pending` jobs and backfill tooling.

## Implementation Rules

- Producer writes `jobs` + `outbox` in a single transaction.
- Outbox relay is the only process that enqueues to BullMQ.
- Consumers must ACK only after durable status write.
- Retry policy: capped attempts + exponential backoff.
- DLQ replay requires explicit operator action with audit log.
- Queue configuration env vars are required in production startup.

## Rollout Plan

1. Introduce shared envelope schema package and producer module.
2. Add outbox table/relay and idempotency constraints.
3. Switch submit path from DB-only to outbox+queue.
4. Wire worker status callbacks for deterministic lifecycle updates.
5. Enable queue health/readiness checks and operational runbook.

## Ticket Traceability

- `WS00-005`
- `F01-001` to `F01-015`
- `F11-001` to `F11-010`
- `OBS-001` to `OBS-010`

## References

- Internal:
  - `apps/isis/generation-api/src/services/job.service.ts`
  - `apps/isis/generation-api/src/app.ts`
  - `docs/domains/isis/isis-domain-completeness-audit-2026-02-21.md`
- External:
  - [BullMQ Docs](https://docs.bullmq.io/)
  - [BullMQ Idempotent Jobs Pattern](https://docs.bullmq.io/patterns/idempotent-jobs)
