# Shared Domain — Architecture

> Architectural overview of the `@oshun/*` shared infrastructure layer: 42
> libraries, dependency philosophy, and how domains consume shared
> infrastructure.

---

The shared domain (`libs/shared/`) is the platform's horizontal foundation. It
solves a common problem in multi-team monorepos: without a shared layer, each
domain team reinvents authentication, logging, error handling, and metrics in
slightly incompatible ways. The result is operational chaos — logs that can't be
correlated, errors that serialize differently per service, and security
primitives that drift out of sync.

The shared domain prevents that by owning every cross-cutting concern once. A
domain engineer should never write their own JWT verification, Redis connection
pool, or Pino configuration — they import from `@oshun/*` and stay focused on
domain logic.

Every `@oshun/*` library is a pure infrastructure or utility library. Business
logic, user-facing features, and domain-specific data models live in their
respective domain directories (`libs/isis/`, `libs/lilith/`, etc.).

---

## Role of the Shared Domain

The shared domain establishes four system-wide contracts:

1. **A consistent runtime contract** — every service logs the same way, traces
   the same way, handles errors the same way
2. **Infrastructure abstraction** — domains write to `@oshun/database`, not raw
   `pg`; they publish via `@oshun/event-bus`, not raw Redis pub/sub
3. **Security baseline** — authentication, authorization, rate limiting, and
   audit logging are infrastructure, not each domain's responsibility
4. **Operational visibility** — metrics, tracing, and health checks are wired in
   uniformly across all services

---

## Library Map

The 42 packages are organized into functional groups. The tree below shows the
directory name under `libs/shared/`, the npm package name, and a brief note on
what each package provides.

```
libs/shared/   (42 packages)
│
├── CORE TYPES AND ERRORS
│   ├── types/              → @oshun/types        (branded IDs, Result, pagination, envelopes)
│   ├── errors/             → @oshun/errors       (OshunError hierarchy, code registry)
│   └── config/             → @oshun/config       (env-var loading, Zod validation, flags)
│
├── AUTHENTICATION
│   ├── auth/               → @oshun/auth         (auth service, RBAC, lockout, OAuth helpers)
│   ├── auth-primitives/    → @oshun/auth-primitives (JWT, session, password, API key, TOTP)
│   └── identity/           → @oshun/identity     (verify-only JWT, gateway auth, mTLS)
│
├── DATA AND STATE
│   ├── database/           → @oshun/database     (PostgreSQL pool, query builder, migrations)
│   ├── cache/              → @oshun/cache        (Redis cache-aside, locks, circuit breaker)
│   ├── storage/            → @oshun/storage      (S3/MinIO, pre-signed URLs, multipart)
│   ├── queue/              → @oshun/queue        (BullMQ job queue, durable substrate, DLQ)
│   └── migration/          → @oshun/migration    (cross-domain migration framework)
│
├── COMMUNICATION
│   ├── event-bus/          → @oshun/event-bus    (Redis pub/sub, typed envelopes, topic registry)
│   ├── websocket/          → @oshun/websocket    (WebSocket server, rooms, Redis adapter)
│   └── http-client/        → @oshun/http-client  (fetch wrapper, circuit breaker, retry, SSRF)
│
├── OBSERVABILITY
│   ├── logging/            → @oshun/logging      (Pino, structured, context propagation)
│   ├── metrics/            → @oshun/metrics      (Prometheus, OpenTelemetry metrics)
│   └── tracing/            → @oshun/tracing      (OpenTelemetry traces, Jaeger/X-Ray)
│
├── PLATFORM SERVICES
│   ├── gateway/            → @oshun/traefik-config      (Traefik v3 config builders/generators)
│   ├── health/             → @oshun/health       (liveness/readiness probes, aggregator)
│   ├── rate-limit/         → @oshun/rate-limit   (sliding/fixed/token-bucket, quotas, abuse)
│   ├── security/           → @oshun/security     (audit log, content scanning, secrets)
│   └── service-discovery/  → @oshun/service-discovery (Redis registry, load balancing)
│
├── AI AND GPU
│   ├── ai/                 → @oshun/ai           (LLM providers, routing, prompts, local LLM)
│   ├── ai-advanced/        → @oshun/ai-advanced  (adapters, benchmarking, edge, research)
│   ├── gpu-dispatcher/     → @oshun/gpu-dispatcher (RunPod job dispatch, queuing, fallback)
│   └── runpod-client/      → @oshun/runpod-client  (typed RunPod Serverless API client)
│
├── SECURITY, AUDIT, AND COMPLIANCE
│   ├── crypto/             → @oshun/crypto        (noble-backed hashing/signing/AEAD/KDF)
│   ├── audit-platform/     → @oshun/audit-platform (canonical audit ingest, append-only store)
│   ├── data-residency/     → @oshun/data-residency (region/residency enforcement)
│   ├── region-rules/       → @oshun/region-rules  (V2 regional content rules)
│   └── review-persistence/ → @oshun/review-persistence (review-package persistence)
│
├── DOCUMENT AND MEDIA PROCESSING
│   ├── vision-llm/         → @oshun/vision-llm    (vision-locate wrapper over IsisLLMClient)
│   ├── layout-analyzer/    → @oshun/layout-analyzer (PubLayNet layout analysis, mAP eval)
│   ├── ocr/                → @oshun/ocr           (tiered OCR client; Tesseract/vision/cloud)
│   ├── ml/                 → @oshun/ml            (ONNX runtime, model loading, benchmark)
│   └── native-libs/        → @oshun/native-libs   (typed wrappers over pinned native deps)
│
├── INFRASTRUCTURE AND OPERATIONS
│   ├── infrastructure/     → @oshun/infrastructure (performance/security/monitoring managers)
│   ├── release-management/ → @oshun/release-management (rollback plans, rehearsal engine)
│   └── documentation/      → @oshun/documentation  (architecture-documentation models)
│
├── INTEGRATION SURFACES
│   ├── inbound-integrations/   → @oshun/inbound-integrations (LMS/OneRoster/calendar/BYOM/…)
│   └── tara-live-class-booking/ → @oshun/tara-live-class-booking (Tara live-class primitives)
│
└── DEVELOPER TOOLS
    └── testing/            → @oshun/testing      (mocks, fixtures, test containers, fuzz)
```

A few important clarifications about packages that are sometimes
mischaracterized:

> `@oshun/event-bus` uses **Redis pub/sub** (`ioredis`), not Kafka. Its
> at-least-once replay source is a TTL-bounded Redis key plus a per-event
> processed-marker hash — there is no Kafka topic and no transactional outbox
> worker. `@oshun/release-management` implements rollback-plan validation and
> rehearsal for six v1 surfaces; it is not a versioning/changelog/beta-program
> tool. `@oshun/documentation` implements architecture-documentation models
> only; API-reference generation, tutorials, and certification are not present
> in source. `@oshun/errors` serializes via `toJSON()` / `toResponse()` — there
> is no RFC 7807 (`toProblemDetails`) serializer.

---

## Dependency Hierarchy

The shared domain sits at the very bottom of the monorepo's internal dependency
graph. No `@oshun/*` library may depend on any domain library. Arrows flow only
downward, ensuring domain teams can upgrade shared libraries without circular
rebuild problems.

```
All domain libraries (@isis/*, @lilith/*, @yemaya/*, ...)
        │
        ▼
@oshun/* shared libraries
        │
        ▼
npm packages (pg, ioredis, bullmq, pino, prom-client, @opentelemetry/*,
              @noble/*, ws, AWS SDK S3, etc.)
        │
        ▼
Infrastructure (PostgreSQL, Redis, S3-compatible storage, RunPod)
```

A few `@oshun/*` packages depend on `@oshun/contracts` (the shared Zod contract
package) — specifically `audit-platform`, `data-residency`, `layout-analyzer`,
`review-persistence`, and `vision-llm`. These remain horizontal:
`@oshun/contracts` is itself a shared contract package, not a domain library.

---

## Service Integration Pattern

Every Oshun domain service follows the same four-phase startup sequence when
integrating shared infrastructure. The pattern ensures that configuration is
validated before anything connects, and that all observability and security
middleware is installed consistently across every service.

```typescript
// 1. Load configuration
//    @oshun/config validates all env vars at startup, failing fast with a clear
//    error rather than silently using a zero-value or undefined.
const config = loadConfig();

// 2. Initialize shared infrastructure
//    Each client is created against the validated config. Connection pooling,
//    circuit breakers, and retry policies are all wired here.
const db = createPostgresClient(config.database);
const cache = createRedisClient(config.redis);
const eventBus = createEventBus({
  redisUrl: config.redisUrl,
  sourceDomain: 'isis',
});
const logger = createLogger({ service: 'isis-api' });
const metrics = createMetricsServer({ port: 9090 });
const tracer = createTracer({ service: 'isis-api' });

// 3. Build application
//    Auth, rate limiting, logging, and tracing middleware are registered once
//    at the application level — domain route handlers do not need to re-install them.
const app = new Hono();
app.use(authMiddleware()); // from @oshun/auth
app.use(rateLimitMiddleware()); // from @oshun/rate-limit
app.use(loggingMiddleware()); // from @oshun/logging
app.use(tracingMiddleware()); // from @oshun/tracing
app.get('/health', healthHandler(db, cache)); // from @oshun/health

// 4. Register domain routes
//    Domain logic receives the already-initialized infrastructure clients as
//    constructor arguments, keeping domain code free of infrastructure concerns.
app.route('/api/v1', domainRoutes(db, cache, eventBus));
```

---

## Multi-Database Architecture

Each domain has its own isolated PostgreSQL database to prevent accidental
cross-domain data coupling. A bug in one domain's migrations cannot corrupt
another domain's schema, and each domain can scale its database independently.

`@oshun/database` and `@oshun/config` provide the connection tooling; domains
own their schemas entirely. The core domain-isolated databases used in local
development are:

| Database    | Domain                          |
| ----------- | ------------------------------- |
| `oshun_dev` | Shared (users, sessions, audit) |
| `yemaya`    | Yemaya                          |
| `lilith`    | Lilith                          |
| `isis`      | Isis                            |
| `sophia`    | Sophia                          |
| `hathor`    | Hathor                          |
| `bellona`   | Bellona                         |

Additional domains carry their own `<DOMAIN>_DATABASE_URL` connection strings in
the root `.env`. Each service loads its database config with
`loadDatabaseConfig()` from `@oshun/config`; there is no per-domain loader
function — all domain databases use the same loader with the appropriate
connection-string environment variable.

Cross-domain data references use `userId` (a shared identifier) rather than
foreign keys across database boundaries. This is the intentional isolation
boundary: domain databases contain only domain-specific tables.

---

## Event Bus Architecture

The event bus solves inter-domain communication without tight coupling. When
Isis generates an asset, it publishes a typed event. Sophia, Yemaya, and the
shell can subscribe to that event independently, and Isis knows nothing about
its subscribers.

The transport is Redis pub/sub with a TTL-bounded durability layer — not Kafka
and not a transactional outbox. The flow looks like this:

```
Domain Service A                    Domain Service B
      │                                   │
      │ publish('isis.asset.generated')   │
      ▼                                   │
@oshun/event-bus                          │
      │                                   │
      ├── SETEX  oshun:events:event:<id>  │  (TTL-bounded replay copy)
      └── PUBLISH oshun:events:channel:isis.asset.generated
                  │                       │
                  └────────────────────── ▼
                                    @oshun/event-bus
                                    PSUBSCRIBE 'isis.asset.*'
                                    consumer group claims via SET NX EX
                                    (one member runs the handler)
```

Delivery is **at-least-once**. The durability mechanism works in four steps:

1. `publish()` stores a TTL-bounded copy of the event under
   `oshun:events:event:<id>` (default TTL 24 h) before fanning it out over Redis
   pub/sub. This is the replay source.
2. A per-event Redis HASH records which subscriptions and consumer groups have
   `ack`ed. This is what makes replay safe — already-processed events are not
   re-delivered.
3. `replayUnacked()`, called on boot after wiring subscriptions, scans the
   persisted events and redelivers any that still lack a processed marker —
   recovering work lost to a crash between publish and ack.
4. Delayed and `nack`ed events live in a durable Redis sorted set
   (`oshun:events:scheduled`) drained by a scheduler loop every 250 ms, so retry
   state survives a process restart.

---

## Observability Architecture

All three observability signals flow through shared libraries to a common
backend. This means every service's logs, metrics, and traces can be correlated
in a single dashboard — a feature that would be impossible if each team chose
their own logging framework.

```
Every Domain Service
      │
      ├── @oshun/logging → Pino JSON → Log aggregator (Loki/ELK)
      │
      ├── @oshun/metrics → Prometheus /metrics → Prometheus scraper → Grafana
      │
      └── @oshun/tracing → OTLP → Jaeger (dev) / AWS X-Ray (prod)
```

Trace spans carry OpenTelemetry semantic-convention attributes so Jaeger can
build accurate service maps and identify bottlenecks by operation type:

- `service.name`: identifying the specific service
- `http.method` and `http.url` for HTTP spans (`HttpSpanAttributes`)
- `db.system` and `db.statement` for database spans (`DbSpanAttributes`)
- RPC attributes for service-to-service calls (`RpcSpanAttributes`)
- correlation-id and user-context attributes where available

---

## Security Architecture

Security controls are layered at the platform level so a domain engineer cannot
accidentally skip authentication or rate limiting on a new route. Each layer has
a specific responsibility and passes the request to the next layer only on
success.

```
Inbound request
      │
      ▼
Traefik gateway  (config generated by @oshun/traefik-config)
  - TLS termination
  - routing per generated dynamic config
      │
      ▼
Auth Middleware (@oshun/auth / @oshun/identity)
  - JWT signature validation
  - Token expiry check
  - Role and scope extraction
  - mTLS peer verification for service-to-service calls (@oshun/identity)
      │
      ▼
Rate Limit Middleware (@oshun/rate-limit)
  - Per-user sliding window
  - Per-IP / per-API-key limits, abuse controls
      │
      ▼
Request Handler
  - Domain business logic
      │
      ▼
Audit Log (@oshun/security and @oshun/audit-platform)
  - Every privileged action recorded
  - Append-only, tamper-evident
```

---

## Testing Architecture (`@oshun/testing`)

Tests are structured in two tiers: unit tests using in-memory mocks for
everything, and integration tests using real Docker-backed infrastructure. The
split allows the unit test suite to run in milliseconds with no external
dependencies while still exercising real database and cache behavior in the
integration suite.

**Unit tests** use in-memory mocks for all infrastructure:

- `@oshun/testing` provides `createMockDatabaseClient()`,
  `createMockRedisClient()`, `createMockHttpClient()`, `createMockLogger()`,
  `createMockEventEmitter()`, and `createMockTimers()`.
- Event-driven logic is asserted via `MockEventEmitter`, which records emitted
  events for assertion. The `@oshun/event-bus` package also ships in-memory-
  friendly behaviour, and `@oshun/queue` ships `MemoryQueue` / `MemoryWorker`.
- Fixtures come from `createFixtureFactory` and the user/content factories,
  producing realistic test data with configurable overrides.
- `createVitestConfig` / `createServiceVitestConfig` /
  `createIntegrationVitestConfig` produce consistent Vitest configs with
  enforced coverage thresholds.

**Integration tests** use real infrastructure:

- `docker/docker-compose.dev.yml` starts PostgreSQL, Redis, MinIO, and Mailpit
  (Kafka and other engines are optional Compose profiles).
- `@oshun/testing` containers (`PostgresTestContainer`, `RedisTestContainer`,
  `ContainerManager`) manage Docker container lifecycle for tests that need real
  external services, starting and stopping containers around the test suite.

**Planned verification strategy (Phase 13, Phase 18.12)** — beyond the two
implemented tiers, the roadmap assigns Shared the platform-wide verification
envelope:

- **End-to-end testing** — Playwright suites for the web shells and Detox for
  the React Native mobile apps, run against production builds.
- **Consumer-driven contract testing** — Pact (or equivalent) contracts between
  domain APIs and their consumers, verified in CI on both sides, so a provider
  cannot break a consumer without a failing contract build.
- **Performance and load testing** — load baselines per public API with
  regression gates, plus soak tests for long-running workers.
- **Security testing** — dependency and container scanning, SAST (CodeQL), and
  periodic dynamic scans of exposed surfaces.
- **Boundary validation** — automated Nx module-boundary checks
  (`@nx/enforce-module-boundaries`) and dependency-graph audits, enforcing the
  tag taxonomy described under Workspace Foundation below.
- **Coverage expansion** — a coordinated strategy to raise coverage across
  domains with thresholds enforced by the shared Vitest config factories.

---

## Workspace Foundation (Phases 0–2)

The Shared domain owns the monorepo's foundational tooling — the layer every
other domain builds on but must never re-implement:

- **Package management** — a single pnpm workspace (`pnpm-workspace.yaml`) with
  a central dependency catalog: packages reference `catalog:` versions so the
  workspace pins each third-party dependency exactly once. `.npmrc` uses the
  hoisted node-linker, which the workspace's phantom-dependency profile depends
  on. Turbo was removed in favor of Nx as the single task runner.
- **Nx workspace** — `nx.json` defines the task runner, caching, and affected
  configuration. Projects carry a tag taxonomy (`scope:<domain>`,
  `type:app|lib|e2e`, `layer:*`) enforced by `@nx/enforce-module-boundaries`
  depConstraints in the root ESLint config: domain code may depend on Shared and
  Contracts, never on another domain's internals.
- **TypeScript configuration** — `tsconfig.base.json` holds the compiler
  baseline and the `@oshun/*` / `@<domain>/*` path aliases that make library
  imports resolve from source across the workspace.
- **Code quality gates** — root ESLint and Prettier configs, plus husky-driven
  git hooks (staged-file typecheck/lint, commitlint conventional-commit
  enforcement, stub-indicator scan) run before every commit.
- **Architecture decision records (Phase 0)** — the monorepo-level decisions
  (git strategy, package-manager choice, single-provider identity, event-bus
  strategy, Zod+OpenAPI+Proto contract approach, versioning/release policy) and
  the consolidated data-ownership matrix: each domain owns its own database
  (`oshun_dev`, `yemaya`, `lilith`, `isis`, `sophia`, `hathor`, `bellona`, …)
  and no domain reads another domain's tables directly — cross-domain data moves
  over events and APIs only.

---

## Infrastructure and Delivery Topology (Phases 12, 18, 19)

Shared also owns the deployment and delivery trees (`docker/*`, `deploy/*`,
`infra/*`) and the platform CI/CD posture:

- **CI/CD (Phase 12.1)** — consolidated GitHub Actions workflows (`ci.yml`,
  `deploy.yml`) built on Nx-affected pipelines so pull requests build and test
  only what they touch, with CodeQL security analysis and accessibility/
  performance gates on web surfaces. A shared setup composite action and the Nx
  cache keep per-PR cost bounded.
- **Container strategy (Phase 12.2)** — base Dockerfiles for services, web apps,
  and workers that domain images extend, so patching a base image patches the
  fleet.
- **Orchestration and IaC (Phases 12.3–12.6, 19)** — the compute target is **AWS
  ECS on Fargate** (clusters, task definitions, and services for domain APIs and
  workers) with Terraform as the IaC source of truth under
  `infrastructure/terraform`; Kubernetes manifests and Helm charts remain for
  the services still deployed to K8s (`infra/k8s/*`, `deploy/helm/*`). GPU
  workloads dispatch to **RunPod Serverless** via `@oshun/runpod-client` and
  `@oshun/gpu-dispatcher` rather than resident GPU nodes.
- **Observability stack (Phase 12.7)** — beyond the in-process libraries
  (`@oshun/metrics`, `@oshun/tracing`, `@oshun/logging`), Shared owns the hosted
  stack: Prometheus scraping, Grafana dashboards per domain, Jaeger tracing, and
  centralized log aggregation (Loki/ELK).
- **Secrets management (Phase 12.8)** — a central secrets store (Vault or the
  cloud-native equivalent) feeding runtime configuration; secrets never live in
  the repository or images.
- **Production readiness and operations (Phases 18, 19.10–19.11)** — the
  critical-blocker remediation posture for shared libraries (connection pooling,
  retry/backoff, graceful shutdown, health probes on every service), deployment
  runbooks per service, and the decommissioning procedure for superseded
  infrastructure so replaced stacks are torn down rather than left running.

The Neith-stack counterparts of this layer (`@neith/cloud`,
`@neith/observability`, `@neith/security`, `@neith/testing`, `@neith/qa`,
`@neith/docs`, `@neith/training`, `@neith/release` — Phases 49–52) are owned by
Neith and documented in `DOMAINS/neith/features.md`.