Domain · Architecture

OpenAPI Domain — Architecture

domain API servers.

7sections13 minread1diagrams
Markdown source
On this page

Architectural overview of @oshun/openapi: the platform's single source of truth for REST API contracts — a typed spec registry, a Zod-driven spec/client generation pipeline, and a set of CI gates that keep specs, generated TypeScript types, and typed clients from ever drifting apart.

@oshun/openapi is the contract layer for every REST API in the Oshun platform. It lives at libs/openapi/ and exists to solve one structural problem: in a monorepo with more than a dozen API-owning domains (Lilith, Yemaya, Isis, Sophia, Hathor, Bellona, Nyx, Calliope, Concordia, Metis, the Oshun V1 domains, and the V2/V3 surfaces), if each team hand-maintained its own OpenAPI YAML with no shared tooling, the specs would drift silently from the handlers that implement them, from the TypeScript types that consumers import, and from each other. This library gives all of those contracts a single home, a typed catalog keyed off a small metadata model, and a fully automated generate-validate-and-gate pipeline.

The crucial architectural fact — and the one most often gotten wrong — is that @oshun/openapi is a build-time and CI tooling artifact, not a runtime dependency. Production services never load YAML spec files at request time. They import the TypeScript types and the typed fetch clients that this library generates from those specs. package.json declares "private": true, "type": "module", version 0.1.0; the only thing it ships into other packages at runtime are the generated src/generated/*.ts type modules and the generated src/v3-clients/ fetch clients (exposed through the ./generated and ./v3-clients subpath exports). The single runtime exception is the V2 Companion public API, which contracts against the registry to keep its published spec and its live /api/v2 endpoints pinned together (covered under cross-domain boundaries below).

A new engineer should hold three mental layers:

  1. The library core (src/) — the typed registry, the loader utilities, and the Oshun V1 spec builder. This is the importable, unit-tested surface.
  2. The tooling layer (scripts/) — eleven tsx scripts that validate specs, detect breaking changes, detect spec↔type drift, generate specs and clients from canonical Zod sources, generate TypeScript types from specs, and build HTML docs. Each has a paired *:check mode that is the CI gate.
  3. The artifacts (src/specs/, src/generated/, src/v3-clients/, v2/, docs/) — the YAML specs (hand-authored and generated), the generated TypeScript type modules, the generated V3 tenant clients, the V2 companion spec, and the static documentation site.

Where It Sits in the Stack#

@oshun/openapi sits above the canonical contract sources and below the domain API servers. It draws raw material from four workspace libraries and emits consumable artifacts for the rest of the platform.

Its four workspace dependencies are, exactly (package.json):

  • @oshun/persistence — the V1 object/enum persistence contracts (V1_OBJECT_PERSISTENCE_CONTRACTS, V1_ENUM_PERSISTENCE_CONTRACTS) and the ContractPersistenceDomain union. These Zod contracts are the source for the six V1 domain specs and the BFF spec.
  • @oshun/contracts (via the /v3 subpath) — buildV3OpenApiDocument, V3_CONTRACT_REGISTRY, getV3ContractsForTenant, and the V3 fixtures. Source for v3.yaml and the four V3 tenant clients.
  • @calliope/core — CalliopeOpenApiComponentSchemas (a Record<string, z.ZodTypeAny>). Source for calliope-api.yaml.
  • @oshun/domain-registry — DOMAIN_REGISTRY (display names, shell narratives, BFF base paths), used to title and tag the generated V1 specs.

Its consumers are the domain API servers (which import @oshun/openapi/generated types and the typed clients), the CI pipeline (which runs the *:check gates), and the developer portal / docs site (which renders the generated docs/ output). The dependency arrow always points from consumers toward this library; the library never imports a domain's business logic.

flowchart TD subgraph Sources["Canonical Zod sources"] PERS["@oshun/persistence\nV1 object/enum contracts"] CONTR["@oshun/contracts/v3\nbuildV3OpenApiDocument"] CALL["@calliope/core\nComponentSchemas"] DREG["@oshun/domain-registry\nDOMAIN_REGISTRY"] end subgraph Gen["Generators (scripts/)"] V1SPEC["generate-oshun-v1-specs"] V1CLI["generate-oshun-v1-api-clients"] CALSPEC["generate-calliope-spec"] V3SPEC["generate-v3-spec"] V3CLI["generate-v3-clients"] TSGEN["generate (openapi-typescript)"] end subgraph Artifacts["Committed artifacts"] YAML["src/specs/*.yaml\n+ v2/companion.yaml"] HAND["hand-authored\n<domain>-api.yaml, main.yaml"] GENTS["src/generated/*.ts"] V3C["src/v3-clients/*"] DOCS["docs/ (Redoc)"] end subgraph Core["Library core (src/)"] REG["utils/registry.ts\nSPEC_PATHS, SPEC_REGISTRY"] LOAD["utils/loader.ts\nloadSpec / mergeSpecs"] BUILD["oshun-v1/spec-builder.ts"] end subgraph Gates["CI gates (*:check)"] VAL["validate.ts"] DIFF["diff.ts"] DRIFT["drift-check.ts"] VV3["validate-v3-spec.ts"] end PERS --> BUILD --> V1SPEC --> YAML PERS --> V1CLI DREG --> BUILD CONTR --> V3SPEC --> YAML CONTR --> V3CLI --> V3C CALL --> CALSPEC --> YAML HAND --> YAML YAML --> TSGEN --> GENTS YAML --> LOAD YAML --> REG YAML --> DOCS YAML --> VAL YAML --> DIFF YAML --> DRIFT GENTS --> DRIFT YAML --> VV3 GENTS --> SERVERS["Domain API servers\n(@oshun/openapi/generated + clients)"] V3C --> SERVERS

Library Core (src/)#

The importable surface is a small barrel (src/index.ts) re-exporting three groups: loader utilities, the registry, and the V1 spec builder. There is no deep machinery here — the core is deliberately thin so that the heavy logic lives in the scripts and the canonical Zod sources.

Registry — src/utils/registry.ts#

The registry is the typed catalog that answers "what APIs exist, who owns them, and where are their spec files?" without anyone grepping the tree. It has two const sources of truth and a handful of helpers.

  • ApiDomain is a string-literal union (not a runtime enum) of every Oshun API domain: tara, arete, veritas, lilith, yemaya, isis, sophia, hathor, bellona, calliope, nyx, nisaba, metis, v2, v3, oshun-bff, shared. Each carries an inline source comment describing its scope. A union means the compiler checks domain strings with zero runtime cost.
  • SPEC_PATHS is declared as const, so keys and string values are typed literals — this prevents path typos in loaders, validators, and generators. It mirrors the domain hierarchy: top-level keys per domain, nested objects for per-contract/per-feature keys.
  • SPEC_REGISTRY is a Record<string, SpecMetadata> registering 20 specs. Each SpecMetadata carries name, version, description, specPath (a SPEC_PATHS value), domain, tags, basePath, and an optional deprecated. No entry currently sets deprecated.
  • Helpers: getSpecsByDomain(domain) filters registry values by domain; getSpecsByTag(tag) filters by tag membership (enabling cross-cutting views like "every astronomy spec"); getDomainPaths(key) returns all spec paths under a SPEC_PATHS key. extractEndpoints(spec) walks paths across the seven HTTP methods into flat { method, path, operationId, tags, summary } descriptors; extractSchemas(spec) flattens components.schemas into { name, type, description }, joining OpenAPI-3.1 array types with |.

Loader — src/utils/loader.ts#

The loader is the only code that touches spec files on disk, so callers never construct paths. resolveSpecFilePath applies three rules: absolute paths pass through; a v2/v2/... path resolves against the package root (libs/openapi/); everything else resolves against SPECS_DIR (src/specs/). loadSpec/loadSpecSync read and YAML.parse into an OpenAPIV3_1.Document; listSpecs recursively scans src/specs/ and v2/ (both guarded by fs.existsSync); mergeSpecs shallow-merges paths and the five components sub-maps (schemas, responses, parameters, requestBodies, securitySchemes) and de-duplicates tags by name — the mechanism for building a single browsable document for the portal.

V1 Spec Builder — src/oshun-v1/spec-builder.ts#

This module is the heart of the generation story. It deterministically derives OpenAPI 3.1 documents and typed-client resource descriptors directly from the @oshun/persistence Zod contracts plus @oshun/domain-registry metadata — keeping the builder logic separate from the scripts that invoke it so it is independently unit-testable.

For each of the six V1 domains (OSHUN_V1_API_DOMAINS = ['tara', 'arete', 'veritas', 'nyx', 'nisaba', 'metis']), buildContractPaths turns each object contract into a five-operation REST resource — list/create/get/upsert/tombstone (GET/POST/GET/PUT/DELETE) — with operation-ID stems built from an optional Bff prefix, the PascalCase domain, and the contract name. buildComponents emits the shared envelope every spec carries: a bearerAuth HTTP-bearer-JWT scheme; four reusable parameters (SourceRecordIdParam, LimitParam 1–250/default 50, CursorParam, IncludeTombstonesParam); six reusable error responses (BadRequest, Unauthorized, Forbidden, NotFound, Conflict, InternalError); and three fixed envelope schemas (ErrorResponse, PageInfo, TombstoneResponse) plus the per-contract schemas. Each Zod schema is converted with z.toJSONSchema(schema, { io: 'input', unrepresentable: 'any' }) and then recursively stripped of $schema keys so it embeds cleanly into components. buildBffSpec() combines all six domains under their bff-base-path plus a /api/oshun/shared group from the cross-cutting contracts, using domain-prefixed schema names (TaraRitualTemplate, SharedEvidencePack) to keep the combined namespace unambiguous.

Tooling Layer (scripts/)#

Eleven tsx scripts implement the validation and generation pipeline. They fall into three roles: validators, change-detectors, and generators. Each of the five canonical-source generators (generate-calliope-spec, generate-oshun-v1-specs, generate-oshun-v1-api-clients, generate-v3-spec, generate-v3-clients) has a --check/:check variant that rebuilds the artifact in memory, compares it byte-for-byte to what is committed, and exits non-zero on any difference — that is the mechanism that makes "single source of truth" enforceable. (The umbrella generate and generate-docs orchestrators run the generators but carry no :check gate of their own.)

Validators. validate.ts is a self-contained OpenAPI 3.1 structural validator with zero external schema-library dependency — it field-checks openapi/info, paths and parameters, response codes, schemas, security schemes, and broken #/components/schemas refs, treating a duplicate operationId as a hard error. (validate runs --domain-only; validate:all covers every file.) validate-v3-spec.ts is a semantic validator: it loads v3.yaml and, for every contract in V3_CONTRACT_REGISTRY, asserts the contract's fixture validates both as a POST request body and as a 200 GET response body via the oas3-validator package. A separate Nx target, openapi:validate, runs the external @redocly/cli lint over src/specs/.

Change detectors. diff.ts compares seven tracked domain specs against a git base ref (git show <base>:...) and flags five breaking-change categories (path-removed, operation-removed, parameter-required, response-removed, and a reserved schema-breaking); path matching is template-aware, so renaming a path parameter is not breaking, and removing a non-2xx response is not breaking. drift-check.ts exports a testable detectDrift that parses each YAML into a SpecModel and the matching generated .ts (by regex) into the same model, then reports any path / operationId / schema present in one but not the other, plus a header check that the generated file still points at its source YAML — across eleven DOMAIN_MAPS pairs.

Generators. generate-oshun-v1-specs.ts and generate-oshun-v1-api-clients.ts drive the V1 spec builder; generate-calliope-spec.ts builds calliope-api.yaml from @calliope/core; generate-v3-spec.ts and generate-v3-clients.ts build v3.yaml and the four V3 tenant clients from @oshun/contracts/v3; generate.ts is the umbrella type generator; generate-docs.ts builds the Redoc site.

The umbrella generate.ts — and what it does not do#

generate.ts (the generate script / openapi:gen Nx target) runs, in order: (1) generate-oshun-v1-specs.ts, (2) generate-oshun-v1-api-clients.ts, (3) generate-calliope-spec.ts, then (4) openapi-typescript over every .yaml/.yml in src/specs/ and v2/, plus (5) the two EXTERNAL_SPECS owned by other packages (services/metis/openapi/metis.openapi.json → metis.ts and apps/oshun/bff/openapi/oshun-bff.openapi.yaml → oshun-bff.ts; a missing external spec is skipped with a warning). Output naming: a v2/ spec becomes v2-<name>.ts; a <name>-api.yaml becomes <dirname>.ts; otherwise the base name is kept.

The non-obvious gotcha: generate.ts does not run the V3 generators. V3 spec and client generation is a separate generate:v3 (generate-v3-spec.ts && generate-v3-clients.ts). So generate.ts runs openapi-typescript over whatever v3.yaml is currently on disk — a full regeneration from scratch requires running generate:v3 and generate. This ordering matters when changing V3 contracts.

Artifacts and Their Authoring Provenance#

The src/specs/ tree holds one spec per implemented domain (there are no per-feature files like lilith/chat.yaml — those keys in SPEC_PATHS are forward declarations, see below). Specs fall into three authoring categories, and the distinction is load-bearing because the *:check gates only police the generated ones:

  1. Hand-authored YAML — main.yaml (the legacy consolidated "AI Wisdom Platform API"), the seven <domain>-api.yaml files (lilith/yemaya/isis/sophia/hathor/bellona/nyx), concordia/concordia-api.yaml (Phase 179, 11 operations over the cooperative-mediation lifecycle), and v2/companion.yaml. Edited directly; policed only by the structural validator.
  2. Generated by the V1 spec builder — the six <domain>-v1-contracts.yaml files plus oshun-bff-v1-contracts.yaml, each carrying the # This file is generated by …generate-oshun-v1-specs.ts. header.
  3. Generated by dedicated generators — calliope-api.yaml and v3.yaml, each with its own "do not edit" header.

The generated TypeScript lives in src/generated/ (one module per spec, plus a barrel index.ts that export * as <domain> and aliases each paths type as <Domain>Paths). The generated V3 clients live in src/v3-clients/: a shared base.ts (the V3Fetch transport, V3ClientOptions, the V3ApiError class carrying status/code/requestId/body, and v3JsonRequest<T> which falls back to globalThis.fetch, sets the JSON headers, and throws V3ApiError including the x-request-id header on non-OK), plus four tenant modules (lilith-platform, tara-studio, saraswati-stage, lilith-commons), each exposing a get<Contract>(id) / create<Contract>(input) pair per V3 contract.

Invariants, Failure Modes, and Extension Points#

The single-source-of-truth invariant. A committed generated artifact must be byte-identical to a fresh regeneration from its canonical Zod source. This is the whole point of the *:check mode: generate-v3-spec.ts --check, for example, re-renders the document and throws "V3 OpenAPI spec is out of date" if disk differs. The gate scripts intended to enforce the domain's health are validate:all, diff:check, drift:check, the four generate:*:check gates (calliope, oshun-v1-specs, oshun-v1-clients, v3 — the last being v3-spec:check && v3-clients:check), and validate:v3. Each rebuilds-and-diffs; none writes files. (These are the gate scripts; wiring them into a specific CI workflow is by convention — treat the scripts, not a particular pipeline file, as the contract.)

Common failure modes.

  • Drift — editing a YAML spec without regenerating its .ts (or vice versa): caught by drift:check. The most common cause is hand-editing a generated spec, which both drift:check and the relevant generate:*:check reject.
  • Silent breakage — removing a path, operation, or 2xx response, or making a parameter required: caught by diff:check against the base ref.
  • Broken refs / duplicate operation IDs — caught by the structural validator.
  • Forgetting the two-step V3 regeneration — because generate.ts consumes the on-disk v3.yaml, a stale V3 spec silently produces stale v3 types unless generate:v3 is run first.

Forward-declared keys (planned, not implemented). SPEC_PATHS declares many fine-grained keys whose files do not exist on disk — the entire lilith.*, yemaya.*, isis.* (fine-grained), sophia.*, hathor.*, bellona.*, nyx.objects/ephemeris/events/satellites, and the whole shared.* group. The registry.ts source comments this region "to be created during domain migrations." Consequently, seven SPEC_REGISTRY entries point at files that do not yet exist: auth/users/health (the shared/* specs) and nyx-objects/nyx-ephemeris/nyx-events/nyx-satellites. These are deliberate forward declarations that keep the typed key set and portal catalog stable ahead of the files; the loader will resolve them once the YAML lands. Conversely, isis-api.yaml exists on disk but is not registered, so getSpecsByDomain('isis') correctly returns an empty array. Treat the registry as a roadmap-aware catalog, not a guarantee that every entry has a backing file.

metis.api is an external path. SPEC_PATHS.metis.api points at services/metis/openapi/metis.openapi.json — a JSON spec owned by the Metis service, not a YAML under src/specs/. generate.ts reads it (and the BFF external spec) via EXTERNAL_SPECS, skipping with a warning if absent.

Extension points. Adding a new V1 domain is data-driven: extend the @oshun/persistence contracts and the OSHUN_V1_API_DOMAINS tuple, and the builder produces the spec, the BFF paths, and the typed client automatically. Adding a new V3 tenant means extending @oshun/contracts/v3 and the TENANT_FILES table in generate-v3-clients.ts. Adding a hand-authored domain spec means dropping a <domain>-api.yaml under src/specs/<domain>/, adding a SPEC_REGISTRY entry, a drift-check.ts DOMAIN_MAPS pair, and (if it should be browsable) a generate-docs.ts DOMAINS entry.

Cross-Domain Boundaries#

The boundary is intentionally clean: this library governs the shape and contract of every API; the owning domain libraries govern the behavior. The generated types flow outward to every domain API server, which implements its routes to match — typically deriving route schemas from the same Zod schemas via @hono/zod-openapi so the implementation and the contract share one source.

The one runtime contract is the V2 Companion public API. The v2-companion spec is registered with basePath: '/api/v2' and resolved from libs/openapi/v2/companion.yaml. In apps/oshun/web, the framework-agnostic handler layer (api/v2/public-api.ts, defining V2_PUBLIC_API_BASE_PATH and the read-only handlers) and the thin Next.js App Router adapters (src/app/api/v2/.../route.ts) both pin to that document — the openapi/route.ts adapter re-serves the exact spec via SPEC_PATHS.v2.companion — so the published OpenAPI description and the live endpoints cannot drift apart.

Status Summary#

Fully implemented and tested: the registry, loader, and V1 spec builder (src/); all eleven tooling scripts and the five canonical-source *:check gates (scripts/); the generated type modules (src/generated/); the generated V3 tenant clients (src/v3-clients/); the hand-authored and generated spec inventory (src/specs/, v2/); the committed Redoc docs (docs/, nine domains); and three vitest suites (src/openapi.spec.ts, scripts/drift-check.spec.ts, src/v3-clients/__tests__/client.spec.ts).

Planned / roadmap-only: the fine-grained per-feature spec files named by the forward-declared SPEC_PATHS keys and the entire shared.* group, including the seven SPEC_REGISTRY entries (auth, users, health, and the four nyx-* catalog/ephemeris/events/satellites specs) whose YAML files are not yet on disk. These are honest forward declarations carried to stabilize the typed catalog ahead of the domain migrations that will land the files — not implemented endpoints.