The foundation layer is the shared substrate every V1 surface stands on: the contracts that define each object once, the persistence machinery that keeps those contracts and the database in lockstep, the BFF middleware chain that guards every write, and the queue/event/integration/identity plumbing that makes the customer-facing domains and product surfaces interoperable. It serves engineers across every domain — none of them re-implement idempotency, residency, audit, or eventing — and it sits underneath the High-Level Architecture, between the domain adapters and the raw infrastructure. The work is tracked in TODOS § 1, and almost every piece below is real code with tests already in the repo. Where something is contract-level rather than a live deployment, this page says so.
Status candor. This area is overwhelmingly implemented, not aspirational. Contracts, the persistence drift/index/migration/tombstone test quartet, the OAuth 2.1/PKCE module, the role/scope model, the partial-failure envelope contract, the durable queue + DLQ + SLA monitor, the topic-registry + outbound-delivery + webhook-simulator event bus, the inbound connectors, the data-residency enforcer, the idempotency middleware, and the audit-platform hash chain all exist with tests. The two honestly-scoped caveats: the OAuth module is contracts/state-machine level (token types, PKCE validation patterns, refresh/revocation surface) and does not stand up a live authorization-server deployment. The BFF request-lifecycle diagram below describes a real composition of building blocks that are all present, but no single production route was traced end-to-end proving every middleware fires in exactly that order.
1. Contracts — libs/contracts/#
V1 holds to one source of truth per object class. Each contract is a Zod schema with a contract-test fixture that round-trips a known-good payload, so a schema change that breaks an existing object fails CI rather than silently shipping.
- Common contracts live under
libs/contracts/src/common/. That directory now contains 285 files — 161 implementation.tsmodules plus their co-located.spec.ts/.test.tssiblings — covering admin, billing, incident, persona, voice, persistence, and many more cross-cutting object classes. (Earlier drafts of this doc cited "≈265 files," which was stale and low.) - Domain contracts live under
libs/contracts/src/<domain>/. - Everything re-exports through
libs/contracts/src/index.tsas@oshun/contracts. To avoid naming collisions across families, several domains are exported as namespaces rather than flat symbols. The index usesexport * asforNisabaContracts(./nisaba),MetisContracts(./metis), andVeritasContracts(./veritas) — beyond the three customer domains, also forV3Contracts(./v3/index),V6Contracts(./v6/index),V9Contracts(./v9/index), andLivingSceneContracts(./living-scene/index). Where a generic name would collide, the index re-exports under an explicit alias — e.g.ScoreSchema as LivingSceneScoreSchemaanddeepParseScore as deepParseLivingSceneScore— so consumers can import a Living Scene score without shadowing another domain'sScoreSchema. - Specialized contract packages exist for Veritas, Psyche, Iris, Maat, Cybele, Concordia, and Saraswati when a consumer needs a smaller surface than the umbrella package.
The partial-failure envelope contract#
The standardized partial-failure envelope (§5, below) is not a convention — it
is a first-class Zod contract at
libs/contracts/src/common/partial-failure-envelope.ts. Its exported surface:
| Symbol | What it is |
|---|---|
PartialFailureEnvelopeSchema |
The base schema: { results: unknown[], errors: PartialFailureError[], partial: boolean }. |
PartialFailureErrorSchema / PartialFailureError |
A single failure: { domain: string, stage?: string, message: string } (each string non-empty; stage optional). |
createPartialFailureEnvelopeSchema(resultSchema, errorSchema?) |
Generic factory that produces a typed envelope schema for a specific result/error shape, defaulting the error schema to PartialFailureErrorSchema. |
buildPartialFailureEnvelope({ results, errors, partial? }) |
Constructs a well-formed envelope; throws if a supplied partial flag disagrees with whether errors is non-empty. |
isPartialFailureEnvelope(value) |
Structural type guard. |
The base schema's cross-field refinement is what makes it trustworthy: a
superRefine enforces that partial: true requires at least one error and
partial: false requires an empty errors array. An envelope that claims
to be partial but carries no errors — or claims success while carrying errors —
is rejected at parse time:
PartialFailureEnvelopeSchema.parse({
results: [
/* 8 ok */
],
errors: [{ domain: 'metis', stage: 'grading', message: 'timeout' }],
partial: true, // valid: partial=true ⇒ errors non-empty
});
This is why the BFF can normalize a fan-out across several domains into one response that is honest about which slices succeeded — the contract makes a lie unrepresentable.
2. Adapters — libs/oshun/domain-* and substrate adapters#
-
Every customer-facing domain exports one canonical adapter (the typed read APIs the shared shell, admin, and assistant consume) and one adapter variant for boundary-specific concerns (BFF routes, fixtures).
-
Substrate adapters present the same shape regardless of whether the underlying substrate is local-only or distributed, so callers don't branch on deployment topology.
-
Adapters are wired through the BFF via the domain registry. The registry at
libs/oshun/domain-registry/src/registry.tsexportsOSHUN_DOMAIN_IDSandDOMAIN_REGISTRY, and pins the shell's primary domain viaOSHUN_SHELL_PRIMARY_DOMAIN = 'tara'. Each of the six customer domains carries a'bff-base-path':Domain bff-base-pathTara /api/oshun/domains/taraVeritas /api/oshun/domains/veritasNyx /api/oshun/domains/nyxArete /api/oshun/domains/areteNisaba /api/oshun/domains/nisabaMetis /api/oshun/domains/metis
See the Subsystem Glossary and Customer-Facing Domains for what each domain owns.
The platform-foundations package#
Several of these adapter-adjacent concerns are not scattered across the tree —
they have a single home. libs/oshun/platform-foundations/
(@oshun/platform-foundations) re-exports exactly nine subsystems from its
src/index.ts:
| Subsystem | Concern |
|---|---|
service-discovery |
Resolving and addressing domain services. |
public-api |
The §27 Public API platform, incl. OAuth 2.1/PKCE (§10). |
shared-contracts |
Cross-subsystem contract surface. |
role-model |
Canonical roles, scopes, field filtering (§10). |
step-up |
Step-up authentication for sensitive actions. |
secrets |
Secret handling. |
configs |
Configuration. |
rollback |
Rollback workflows. |
abuse-controls |
Abuse/throttling controls. |
3. Persistence — libs/oshun/persistence/#
The persistence layer's job is to keep Zod contracts and the database from drifting apart, and to make user-data deletion both safe and auditable.
- Schema rendering.
prisma-renderer.tsandzod-prisma-introspection.tsdrive Prisma schema from Zod, not the reverse, so the contract remains the single source of truth. - Drift test.
contract-persistence-registry.test.ts(backed bycontract-persistence-registry.ts) fails on field divergence between the Zod contract and the persistence layer — a renamed or dropped field is a red build, not a runtime surprise. - Tombstone semantics. Mandatory on every user-data table: deletion is a
soft tombstone that propagates and is audit-logged; there is no silent
re-creation. Covered by
tombstone-semantics.test.ts. - Required indexes.
index-requirements.test.tsvalidates per-table indexes viaEXPLAINchecks in CI, so an unindexed hot path can't land quietly. - Migration plans.
migration-plan.test.tskeeps migrations idempotent and dry-runnable.
Beyond the core quartet, the same library houses the DSAR deletion machinery
— dsar-deletion-cascade.ts (with both unit and .integration tests) and
dsar-erasure-runtime.ts — which is how a privacy-operator's erasure request
fans the tombstone across every owning table. This is the persistence-side
counterpart to the Trust, Safety, and Privacy
workflows.
4. OpenAPI and codegen — libs/openapi/#
- Specs live under
libs/openapi/src/specs/. There are 15 real spec directories:arete,bellona,calliope,concordia,hathor,isis,lilith,metis,nisaba,nyx,oshun-bff,sophia,tara,veritas, andyemaya— plus top-levelmain.yamlandv3.yaml. - Schemas drive specs (Zod → OpenAPI), never the reverse, so the contract layer remains authoritative.
- Typed clients are generated per domain into
libs/<domain>/api-client/. - CI gate: spec drift against runtime fails the build.
5. Idempotency, partial-failure, response normalization#
- Idempotency middleware lives in
libs/shared/http-client/src/idempotency.ts. It defaults to theIdempotency-Keyheader (DEFAULT_IDEMPOTENCY_HEADER) with a 24-hour replay window (DEFAULT_IDEMPOTENCY_TTL_MS = 24 * 60 * 60 * 1000). TheIdempotencyMiddlewareclass — constructed viacreateIdempotencyMiddleware()— gives request deduplication and replay-safe semantics: a repeated key returns the stored envelope instead of re-executing the write. - The same
libs/shared/http-client/package ships the rest of the outbound hardening:circuit-breaker.ts,ssrf-guard.ts,retry.ts,tenant-context.ts, andtracing.ts. So an outbound call from the BFF to a domain (or to an external connector) is idempotent, breaker-protected, SSRF-guarded, retried, tenant-scoped, and traced through one composed client. - Partial-failure envelope. The standardized shape
{ results[], errors[], partial }is the §1 contract above — used across the BFF and domain services so a multi-domain fan-out can report exactly which slices failed without collapsing into a single opaque error. - Job contracts. Retry/dedup/replay/dead-letter contract tests run on every job submitter.
6. Multi-tenant routing — libs/shared/data-residency/#
- Tenant-aware middleware reads
X-Tenant-ID(or the token claim) and propagates the tenant context through every downstream call. - Residency-aware traffic shaping.
traffic-shaping.ts,home-zone.ts, andenforcer.tsroute a request to the user's primary (home-zone) data plane unless explicit cross-region consent is present.dsr-routing.tshandles data-subject-request routing, and an injectedResidencyAuditPublisherseam (exercised byaudit-platform-bridge.spec.ts) ties residency decisions into the audit substrate (§10). - Tenant-aware caching keys prevent cross-tenant cache pollution.
- Tenant-aware feature flags and experiment scoping.
- Cross-tenant leakage tests live under
tests/security/tenant-isolation/.
This is the enforcement spine behind Data Architecture and Tenancy and the residency obligations in Security, Privacy, and Compliance.
7. Background-job substrate — libs/shared/queue/#
A single durable queue is shared by editorial, asset, agentic-AI, notification, billing, and integration jobs — there is not a bespoke queue per domain.
durable-queue.ts— priority classes, replay, deduplication, observability hooks.dead-letter-queue.ts— the DLQ for jobs that exhaust retries.sla-monitor.ts— a per-job-class SLA monitor.memory-queue.ts— an in-memory implementation for tests/local development.worker.ts— the worker loop that drains the queue.
8. Webhook and event-bus plumbing — libs/shared/event-bus/#
The event bus is @oshun/event-bus; its surface is
export class EventBus implements IEventBus with a
createEventBus(config: EventBusConfig) factory. Alongside it:
topic-registry.ts— topics with schema versioning (DEFAULT_EVENT_TOPIC_REGISTRY), so a payload that doesn't match the registered topic schema is rejected before fan-out.outbound-delivery.ts— outbound signing keys, retry/backoff, dead-letter inspection, replay, and per-event audit for tenant-facing webhooks.webhook-simulator.ts— a simulator tenant integrations can use in a dev sandbox.
How the bus actually works — it is not Redis Streams#
Earlier drafts (and the cascade diagram below) labeled the bus "over Redis
Streams." That is the one outright-wrong technical claim, and it's worth
correcting precisely, because the real design is more interesting. The
implementation imports import { Redis } from 'ioredis' and opens two
separate connections — this.pub and this.sub — but it uses none of the
native Streams commands (no XADD, XREAD, XREADGROUP, or XGROUP). Instead
it composes Streams-like guarantees out of simpler primitives:
| Concern | Mechanism |
|---|---|
| Fan-out | Redis pub/sub — this.pub.publish(channel, …) out, pattern-subscribe (pmessage) in. |
| Replay source | A TTL-bounded key per event (setex, default eventTtl = 24h), so a crash-restart can re-deliver unacked work. |
| Delay / nack | A durable sorted set named scheduled (zadd(scheduledKey, deliverAt, …)); a scheduler loop (default 250 ms) pulls due entries. setTimeout state is never the only copy. |
| Consumer groups | A per-(eventId, group) SET … NX claim key (this.pub.set(key, '1', 'EX', eventTtl, 'NX')): only the winner runs the handler. Without a group, every matching subscription runs (classic broadcast). |
| Dead letter | A list (ordered pagination) plus a hash (O(1) id lookup); removeDeadLetter uses LREM for atomic removal. |
So the bus is genuinely durable and consumer-group-capable, but via pub/sub +
TTL keys + a sorted set + SET NX claims — not XADD/XREADGROUP. Treat any
remaining "Redis Streams" wording in the hub doc as a known label error.
9. Inbound integration plumbing — libs/shared/inbound-integrations/#
The package's src/index.ts re-exports 15 connector modules — more than the
nine the original table listed. Several are substantial real implementations
(e.g. oneroster.ts ≈1081 LOC, lms.ts ≈1425 LOC, identity.ts ≈1048 LOC),
not thin wrappers.
| Module | Connector family / role | V1 use |
|---|---|---|
types.ts |
Shared types | Connector contracts used by all the modules below. |
lms.ts |
LMS | LTI 1.3, LTI Advantage, SCORM fallback for Metis institutions. |
lti-verification.ts |
LMS | LTI launch/credential verification, split out from lms.ts. |
scorm-rte.ts |
LMS | SCORM run-time environment (legacy). |
scorm-2004-rte.ts |
LMS | SCORM 2004 run-time environment (second, distinct RTE). |
oneroster.ts |
OneRoster | Rostering sync with conflict reporting and dry-run. |
identity.ts |
Identity | SAML 2.0, OIDC, SCIM 2.0 for tenant SSO. |
calendar.ts |
Calendar | Google/Apple/Outlook two-way sync for Tara/Arete/Nyx/Metis. |
calendar-google-transport.ts |
Calendar | Google-specific transport for the calendar connector. |
payment.ts |
Payment (fiat) | Fiat-rail entitlement upgrades (Stripe-class; Telegram payments). V1.x optional — the V1 primary path is non-custodial crypto via the Aje domain (libs/aje/) plus libs/oshun/payments-bridge/. |
telemetry.ts |
Telemetry | xAPI/cmi5/Caliper export to institutional sinks. |
byom.ts |
BYOM | Tenant-provided model endpoints for Metis study. |
byom-model.ts |
BYOM | The model abstraction backing byom.ts, exported separately. |
notification.ts |
Notification | Slack/Teams notification sinks for institutional surfaces. |
health.ts |
Health | Connector health probes, circuit breakers, version pinning. |
See Messaging Channels and the Aje bridge in Support, Entitlements, Billing, and the Aje Entitlement Bridge for how these surface to customers.
10. Identity, auth, and audit#
Auth primitives — libs/shared/auth-primitives/#
Richer than "JWT and session primitives." The package ships, among others:
jwt.ts, session.ts, api-key.ts, password.ts, oauth-client.ts,
oauth-revoke.ts, token-refresh.ts, token-audit.ts, totp.ts (TOTP for
step-up), platform-roles.ts, and tenant-isolation.ts — each with a
co-located .spec.ts. So API keys, password hashing, OAuth client + revocation,
refresh rotation, token auditing, TOTP step-up, and tenant isolation are all
primitives, not bespoke per-app code.
Workspace identity client — libs/oshun/auth/#
@oshun/auth-client is the workspace-side identity and session client every
Oshun app consumes.
The role/scope model — platform-foundations/.../role-model/#
libs/oshun/platform-foundations/src/role-model/role-model.ts defines the
ten CANONICAL_ROLES — note the precise names; prose elsewhere uses looser
labels like "support" that don't match the code:
| Canonical role | Scopes granted (scopesFor(role)) |
|---|---|
customer |
customer.self, customer.shell |
creator |
customer.self, customer.shell, creator.studio |
support-agent |
support.case.read, support.case.write |
reviewer |
review.queue.read, review.queue.decide |
moderator |
review.queue.read, review.queue.decide, moderation.decide |
privacy-operator |
privacy.dsar.execute, privacy.audit.read, admin.audit.global |
model-operator |
model.registry.read, model.registry.promote |
persona-operator |
persona.registry.read, persona.registry.publish |
tenant-admin |
tenant.console, admin.audit.global |
admin-leadership |
admin.users.act, admin.scope.grant, admin.breakglass.act, admin.audit.global |
The full SCOPE_KEYS list (21 scopes) is the vocabulary authorizeAction(),
authorizeObjectAccess(), and filterFieldsByRole() enforce. creator and
tenant-admin are distinct canonical roles — a fact older role prose omits. The
support role is support-agent, not support. Use these exact strings when
wiring RBAC.
OAuth 2.1 / PKCE — platform-foundations/.../public-api/oauth.ts#
The §27 Public API platform's OAuth module is contracts/state-machine level:
it validates and computes, but does not host a running authorization server.
Concretely it exports OAUTH_GRANT_TYPES,
TOKEN_TYPES = ['access', 'refresh'], and a PKCE validator built on
PKCE_CODE_CHALLENGE_PATTERN = /^[A-Za-z0-9_-]{43,128}$/ (the RFC 7636
URL-safe-Base64 challenge length window). Its functions cover the real surface:
validateAuthorizationCodeRequest, validateAuthorizationCodeRedeem,
verifyPkceChallenge, validateAccessToken, evaluatePublicApiQuota,
buildPublicApiAuditEvent, redeemRefreshToken, and revocationCascade. Error
codes include 'pkce-required' (authorization code request with no challenge)
and 'pkce-verification-failed' (redeem whose verifier doesn't match). PKCE
validation patterns and the refresh/revocation surface are confirmed; a live
authorization-server deployment is not part of this module.
RBAC checks — libs/shared/identity/#
RBAC and permission checks are consumed across domains.
Audit substrate — libs/shared/audit-platform/#
Far more than one sentence's worth: libs/shared/audit-platform/ is a large
real substrate (~40 implementation modules plus tests). At its core is
hash-chain.ts, an append-only hash chain, so an audit record cannot be
silently altered or deleted without breaking the chain. Around it:
schema-versioning.ts, retention.ts, escalation-rules.ts,
support-escalation.ts, compliance-attestation.ts,
watermark-verification.ts, synthetic-media-labeling.ts,
voice-likeness-consent.ts, and provenance-bundle validation (with
.integration specs). This is the substrate behind Iris admin inspection, DSAR
review, the deletion workflow, review packages, synthetic-media labeling, and
voice-likeness consent — the trust-and-safety record of last resort.
How the foundation composes — the BFF request lifecycle#
Every customer- or operator-issued write passes the same middleware sequence
before reaching a domain adapter. Idempotency is checked first (replays return
the cached envelope); tenant and residency resolve next so the rest of the
pipeline can short-circuit on policy. The body is validated against the Zod
contract; the adapter commits owned data with tombstone semantics; an event is
published and an audit record appended. Each station maps to a real building
block above — idempotency → http-client/idempotency.ts; tenant + residency →
data-residency/; validation → @oshun/contracts; adapter → the domain
adapters keyed by DOMAIN_REGISTRY; event → @oshun/event-bus; audit →
audit-platform/hash-chain.ts. (The ordering shown is the intended composition;
no single production route was traced proving every middleware fires in exactly
this order.)
A partial fan-out at the BFF — say one operation that touches three domains — collapses into the §1 partial-failure envelope rather than a single opaque error, so the client learns exactly which slice failed.
The asynchronous side and the Veritas retraction-cascade walkthrough live on the
dedicated Communication Patterns page (which also
inherits the corrected "not Redis Streams" framing from §8). For internal
service-to-service calls, the gRPC layer is real too: libs/proto/ carries
buf.work.yaml, a generated/ tree, and ~24 proto domain directories under
src/ — agent, ai, asset, auth, bridge, collaboration, concordia,
generation3d, hathor, health, isis, loadbalancing, oshun, oya,
pipeline, procedural, project, reflection, rendering, shared,
sophia, splatting, user, and common — well beyond the Psyche/Isis/Sophia
trio commonly cited.
Related#
- High-Level Architecture — where the foundation sits.
- Communication Patterns — the sync/async/gRPC layer built on §7–§8.
- Data Architecture and Tenancy — the residency/persistence story extended.
- Customer-Facing Domains — the domains the adapters (§2) front.
- Subsystem Glossary — names and ownership.
- Security, Privacy, and Compliance — RBAC, residency, and audit obligations.
- Trust, Safety, and Privacy — DSAR, tombstones, and the audit substrate in use.
- Hub: ../ARCHITECTURE.md