# Proto — Systems Deep Dive

> The `libs/proto/` area: a single Nx library (`@oshun/proto`) that owns the
> platform's **Protocol Buffer / gRPC wire definitions** — 30 `.proto` files
> plus the TypeScript loader, service registry, and a parity gate that keeps the
> proto surface in lockstep with the Zod contracts.

## What this area is

`libs/proto/` is **one Nx project**, not a family. The single `project.json` at
`libs/proto/project.json` declares the library `@oshun/proto`
(`tags: ["scope:shared", "layer:contracts"]`). Everything else under the
directory — every `.proto`, the loader, the buf config, the parity check — is
part of that one package. So the "entity catalog" for this area has exactly one
node, and the interesting structure is _inside_ it.

What the package owns is the cross-service **binary wire surface**. Where
`libs/contracts/` holds runtime-validated Zod schemas for the HTTP/JSON
boundary, `@oshun/proto` holds the Protocol Buffer message and gRPC service
definitions for service-to-service traffic. The `src/` tree carries 30 proto
files (verified via `git ls-files 'libs/proto/**/*.proto'`) organised by domain:
core services (`ai/ai.proto`, `agent/agent.proto`, `asset/asset.proto`,
`auth/auth.proto`, `collaboration/`, `project/`, `user/`), domain services
(`isis/` the generative factory, `sophia/` the knowledge engine, `hathor/`
worldbuilding at 1,257 lines, `concordia/` mediation, `oya/` the embodied-hive),
the `shared/` substrate (`evidence`, `memory`, `persona_policy`,
`generation_control`, `common`), rendering and 3D (`generation3d/`,
`rendering/`, `splatting/gaussian_splatting`, `procedural/`), the engine bridges
(`bridge/blender`, `bridge/godot`, `bridge/unreal` at 990 lines), and
infrastructure (`health/`, `loadbalancing/`, `reflection/`,
`pipeline/autonomous_pipeline`, and the V2
`oshun/v2/persistent_economy/economy.proto`). These are real, substantial
definitions — the proto files total well over 15,000 lines.

The TypeScript surface (`src/index.ts`) is deliberately thin and is fully
implemented, not a scaffold. It exposes three things: **proto loaders**
(`src/loader.ts`), a **service registry** (`src/services.ts`), and re-exported
`@grpc/grpc-js` types for caller convenience. `loader.ts` wraps
`@grpc/proto-loader` and `@grpc/grpc-js` to turn a `.proto` path into a live
`grpc.GrpcObject`, with `DEFAULT_LOADER_OPTIONS` (`keepCase`, `longs: String`,
`enums: String`, `defaults`, `oneofs`, and an `includeDirs` set rooted at the
package so cross-file imports resolve), a `PROTO_PATHS` const mapping each
domain to its file, and `loadAllProtos()` to merge every package definition.
`services.ts` is a hand-maintained registry: `SERVICE_NAMES` maps ~50
fully-qualified gRPC service names (e.g. `oshun.ai.AIService`,
`oshun.shared.evidence.OshunEvidenceService`), `DEFAULT_CHANNEL_OPTIONS` sets
keepalive and a 50 MB max message size, and `getServiceMetadata()` returns
per-service `{ protoPath, package, methods[] }` records — the method lists are
real (e.g. the AI service's
`GenerateText`/`StreamGenerateText`/`GenerateImage`/`GenerateAudio`/`Generate3DModel`/`GenerateEmbeddings`).

## How the area is shaped (inside the one package)

Three layers live together under `libs/proto/`:

- **The proto definitions** (`src/**/*.proto`) — the source of truth for the
  binary wire format. Field numbers are explicitly documented as stable wire
  identity that must never be reused (see the header of `src/oya/oya.proto`).
- **The TypeScript runtime helpers** (`src/loader.ts`, `src/services.ts`,
  `src/index.ts`) — the loadable, type-safe access layer, plus the
  `proto.spec.ts` test suite that actually parses several protos (evidence,
  memory, persona, generation-control, concordia, oya, persistent-economy) and
  asserts the registry constants.
- **The codegen + governance tooling** — `buf.work.yaml` / `src/buf.yaml` (buf
  lint with `DEFAULT` + `COMMENTS`, `enum_zero_value_suffix: _UNSPECIFIED`,
  breaking-change detection, googleapis dep), `buf.gen.yaml` (ts-proto + Go +
  JSON-schema plugins), a `scripts/generate.ts` protobufjs fallback generator,
  and a checked-in `generated/buf-image.json` (~2 MB compiled descriptor image).
  The Nx targets reflect this: `build` (tsc, copying `**/*.proto` as `protos`
  assets), `test` (vite), and the run-commands targets `proto:gen`
  (`buf generate`) and `proto:lint` (`buf lint`).

A distinctive piece is `oya/check-proto-parity.mjs` — a fail-loud governance
gate, not a stub. It loads the `@oshun/contracts/oya` Zod schemas (through the
`tsx` ESM loader) and the parsed `oya.proto`, then asserts coverage (every
contract schema has a mapped proto message), field-set parity (proto fields
match Zod keys after case normalisation), and enum parity (proto enums equal Zod
enums plus the mandatory `*_UNSPECIFIED = 0`). It even self-tests its own drift
detector first — mutating a clone of the proto to drop/rename a field and
confirming the checker reports it — and exits non-zero (code 2) if the detector
itself is broken, so a green run is trustworthy.

## How it fits the wider system

`@oshun/proto` sits at `layer:contracts`, `scope:shared` — the bottom of the
dependency graph, like `@oshun/contracts`. Its only runtime deps are
`@grpc/grpc-js`, `@grpc/proto-loader`, and `protobufjs` (see `package.json`); it
takes **no upstream Oshun dependency**, so any service — producer or consumer —
can import the same wire definition without pulling in the other's runtime.

Consumers are gRPC clients and servers across the platform: a service imports
`SERVICE_NAMES` and `getServiceMetadata()` to know what to register or dial,
calls `loadProto()` / `loadAllProtos()` to obtain the `GrpcObject`, and applies
`DEFAULT_CHANNEL_OPTIONS` / `createCredentials()` for the channel. The proto
files map directly onto the domains documented elsewhere in this repo (Isis,
Sophia, Hathor, Concordia, Oya, the shared evidence/memory/persona/generation
substrate, the Blender/Godot/Unreal engine bridges, and the V2 persistent
economy). The boundary contract is the same one stated in the proto headers: the
proto messages are the on-the-wire form of the same vocabulary the Zod contracts
and (for Oya) the Rust engine speak, and the parity gate is what keeps those
three representations from drifting.

## Entity reference

### @oshun/proto

The platform's Protocol Buffer / gRPC wire-definition library and the **only**
Nx project under `libs/proto/` (`libs/proto/project.json`, package
`@oshun/proto`, `tags: ["scope:shared", "layer:contracts"]`). It is fully
implemented, not a scaffold: `src/` holds 30 real `.proto` files (core, domain,
shared-substrate, rendering/3D, engine-bridge, and infrastructure services,
totalling 15k+ lines), and a thin TypeScript access layer — `src/loader.ts`
(`loadProto`/`loadProtos`/`loadAllProtos`/`getProtoPath` over
`@grpc/proto-loader`, plus `PROTO_PATHS` and `DEFAULT_LOADER_OPTIONS`),
`src/services.ts` (the `SERVICE_NAMES` registry of ~50 fully-qualified gRPC
service identifiers, `DEFAULT_CHANNEL_OPTIONS`, `createCredentials`, and
`getServiceMetadata` with real per-service method lists), and `src/index.ts`
re-exporting that surface plus `@grpc/grpc-js` types. Tooling is genuine too:
buf lint/breaking config (`src/buf.yaml`, `buf.work.yaml`), codegen
(`buf.gen.yaml` ts-proto/Go/JSON-schema plugins; `scripts/generate.ts`
protobufjs fallback), a ~2 MB compiled descriptor image at
`generated/buf-image.json`, an exercised test suite (`src/proto.spec.ts` loads
and asserts several protos), and the fail-loud Oya parity gate
`oya/check-proto-parity.mjs` that keeps `oya.proto` in lockstep with the
`@oshun/contracts/oya` Zod schemas (with a self-test of its own drift detector).
It exists so service-to-service binary traffic shares one canonical wire
definition, sitting at the bottom of the dependency graph with no upstream Oshun
dependency.