# Remote-Control Protocol Evolution

> Scope: how the Bellona remote-control protocol changes over time without
> breaking deployed hosts, gateways, clients, or **stored audit**. Covers the
> versioning scheme, stored-data migrations, host/gateway version drift, the
> command/field deprecation policy, and the forward-compatibility guarantee for
> stored audit across protocol upgrades.
>
> Implemented in `libs/bellona/remote-protocol/src/protocol-evolution.ts`
> (`180.C.29.15`), reconciled with the stored-data snapshot migration framework
> in `apps/bellona/remote-gateway/src/persistent-store.ts` (`180.C.08.11`) and
> the peer-version negotiation in `libs/bellona/remote-protocol/src/version.ts`.

## 1. Versioning scheme

There are three independently-versioned axes:

| Axis | Meaning | Where |
| --- | --- | --- |
| **Protocol version** (`MAJOR.MINOR.PATCH`) | The wire/contract version two peers speak. | `RemoteProtocolVersion`, negotiated per session. |
| **Store schema version** (integer) | The on-disk shape of the gateway's whole record store (devices/sessions/commands/approvals/audit). | `REMOTE_GATEWAY_PERSISTENT_STORE_SCHEMA_VERSION` (currently `3`). |
| **Record schema version** (integer, per domain) | The shape of an INDIVIDUAL record within a domain. | `RemoteStoredRecordVersionStamp` (`{ domain, schemaVersion, protocolVersion, stampedAt }`). |

A stored record carries BOTH its record schema version and the protocol version
it was written under, so migration is deterministic regardless of when or where
the record is later read.

Version comparison is the shared `compareRemoteProtocolVersions` /
`isRemoteProtocolVersionAtLeast` (semver-style component compare). There is one
comparator; nothing re-implements it.

## 2. Stored-data migrations

Two layers, deliberately separated:

### 2.1 Whole-snapshot migrations (`180.C.08.11`)

`migrateRemoteGatewayPersistentStoreSnapshot(input, targetSchemaVersion)` walks
the **whole store container** forward or backward one store-schema-version at a
time (`1 → 2 → 3` and back), appending a `migrationHistory` entry and refreshing
`recordVersionStamps` for the five domains (`devices`, `sessions`, `commands`,
`approvals`, `audit`). This owns the on-disk file and is what the gateway runs
when it loads a store written by an older build.

### 2.2 Per-record migrations (`180.C.29.15`)

`migrateStoredRecord(registry, domain, record, fromVersion, toVersion)` migrates
ONE record between record schema versions by applying ordered, **adjacent**
steps (`v → v+1` via `up`, `v+1 → v` via `down`). The registry validates on
construction that each domain's steps form an unbroken `1 → 2 → … → latest`
ladder — a gap or a non-adjacent step is rejected up front, so a migration can
never skip a version. A backward request whose path lacks a `down` step is
**refused fail-loud**, never silently returned unchanged.

The shipped ladder lives in `REMOTE_RECORD_MIGRATIONS` /
`REMOTE_RECORD_MIGRATION_REGISTRY`. Example — `audit v1 → v2` normalizes the
absent evidence/artifact accumulators and metadata bag to their empty defaults
(`[]` / `[]` / `{}`), which is exactly the value the canonical audit writer has
always hashed (see §5).

**Why two layers?** The snapshot framework migrates the *container* and stamps
each record set; the per-record migrator reshapes an *individual* record. A
reader that loads a single legacy record outside the gateway store (e.g. a
replay bundle, an exported audit entry) uses the per-record migrator directly;
the snapshot framework can call it per record set when a domain's record shape
changes within a store version. They share the monotonic, gapless,
version-stamped invariant.

## 3. Host / gateway version drift

A host and a gateway can run different builds. Drift is resolved at the
handshake by `negotiateProtocolVersion(hostVersion, gatewayVersion, policy)`,
which returns the agreed protocol version or a **refusal**.

The policy (`RemoteProtocolDriftPolicy`) declares:

- `minimumHostVersion` / `minimumGatewayVersion` — the support **floor**; a peer
  below it is refused.
- `supportedProtocolVersions` — the versions both sides understand.
- `maxDriftDistance` — the support **window**: the maximum component distance
  (`|Δmajor| + |Δminor| + |Δpatch|`) tolerated between the two runtimes.

The agreed version is the **highest supported version `≤` both runtimes** (so
neither peer is asked to speak a version newer than it runs). The result is:

| Status | When |
| --- | --- |
| `agreed` | Peers run the same version; a shared version is selected. |
| `agreed-degraded` | Peers differ but drift is within the window and a shared version exists. The session proceeds; the caller is told it spans a version gap. |
| `refused` | A runtime is below the floor, **or** drift exceeds the window, **or** no shared version is `≤` both runtimes. |

A drift beyond the support window is refused **even if a shared version
technically exists** — the window is a hard operational limit, not advisory.

`driftPolicyFromCompatibilityPolicy` derives a drift policy from the richer
`RemoteProtocolCompatibilityPolicy` (`version.ts`) so the string-friendly
handshake check and the full peer-descriptor negotiation
(`selectRemoteProtocolVersion`) never disagree on the floor.

## 4. Deprecation policy

Commands and individual argument fields are deprecated through a
`RemoteDeprecationRegistry`. Each `RemoteDeprecation` carries:

- `kind` — `command` or `field` (a field deprecation also names a `fieldPath`).
- `announcedInVersion` — when the deprecation takes effect.
- `sunsetInVersion` — when the target is **removed** (must be strictly newer than
  the announcement).
- An optional `replacementCommandName` / `replacementFieldPath` and
  `migrationGuideUri`.

`registry.check(commandName, runtimeProtocolVersion, fieldPath?)` returns:

| Status | Allowed? | When |
| --- | --- | --- |
| `active` | yes | Runtime is **older than** `announcedInVersion` (not deprecated yet), or no deprecation matches. |
| `deprecated` | yes (with a warning) | Runtime is at/after the announcement but **before** the sunset. |
| `sunset` | **no** (fail-loud) | Runtime is at/after `sunsetInVersion`. |

When a `fieldPath` is supplied, **both** the command-level and the field-level
deprecations are evaluated and the **more severe** outcome wins (sunset >
deprecated > active) — a field check can never escape a sunset command by
lacking its own entry. `assertCommandNotSunset(...)` throws a
`RemoteDeprecationSunsetError` once past the sunset; before that it returns the
deprecation warning the caller should surface or log.

## 5. Forward-compatibility of stored audit (the critical guarantee)

**An audit entry — and a whole hash chain — written under an OLD protocol schema
must still verify and read under the NEW one.** Audit is the system of record;
an upgrade that silently invalidated stored audit would be catastrophic.

How it holds:

- The canonical audit hash (`computeRemoteAuditEntryHash`,
  `audit-chain.ts`) is `sha256` over a deterministic, sorted-key serialization
  that **drops `undefined` keys**. The `RemoteAuditEntrySchema` **defaults**
  `evidenceIds` / `artifactIds` to `[]` and `metadata` to `{}`, and the only
  audit writer (`appendToAuditChain`) has **always** hashed those empties for
  absent values. So a genuinely-legacy entry that omitted those keys on disk
  hashes **identically** to the migrated entry that carries the empty defaults.
- `migrateAuditEntryPreservingHash(entry, from, to)` migrates an entry through
  the record ladder, re-parses it (filling schema defaults), then **recomputes**
  its hash and asserts it equals the **stored** `entryHash`. If a migration ever
  changed a hash-bound field, the recomputation diverges and it throws
  `RemoteAuditForwardCompatError` — fail-loud. This makes "audit migrations must
  not touch hashed fields" an **enforced** invariant, not a convention.
- `migrateAuditChainPreservingIntegrity(chain, from, to)` migrates every entry
  hash-preservingly and re-runs the authoritative `verifyAuditChain`. The chain
  must still verify end-to-end (genesis anchor, linkage, per-entry hash) after
  the upgrade, or it throws.

Consequence: only **additive, hash-neutral** changes are legal for audit between
versions (new OPTIONAL fields, defaulted empties). Anything that would alter the
hashed content of an existing entry is rejected by the forward-compat checker.

## 6. Adding a protocol change — checklist

1. **New optional field on a stored record** → add a `v → v+1` migration to
   `REMOTE_RECORD_MIGRATIONS` that defaults it; for audit, confirm the default
   is hash-neutral (the forward-compat test will catch it if not).
2. **New store shape** → extend `persistent-store.ts` snapshot migrations
   (`180.C.08.11`) and bump `REMOTE_GATEWAY_PERSISTENT_STORE_SCHEMA_VERSION`.
3. **Deprecating a command/field** → add a `RemoteDeprecation` with an announce
   and a (strictly newer) sunset version and a replacement.
4. **Raising the support floor** → bump `minimum{Host,Gateway}Version` in the
   compatibility/drift policy; old peers are then refused at the handshake.
5. **New command** → use the `bellona-new-command` generator + conformance
   runner (`180.C.29.14`) to wire all nine touchpoints.

## 7. Tests

`libs/bellona/remote-protocol/src/protocol-evolution.test.ts` asserts the real
behavior:

- a `v1 → v2` stored-record migration round-trips (and a genuinely-legacy record
  that omitted the empty accumulators migrates correctly);
- a `v1` audit **chain still verifies after upgrading** to `v2`, with every
  entry's stored hash unchanged;
- a migration that changes a hashed field is **rejected** (forward-compat
  fail-loud);
- an incompatible version drift (below floor, out-of-window, or no shared
  version) is **refused**;
- a deprecated command **past its sunset** is **refused**, and a runtime older
  than the announcement is still `active`.
