# Runbook: V1 Signing Key Compromise (`gd:signing-key-compromise`)

Owner: delivery pipeline on-call (`rota:delivery-pipeline`). Last reviewed:
2026-08-14. Version: 1.

> A signing key is disclosed. The material is intact and no longer trustworthy,
> which is why this is a corruption and not an outage. **Rotation is not
> reversal:** every signature made under the disclosed version stays disclosed,
> so this scenario has a recovery procedure and no rollback.

**Read this first, because it changes what the rest of the runbook can
promise.** `verifyAudioC2paManifest` in
`apps/oshun/bff/src/generation/audio-c2pa-signer.ts` verifies the Ed25519
signature against `manifest.publicKeyPem` — **the public key carried inside the
manifest being verified**. Nothing on this estate compares `signerKeyId` or
`publicKeyPem` against an allowlist; `dep:trust-store` is registered as a core
dependency in the S10.5 register and has no implementation.

Two consequences, and both are the incident rather than a footnote to it:

1. **Revocation is not enforceable.** Rotating the key does not invalidate a
   single existing manifest. Nothing reads a revocation list because there is no
   revocation list.
2. **The disclosed key mints manifests that verify.** Anyone holding it produces
   output indistinguishable from ours, and the estate's own verifier admits it.

Rotation therefore protects **future** output only. Say so when communicating,
because an audience told "the key was rotated" will hear "the old signatures are
invalid", and they are not.

## 1. Detection and declaration

**Detection signal.** Disclosure is not observed by the estate — it is reported
from outside it (a leaked build artifact, a public repository, a provider
notice). There is no in-estate detector, and there cannot be one for a key that
still works.

- Detection authority: **delivery pipeline on-call**, contact
  `rota:delivery-pipeline`.
- Declaration authority: **production operations duty officer**, contact
  `rota:prod-ops`.
- **Gap, stated.** `docs/operations/on-call.md` describes a weekly
  primary/secondary rotation and names no person and no rotation config; none of
  the register's `rota:` identifiers appears in it or in
  `docs/operations/alert-routing.md`. The name above is registered; the route
  from it to whoever is carrying the pager is not.
- **Record the disclosure instant, not the discovery instant, if they differ.**
  The exposure window is what decides which signatures are suspect, and it opens
  when the key left, not when somebody noticed.

## 2. Communications

| Audience  | Within | Content                                                                                  |
| --------- | ------ | ---------------------------------------------------------------------------------------- |
| Actors    | 15 min | that content credentials issued before the rotation cannot be relied on to prove origin  |
| Operators | 5 min  | that this runbook is running, and by whom                                                |
| Regulator | 72 h   | only where a disclosure is reportable; the estate has no automatic determination of this |

All travel over `dep:notification-channel`, which this scenario leaves standing.

**Do not describe rotation as revocation.** See the header.

## 3. Dependency assumptions

- `dep:primary-store` is healthy: release rows carry the key version their
  signature was made under, and that column is the only record of which output
  was signed under the disclosed key.
- `dep:blob-store` is healthy: re-signing anything requires the bytes.

## 4. Recovery infrastructure

**In place, with one exception that is the point.** The new key material must be
generated **outside** the compromised estate — on an operator-controlled machine
— and delivered into the BFF environment. A key generated on the host whose
disclosure is under investigation is a key of unknown provenance.

## 5. Secrets

- `OSHUN_C2PA_SIGNING_KEY` (PKCS#8 Ed25519 private key) and
  `OSHUN_C2PA_SIGNER_KEY_ID` (default `oshun-audio-c2pa-v1`), read by
  `resolveAudioC2paSigner`.
- `OSHUN_LIVING_SCENES_C2PA_KEY_ID`, read by
  `apps/oshun/bff/src/routes/living-scenes.ts`. **The estate has more than one
  signing key id.** A rotation that changes one and not the other leaves half
  the output signed under the disclosed identity.

Both are recoverable out of band, and here that is a requirement rather than a
convenience: the replacement is generated off the estate.

## 6. Restore order

1. `dep:trust-store` — **ABSENT.** This is where the revocation entry would be
   published, and there is nothing to publish it to. The step is listed rather
   than dropped so that the order states what is missing instead of implying it
   was done.
2. `dep:signing-service` — set the new `OSHUN_C2PA_SIGNING_KEY`, set a new
   `OSHUN_C2PA_SIGNER_KEY_ID` (do not reuse the disclosed id — the id is how a
   later reader tells the two apart), set `OSHUN_LIVING_SCENES_C2PA_KEY_ID`, and
   restart the process that reads them.

The order is revoke-then-re-key and not the reverse: re-keying first means a
window in which the new key is live and the old one is still advertised as
trustworthy. On this estate that window never closes, because step 1 does not
exist.

## 7. Validation

- Sign a known media file under the new key and verify it with
  `verifyAudioC2paManifest` — media hash, canonical hash, signature.
- Confirm the new `signerKeyId` appears on newly issued manifests and the
  disclosed one does not.
- Confirm both key ids moved: the generation signer and the living-scenes
  signer.

> **The check that cannot be written.** "A manifest signed by the disclosed key
> is refused" is not assertable, because nothing refuses it. Until a trust list
> exists, treat every pre-rotation manifest as unverified provenance rather than
> as invalid provenance — the estate cannot tell the two apart.

## 8. Fencing — ABSENT for keys, and this scenario is what fencing is for

A key id in an allowlist is the fencing token this scenario needs — the thing a
holder of the old key is rejected **by**, rather than instructed about. There is
no such list, which is the reason §7 has a check it cannot write and §2 has a
message it cannot soften.

**The estate knows how to do this; it just has not done it here.**
`live_media_pipeline_job_attempts.lease_token`
(`libs/shared/live-media/src/sql-media-pipeline-job-store.ts`) is a real fencing
token: every heartbeat, lease expiry and completion carries
`AND lease_token = $4 AND status = 'leased'` in its `UPDATE ... WHERE`, so a
worker holding a superseded lease updates zero rows. A superseded **key** has no
equivalent, and the shape of the fix is the same — the check has to be on the
side that accepts the work, not on the side that produces it.

## 9. Failover

Nothing is promoted; there is one signing path before and after. The change is
which key it holds.

## 10. Reconciliation

Enumerate what was signed under the disclosed key. The release rows carry the
key version (`dep:primary-store` → `dep:signing-service` in the reference
graph), so the population is queryable — that column is what makes this scenario
recoverable at all.

Re-signing that population under the new key does **not** retire the old
signatures; both verify. Re-sign only where a consumer will fetch the newer
manifest, and record which artifacts were left under the disclosed key.

## 11. Failback

There is no failback. The estate stays on the new key; returning to the
disclosed one is the incident. The nearest thing to a return is the point at
which the last artifact still advertising the old key id has been re-signed or
retired, and that is a backlog item rather than a step.

## 12. What this runbook does not cover

- A key that is merely missing. That is
  [v1-dr-signing-service-outage.md](v1-dr-signing-service-outage.md), and its
  recovery is restoring the **same** key.
- Provider credentials. `scripts/isis/rotate_runpod_credentials.sh` rotates
  those; they are not content-signing material.
- Building the trust store. That is the gap this runbook names and does not
  close.

## 13. Machine-checkable statement

<!-- oshun:runbook -->

```json
{
  "id": "runbook:v1-signing-key-compromise",
  "scenarioId": "gd:signing-key-compromise",
  "version": "1",
  "detectionAuthority": {
    "role": "service on-call",
    "name": "delivery pipeline on-call",
    "reachableOver": ["dep:notification-channel"]
  },
  "declarationAuthority": {
    "role": "operations duty officer",
    "name": "production operations duty officer",
    "reachableOver": ["dep:notification-channel"]
  },
  "communications": [
    {
      "audience": "actors",
      "channelDependencyId": "dep:notification-channel",
      "withinMs": 900000
    },
    {
      "audience": "operators",
      "channelDependencyId": "dep:notification-channel",
      "withinMs": 300000
    },
    {
      "audience": "regulator",
      "channelDependencyId": "dep:notification-channel",
      "withinMs": 259200000
    }
  ],
  "assumes": ["dep:primary-store", "dep:blob-store"],
  "recoveryInfrastructureDependencyId": null,
  "secrets": [
    {
      "id": "OSHUN_C2PA_SIGNING_KEY",
      "heldInDependencyId": "dep:signing-service",
      "recoverableOutOfBand": true
    },
    {
      "id": "OSHUN_LIVING_SCENES_C2PA_KEY_ID",
      "heldInDependencyId": "dep:signing-service",
      "recoverableOutOfBand": true
    }
  ],
  "restoreOrder": ["dep:trust-store", "dep:signing-service"],
  "fencing": null,
  "promotesAStandby": false,
  "validation": [
    "a file signed under the new key verifies through verifyAudioC2paManifest",
    "newly issued manifests carry the new signerKeyId and not the disclosed one",
    "both signing key ids moved: the generation signer and the living-scenes signer"
  ],
  "failback": "none: the estate stays on the new key and returning to the disclosed one is the incident; the nearest equivalent is retiring the last artifact still advertising the old key id"
}
```