Eve worker and reviewer operation
Current platform memory instruction — Elikem-provided AGENTS.md, 2026-10-04
The instructions supplied directly for the current repair session say: “The platform-ops VM is shared by interactive Claude and automated Codex. Both can use all CPUs; Codex receives at most half the workload memory budget (capped at 12 GiB), retaining the other half for the interactive session.” The same instruction is in the repository: AGENTS.md (commit 81808f47f2b, 2026-10-04) and .claude/roles/platform-operator.md and platform-worker.md. Those files record the instruction as Elikem’s; a primary record in Elikem’s own name is awaiting verification. This is the basis for the platform allocation during fleet recovery. It supersedes the platform memory statement in the dated 2026-10-03 account below for this work; it does not revoke those accounts’ existing local access or enlarge any Eve authority. This correction addresses review e43143a5-cd6a-40e2-9393-8971227cc50a and is submitted for independent review, not self-approved. The fleet recovery runbook is document 2a5bae68-0494-4ea4-acbc-169092bfac7d.
Historical dedicated VM access instruction — 2026-10-03; platform memory amended by the current supplied AGENTS.md instruction below
Elikem explicitly instructed his assistant to give both Codex workers, including platform-ops, plus the reviewer and delegate full resource and operating-system access to their dedicated VMs. This operational instruction supersedes earlier restricted-filesystem, 4/12 GiB and platform half-memory policies for these four accounts. It does not expand Eve scopes, change independent review, authorize publication, permit impersonation, or permit credential disclosure. Historical dated evidence below remains historical.
Implemented and pushed on origin/main at 77f7cc8ddd5848bd457422a8020f51aa7b198731 (initial implementation 68a1f704e9546d64a64af498f64f3fe0de7b04ce). Task: platform-ops:dedicated-codex-vm-access-2026-10-03. All four effective services select this tested root-owned release. Codex arguments use --sandbox danger-full-access, --ignore-rules and approval_policy="never"; assigned models and reasoning efforts are preserved. Reviewer verification runs directly on its dedicated VM with a clean environment and disposable PostgreSQL. Product worker retains gpt-6.1-sol/high; platform-ops gpt-6.1-sol/xhigh; reviewer gpt-6.1-sol/high; delegate gpt-6-astra/xhigh.
Verification: 156 targeted Linux runner tests passed with no skips; 88 reviewer regressions also passed as root with HOME unset and the release-controller configuration marker; 26 Python installer/resource-policy tests passed on Linux; 41 typecheck-ratchet tests passed. Targeted ESLint, Prettier, syntax checks, git diff --check and Eve preflight passed. Each VM passed a service-equivalent OS probe proving its CPU affinity, exact cgroup memory limit, unlimited CPU quota, sudo uid 0, and creation/removal of a temporary root-owned file under /etc/oshun. Each installed Codex CLI accepted the full-access arguments generated by its deployed code and actual runner configuration. The probe did not launch an extra agent or claim work.
Rollout preserved active leases and checkouts. All polling timers are active and the Eve switches remain enabled with no attention flag. At this verification snapshot, these already-running jobs still use their previous process configuration and will load the commissioned settings automatically at their next cycle: worker, ops. Their current jobs were not interrupted.
Rollback: retain the prior immutable runner release, restore the saved runner configuration, remove the assigned account’s /etc/sudoers.d/90-<account>-dedicated-vm and zzzz-dedicated-agent-access.conf only when reverting this owner decision, then daemon-reload and drain/restart through normal recovery procedures. Resource policy stays host-sized unless separately changed. Do not discard an active checkout or recovery record. Commissioning refuses an effective source overridden by another drop-in.
Complete test output, before/after resource and permission evidence, Codex launch validation and runner health snapshot: /api/content/media/ce1e5b7a-daed-852e-98f2-9e44746c0583/file
This runbook describes the separate worker and reviewer roles. Tasks, review reports and documents live in Eve. This document is a draft for independent review; it does not enable automation or authorize a publication.
Interactive operation
Codex and Claude Code both support interactive worker or reviewer sessions. Select the appropriate scoped Eve credential with EVE_API_CONFIG before starting the interactive session. Use the registered oshun-eve MCP tools or ./eve. Do not change the author credential into a reviewer credential.
The unattended worker credential on 46.62.222.247 is /mnt/volume300/eve-codex/worker/.config/oshun/eve-api.json. The reviewer credential on its separate host, 84.247.174.203, is /srv/eve-codex/reviewer/.config/oshun/eve-api.json. These files are private. Do not print or copy their contents into conversations, source files or reports.
A worker checks ./eve reviews --author <identity> --state changes-requested, implements one task, runs relevant checks, commits and pushes to origin/main, then uses ./eve done with its fencing token, full pushed commit SHA and verification evidence. Do not edit historical tracker checkboxes.
A reviewer runs ./eve reviews sync --request-id <uuid>, then ./eve reviews claim [review-id] --request-id <uuid>. Read the exact task commit or submitted document revision and its evidence. Renew the ten-minute review lease with ./eve reviews renew <id> --token <token> --request-id <uuid>. Complete using ./eve reviews complete <id> --token <token> --report-file <private-json-file> --request-id <uuid>, or the equivalent eve_review_command MCP tool. Use a new UUID request ID for each logical mutation, and retain that ID when retrying the same operation. Use ./eve reviews reconcile <id> --request-id <uuid> if a document outcome needs reconciliation after an interruption.
Review findings must identify the problem, its location, evidence and required fix. Decisions are approve, request-changes or needs-evidence. A task review checks acceptance criteria, the exact diff, tests, regressions and linked documentation. Content reviews check the appropriate requirements, reasoning, accuracy, sources, links, diagrams and rendering evidence. Record the interactive engine in the report model field.
A reviewer does not implement fixes. Requested task changes reopen the original task and preserve its history; authors fix, test and resubmit with a new pushed commit. Requested document changes return the document to draft. Resubmission pins a new revision. PRD, ADR and architecture approvals require Elikem, or his explicitly enabled delegate, whose decision is attributed separately; an ordinary worker or reviewer never approves them itself. Neither reviewer approval nor a worker completion publishes content or promotes UAT.
Findings are remarks by default. A remark says what to change; the request-changes that carries it returns the document to draft and changes none of its text, so a remark never starts the change cascade. A finding may instead be a correction (kind correction), bound to one section of the text the review read: section names it as the content API's text read resolves a section (its outline id, its anchor or its heading, matching exactly one), and sectionHash is the sha256 of that section's text as the text read returns it. When a document review is completed, the worker API checks each correction against the text the review read and refuses (bad_findings) one whose section that text lacks or whose hash is not that section's. A task review or a task change request has no reviewed text, so it may not carry a correction.
The author accepts a correction by applying it as an edit or patch-block through the content API with the note correction:<review uuid>. Before the edit is written, the content service reads that review through the worker API as the editor, so the editor needs review access (board:author or board:review); without it the edit is refused as correction_unverifiable. It is refused when the review does not exist (correction_review_not_found), reviews another document (correction_unrelated), has no applied decision (correction_not_accepted: the review must be changes-requested or approved), holds remarks only (correction_not_found), or has no corrected section that still reads as the reviewer read it in the revision being edited (correction_stale). The edit must change a corrected section (correction_not_applied). For a heading-bound correction or an individual-slide correction, it may neither change another section nor add one (correction_overreach): newly added sections are outside that binding. A presentation correction bound to section:0 covers the entire deck and may change or add slides; the document, decision and section-hash validation requirements still apply. This holds for a patch-block as for an edit. The content event records the note, and like any edit it starts the change cascade from the document, marking stale the tasks that cite it.
Unattended operation
Only Codex runs unattended. Dev administrators control Automatic worker and Automatic reviewer independently in https://docs.dev.oshunsynthetics.com/workspace?view=reviews. The repository defines these administrator controls in the Review workspace’s AgentAutomation panel. API integration tests verify administrator authorization; that is not evidence of a signed-in browser session rendering this page. Both default off. Off prevents new unattended claims while the current job may finish. Interactive sessions remain independent. Agent API keys cannot change these switches.
The automated worker runs on 46.62.222.247, the reviewer on 84.247.174.203, and the separately enabled delegate on 84.247.128.180. Interactive product work remains on 204.168.220.77. Platform operations runs on 84.247.129.82; Build and CI run on 84.247.171.63. The migration record is https://docs.dev.oshunsynthetics.com/workspace?document=db49f644-3ce7-450c-868a-c3aea496d571. Linux accounts eve-codex-worker and eve-codex-reviewer have separate private homes and API credentials. The worker uses /mnt/volume300/eve-codex/worker/repository. The reviewer reads a root-owned Git mirror and cannot write implementation code. Provision Codex authentication only under the owner’s authorization. The runner installer does not copy authentication. The migration retained the existing runner sessions; Elikem’s 2026-10-02 request to add an automated platform worker authorized provisioning that new account. No approval of this runbook supplies further authentication or credential authority.
The systemd timers eve-codex-worker.timer, eve-codex-reviewer.timer and eve-codex-delegate.timer poll approximately once a minute and start after reboot. Their services process one job at a time under separate non-root accounts with owner-authorized local sudo on their dedicated VMs. Every Codex runner uses all assigned vCPUs and the workload budget in /etc/oshun/dedicated-resources.json, leaving 2 GiB for the OS. The product worker has four vCPUs, 14,220,787,712 bytes (13.24 GiB) of guarded service memory and a 10,240 MiB Node heap. Reviewer and delegate each have eight vCPUs, 23,050,846,208 bytes (21.47 GiB) of service memory and a 17,408 MiB child heap. Platform-ops is different: its VM is shared with interactive Claude, so its automated worker receives at most half the workload budget, capped at 12 GiB (infra/hetzner/platform/resource_policy.py). Use the cgroup budget and Node heap given under Automated platform operations below, never the reviewer and delegate figures. The full-budget platform-ops figures recorded on 2026-10-03 are historical. The typecheck ratchet inherits this heap through OSHUN_NODE_HEAP_MIB and can request a smaller heap. Check actual host/cgroup availability, run one heavy verification at a time, preserve recovery state and other active sessions. Reviewer and delegate source timers refresh their origin/main evidence mirrors; executable runner sources remain separate root-owned releases. Polling with switches off does not launch Codex.
The worker has board:read, board:work, board:author, content:read and content:write. The reviewer has board:read, board:review, content:read and content:review. Neither has publication or key-administration privileges. Runner keys expire; renew them before expiry through the operator key-management workflow.
The unattended reviewer accepts only bounded evidence-backed reports. It records missing or oversized evidence rather than approving unsupported conclusions. Code review accepts a trusted, passed Dev CI receipt in one of two forms: one naming the exact submitted commit, or a Dev release receipt whose released commit contains it, which the reviewer proves with git merge-base --is-ancestor in its mirror. A receipt supports a review and never replaces it: the reviewer still reads the exact submitted diff and its test coverage. A commit no receipt covers may need an operator-directed test or an interactive review. Presentation approval still requires independent rendering and accessibility evidence.
Recovery and maintenance
If the worker cannot confirm a clean checkout, a pushed commit and a completed Eve submission, it preserves its checkout and /mnt/volume300/eve-codex/worker/.local/state/oshun/worker/worker-recovery.json, writes a task note when possible, and refuses another task. Stop that host’s worker timer, inspect the task journal, Git status and pending API requests, resolve or preserve the interrupted work, then retire the recovery record only after the operator confirms a consistent state and a current fencing token. Stopping one host does not require changing the shared dashboard switch or interrupting the other worker.
The reviewer saves an interrupted completion in /srv/eve-codex/reviewer/.local/state/oshun/reviewer/pending-review.json and retries that same outcome and request identity, including after Off, without another model invocation, until Eve records it. One exception: when Eve refuses it with review_lease_conflict, the review’s lease has ended and the review may be claimed again, so the runner drops the saved completion, logs "the saved decision outlived its lease", and the review is claimed and performed afresh when it is eligible. Do not discard the record by hand before reconciling the Eve result.
If Codex authentication expires, sign in or renew the dedicated account’s login before enabling it; the runner checks login status before claiming work. If the worker loses GitHub access, restore its repository-scoped deploy key. Do not replace it with a personal private key. Do not prune /srv/eve-codex/source.git: the dedicated worker checkout borrows its Git objects.
Install or upgrade runner accounts with infra/hetzner/eve/install-agent-runner.py; it preserves explicit full-host settings on reinstallation. For the four VMs commissioned by Elikem, run python3 infra/hetzner/eve/dedicated_agent.py <worker|ops|reviewer|delegate> --source <root-owned-tested-release>. This applies the detected host resource budget, hostAccess=full, exact memoryBytes, passwordless sudo for only the assigned account, and zzzz-dedicated-agent-access.conf after historical runtime overrides. It verifies the effective ExecStart and WorkingDirectory, keeps the first runner.json.before-dedicated-vm backup, and never restarts an active job or changes Eve switches/credentials/scopes. Keep the polling timer enabled after commissioning so active jobs can finish and subsequent cycles load the new release. The generic installer remains restricted until this explicit commissioning.
Verified setup and publication controls
A new installation must prove that disabled automation refuses new claims, using the isolated integration fixture before live operation. Check systemctl status, journals, the runner identity and the live API’s automation state. Only Elikem may change a dashboard switch. An additional prepared worker may join an already-enabled worker role after his request, without interrupting existing jobs.
Reviewer approval makes a revision eligible for publication. It does not publish it. An administrator explicitly freezes a content publication in Eve’s Publications workspace. Automatic main deployments preserve that selected set of document revisions, media and source paths as the application code changes. Newly reviewed content enters a site only after a separate content selection.
UAT requires explicitly promoting an exact Dev publication with passing deployment evidence. UAT content is a read-only tested snapshot. Its tasks and reviews cannot mutate canonical Dev work, and it has no unattended-runner switches.
Application, Docs Center and Presentation Center hosting is on the dedicated Contabo Dev host 158.220.116.247 and UAT host 84.247.131.239 under oshunsynthetics.com. The authoritative Eve database, worker API and writable CMS run on Dev; UAT retains a read-only content snapshot. Release and publication orchestration runs on Build, 84.247.171.63. GitHub stores repository code and runs checks; it does not host or publish a separate site.
Content CI on the Build server
Checks that read authored documents run in the Eve content CI on the Build server (84.247.171.63): the recovery drills (disaster-recovery and runbook suites, game-day rotation, acceptance drill, follow-up retests) and the 190 V1 workbench inventory commands registered in tools/eve-content/workbench-checks.json. The required content-CI configuration uses an eve-content-ci account, a board:read/content:read key and its own rootless Docker daemon. Verify effective scopes through authenticated whoami before commissioning; never infer them solely from the installer. For each new Dev publication it checks out the exact pinned commit, exports and verifies the publication, and runs tools/eve-content/content_ci.py inside check_workspace.py; the timer runs every 30 minutes. Reports and receipts: /srv/eve-content-ci/state/runs; status: python3 /usr/local/lib/eve-content-ci/content-ci.py status. Heavy CI work shares /mnt/volume300/oshun-platform/ci/deploy.lock with releases, publication preparation, task tests and security scans. Content CI opens that lock read-only under its isolated account and defers while another job holds it; missing or inaccessible locks fail closed.
Automated platform operations — 2026-10-02
Elikem requested a second automated implementation worker on 84.247.129.82, alongside interactive Claude. Its Eve identity is eve-codex-platform-ops-01, Unix account eve-codex-platform-ops, and model gpt-6.1-sol with xhigh. Its private home is /srv/eve-codex/platform-worker, with a dedicated repository, .config/oshun/eve-api.json credential and .local/state/oshun/worker runner state. It claims only platform-ops. The product identity eve-codex-worker-01 excludes that family. Each identity holds at most one unattended lease; the existing Automatic worker switch gates both. A healthy heartbeat cannot clear the other identity’s attention report.
Both sessions may use all eight vCPUs. The current AGENTS.md instructions supplied by Elikem on 2026-10-04 cap the automated platform worker at half the host workload memory budget, with a 12 GiB maximum, retaining the other half for interactive Claude. On this host the selected cgroup budget is 11,524,898,816 bytes and the next-cycle Node heap is 8192 MiB. A dated read-only receipt taken on oshun-platform-ops at 2026-10-05T11:02:34Z shows these settings applied: systemd and the kernel cgroup report MemoryMax 11,524,898,816 bytes and MemoryHigh 10,451,156,992 bytes from the persistent drop-in 99-dedicated-resources.conf (written 2026-10-04 14:17 UTC), so they survive a reboot; runner.json has memoryBytes 11,524,898,816; the selected runner source is release aabb3e95d0c13a6a4b463b67e426c22cb5523390; that release computes an 8192 MiB child heap from this runner.json; and the running Codex child process carried NODE_OPTIONS=--max-old-space-size=8192 and OSHUN_NODE_HEAP_MIB=8192. Receipt: /api/content/media/d001efd2-3bce-81f5-a08b-dfa681c92512/file (SHA-256 84cb656d3283faf86bab3737e2a2db437d2b3f5d1860607e1bf1a970d6d81d17). This differs from the full-host allocation recorded in the dated 2026-10-03 instruction below. The current supplied instruction governs this recovery work; the earlier full-host receipts remain historical. Local VM administration, existing scoped Eve identities, independent review, publication boundaries and heavy-verification resource checks remain unchanged.
The new worker has only board:read, board:work, board:author, content:read and content:write. An authenticated whoami read on 2026-10-02 confirmed these scopes and identity (key id 46669d45-3fc5-4189-a090-7693b9218f38, expiry 2026-12-31T15:10:30.484Z). Its own SSH key is source-IP restricted to the platform host and uses pinned host keys; read-only id/hostname probes reached all eight hosts. No review, publication, key-administration or delegate privilege was granted. The interactive operator’s standing publication authority does not transfer to this worker.
A live codex exec probe on 2026-10-02 used the exact requested gpt-6.1-sol/xhigh arguments, ran id -un and hostname as eve-codex-platform-ops on oshun-platform-ops, and returned PLATFORM_OPS_CODEX_READY. Dedicated account authentication, Git access and scopes were checked separately. The Dev worker API and new runner were deployed from commit 9b4871180e8629e68e1543c48e4d7a6dd115c5f6. At 16:20 UTC the new runner claimed platform-ops:r102-snapshot-redis-config-2026-09-30 with fencing token 1 while the product runner retained main-tests-green:WEB token 2. The authenticated automation response listed both runners separately with recent heartbeats and no attention faults. Actual task commands inspected Dev over the new worker’s pinned SSH alias. It correctly parked when the missing platform testing profile prevented completion. Elikem subsequently approved the two prepared family profiles; both were applied as version 1 under his authenticated Dev administrator identity and read back exactly. R102 returned to ready and the timer resumed at 16:37 UTC. The global worker switch was not changed.
Product-worker recovery — 2026-10-02
Commit f39f4e6b7b3c00c22b9d7594b56f6b3fb3bf6cfd sizes startup headroom from the configured Node heap plus native allocations and the OS reserve, while retaining live memory guards. The entire interrupted checkout, six-file patch and SHA-256 manifest remain under /mnt/volume300/eve-codex/worker/recovery-20261002-operator. The replacement checkout uses a fresh WEB fencing token; no expired lease was reused. HOME, CODEX_HOME and writable paths now use the canonical /mnt/volume300 path, avoiding sandbox failures caused by the /srv symlink. The normal timer is enabled. The resumed task subsequently passed 202 focused assertions, Playwright, lint and full web TypeScript checks and committed its fixes. Its full suite exposed an existing cross-app adapter-boundary prerequisite; it is integrating the verified partial work rather than claiming the entire product task is green. Future polls use the family-aware 9b4871180e8629e68e1543c48e4d7a6dd115c5f6 runner; that upgrade preserved the running task.
Deployment evidence and remaining commissioning
Authenticated content API reads on 2026-10-02 retrieved this guide and migration record db49f644-3ce7-450c-868a-c3aea496d571 with their revision metadata. Those reads establish document contents; anonymous page titles do not establish signed-in administrator UI behavior. Build-host timers and lock deferral were verified, but the first completed post-migration content-CI receipt is still outstanding. A completion record must name its publication, commit and result. The migration record distinguishes healthy live Dev/UAT runtimes from the still-pending first full new-host Dev build. No UAT promotion or new publication is implied by these configuration checks. A fresh authenticated read on 2026-10-05, before this correction, returned this guide at revision 46 (content SHA-256 250abc62a86aa75b472cd8ed62bff63bf8f316325881bc4ecd174fa9b92daaf1) and migration record db49f644 at revision 13 (dfc92a8c3fd3dbd7d613c70bd1c576af0f924b11fbe4651673d145ab3c2f716a). Signed-in administrator automation controls remain unverified by this guide.
Verification for the additional runner: the complete isolated API harness (python3 infra/hetzner/eve/verify-api.py --source /root/platform-worker-validation --node /usr/local/lib/eve-codex/bin/node, with no HOME and a private /root/.config/oshun mount containing only disposable PostgreSQL configuration) passed 628 tests with 18 conditional skips and zero failures; its provisioning and installer suites passed 23 and 11 respectively. All tools/eve-reviewer/*.spec.mjs passed 163 tests locally; Linux runner/resource/recovery and MCP checks passed 102; test_resource_policy.py passed eight. The model/tool probe, authenticated scopes, pinned fleet SSH probes and independent live leases provide runtime evidence beyond these tests. Production credentials were excluded from the final test mount.
Family testing profiles: platform-ops and main-tests-green version 1 were applied on 2026-10-02 by human.dev.666d8bfd-4497-4fd0-87c6-a391c5aa8440 after Elikem explicitly approved them. (The approvals and application readbacks of versions 1 and 2 are not retained here as primary records and are awaiting verification.) Their test commands select changed Node tool specs, Vitest specs from each project and Python infrastructure specs through pytest. Both require browser proof for UI changes, preserve the remaining AGENTS.md gates, enforce isolated test fixtures and serial expensive verification, and grant no extra role authority. pytest is installed on both worker hosts. The application-test model binding is separate from each Codex worker’s configured engine. Current readback (authenticated autonomy-profile read, 2026-10-05T11:02:13Z; full profile contents and update metadata retained at /api/content/media/65ee2cd1-8189-8747-bf7b-214b36146dc5/file, SHA-256 1d5d342b2b9fd4bb2221c84f275d5b635d55ea97738cadc9949ec9b22cd985cb): platform-ops version 4, updated 2026-10-05T08:48:40.904Z, and main-tests-green version 4, updated 2026-10-05T08:49:29.986Z, both updated by human.dev.666d8bfd-4497-4fd0-87c6-a391c5aa8440. Version 4 adds a serial node --test run of changed tools/**/*.test.mjs files to both profiles and extends main-tests-green’s mobile Jest runner to apps/arete/mobile/, apps/hestia/mobile/ and apps/demeter/mobile/. The earlier version-3 snapshot (platform-ops updated 2026-10-03T22:18:36.844Z, main-tests-green updated 2026-10-04T21:54:43.877Z) is superseded; its readback is not retained as a primary record and is awaiting verification.
At 16:21 UTC all five Build/CI timers were restored and the normal controller selected 9b4871180e8629e68e1543c48e4d7a6dd115c5f6. Full application release and the first completed post-migration content-CI receipt are still pending. The API-only upgrade does not establish that application release. UAT remains at its recorded tested code and read-only publication; no UAT promotion was requested.
Recovery and operating evidence, 2026-10-02 20:50 UTC: the additional platform worker uses gpt-6.1-sol with model_reasoning_effort="xhigh", verified in its live codex exec arguments. It completed R10.2 with 60 snapshots and same-day deduplication, pushing a9ae228d08663224aeb474166340b146c19ce580. Its following R03.3 task pushed the comment-only de7f9e4420e02f88612f4a0ebcd2009b4e9f181e and is being parked for the required key-authority action; it did not grant the BFF a new scope. Product ADMIN and ADMIN-MOBILE submitted their fixes. The recovered MOBILE implementation ef3f7af55825ff74733febdb6fa9e062cf02d24d is on main after 247 suites / 1,446 tests, mobile typechecking and Eve checks passed; final product composition and completion are still running at this observation.
Main-tests-green profile version 2 corrects the mobile runner: npx jest --runInBand --runTestsByPath <spec file> (from the project directory), when a change under apps/oshun/mobile/. Vitest remains selected for its named web/admin/BFF/tenant projects and libraries. Elikem’s Dev administrator session applied this correction; its readback is not retained here and is awaiting verification. Both profiles have since moved to version 4 (see the current readback above). Compose-product counts Jest Tests: summaries from f973e2a9de3; all 19 composition tests passed. These profiles retain every other applicable verification gate.
Future worker polls use root-owned runner source cd2cf2503bd28c5aa8b08a1122bfff3d4198d82a. This retains bounded retry of transient heartbeats with the same request identity, safe failure-code diagnostics and unchanged authorization failure handling. NX_CACHE_DIRECTORY points to each worker’s writable .local/state/oshun/worker/nx-cache, explicitly carried into the child process; the platform cache no longer points at the product host’s historical shared mount. The environment and installer regressions passed 12 and 11 tests on both Mac and Linux. The product account also has npm and npx shims alongside its existing Node/pnpm tools under /usr/local/lib/eve-codex/bin; run these from its repository or home, not root’s home.
Dev edge connection repair: Node’s Eve board API closes idle connections after five seconds. The Caddy Eve route now uses transport http { keepalive off } for 127.0.0.1:18883, preventing reuse of stale loopback sockets; it adds no automatic mutation retries. Commit 1aab2e53e94 passed 69 Linux release/host-scope tests and was applied by validated reload with the previous files preserved. Sixty authenticated board reads and six document reads passed afterwards, and both environments passed their 20-route smoke checks. See the migration execution record db49f644-3ce7-450c-868a-c3aea496d571 for application CI status; an API/worker repair is not a complete application-release receipt.
Independent review correction and commissioning evidence — 2026-10-02 21:41 UTC. Review 8dd75df5-c68e-4aa4-9950-d19bd49aa546 found a real claim-control bypass: a dedicated worker could omit mode or ask for interactive mode to escape capacity and the human off switch. Commit 34e989b142b56786fd5ae4843c19cd488725de3c, pushed to origin/main and now reported by the live Dev API health endpoint, applies workerAutomationClaim to both dedicated identities regardless of requested mode. Existing jobs can finish while the switch is off; no new claim is allowed. Independent interactive identities retain their previous behavior. Family routing, fencing and scoped access remain enforced.
Baseline and corrected PostgreSQL proof: the old 9b487 API implementation failed the new omitted-mode second-claim regression with an unexpected completed claim. The corrected real-PostgreSQL API/routing/advance run passed 101/101, provisioner 23/23 and installer 11/11. It exercises both dedicated identities across omitted, interactive and unattended first claims, every second-claim mode, off-switch claims, and independent interactive identities. Command: python3 infra/hetzner/eve/verify-api.py --source /root/platform-worker-validation --node /root/.nvm/versions/node/v24.14.1/bin/node --test api.spec.mjs --test lib/worker-routing.spec.mjs --test lib/advance.spec.mjs. The transient systemd unit mounts only fixture credentials at /root/.config/oshun, uses the disposable localhost PostgreSQL instance, and cannot fall back to a live worker key. Full logs: /api/content/media/6fe52588-e5af-8035-bf89-c54270ae2614/file and baseline /api/content/media/ef99c213-7148-84f6-8ef7-8b9c15d78106/file.
Reviewer context repair: the same commit resolves canonical Eve workspace document links in database-authored task bodies through the scoped content API, pins every page to a consistent revision/hash, and reports unreadable context explicitly. It never fetches arbitrary URLs or treats document assertions as proof. Paged text, exact revision, spoofed host and mismatched identity cases are covered. node --test tools/eve-reviewer/reviewer.spec.mjs tools/eve-reviewer/evidence-files.spec.mjs passed 90/90. Log: /api/content/media/240a084c-7b1a-8fd2-8659-0865955a2928/file. The reviewer host has staged the root-owned 34e989 runner for its next poll; unprivileged imports and systemd validation passed, and its active review was preserved.
Host commissioning readback at 21:30 UTC: /api/content/media/7bba0911-671e-8299-ab7e-fed5f240325d/file. The live codex exec process selects gpt-6.1-sol and model_reasoning_effort="xhigh". The service user is eve-codex-platform-ops, ProtectSystem=strict and NoNewPrivileges=yes, with writes restricted to its own checkout, state and Codex directory. CPU quota is unlimited across the shared VM; MemoryMax=11524898816 and MemoryHigh=10451156992 reserve capacity for interactive Claude and the OS. The four private authentication/SSH files are owned by this account and mode 0600. The pinned known-host set contains all eight fleet hosts. Each host has exactly one matching platform-worker authorized key, restricted by restrict,from="84.247.129.82,127.0.0.1". A separate operator key-list read confirmed identity eve-codex-platform-ops-01 has only board:read, board:work, board:author, content:read and content:write, expiring 2026-12-31. No secret values are included in this record.
Observed execution: R10.2 pushed a9ae228d08663224aeb474166340b146c19ce580; the release-record deduplication fix pushed a0ea277c22d; at 21:29 UTC the platform worker held platform-ops:r065-product-harness-2026-10-01, and the product worker was handling main-tests-green:MOBILE review changes. Both had recent heartbeats and no attention flag. The product, platform, reviewer and delegate identities are distinct. Platform-ops testing profile version 1 and main-tests-green version 2 were applied through Elikem’s approved administrator session; neither adds privileges.
Application-release acceptance is still pending. The older 010420 CI run was stopped before deployment to avoid replacing the new claim-control API with the old version. Normal main CI restarted at 21:40 UTC after verifying 22 GiB available RAM, 8 GiB swap and 198 GiB free disk, with no competing heavy verification. All five CI timers are active and memory supervision runs every 30 seconds. Two preflight attempts encountered identity_unavailable 503; the next full preflight and 20 serial content reads passed. This intermittent identity lookup has not been represented as permanently fixed. The migration record carries the final release receipt separately.
Recovery review correction — 2026-10-02 21:58 UTC. Review 84ca7fbc-ce71-4e9e-a7c8-d06926900fee correctly found that forwarding NX_CACHE_DIRECTORY alone did not make its destination writable in Codex’s narrower sandbox. Commit 0448d50fb0c38a1b4a264b3978509bc0b436aaff is pushed to origin/main. Workers and checkers now share the cache-directory helper, which creates state/nx-cache at mode 0700 and grants that exact directory. Existing credential and Codex authentication denies remain intact. This is a cache grant, not broader filesystem access.
Native sandbox proof: the new regression runs the generated worker and checker permissions through the installed Codex Linux sandbox as eve-codex-platform-ops. Removing the Nx grant makes the write fail; restoring it permits the cache write; reads of synthetic authentication files still fail, and the checker also cannot read its synthetic Eve credential. The fixture lives outside the checkout and /tmp so default writable roots cannot conceal the missing grant. The regression was also repeated under the worker’s actual configured XDG cache parent. No model or real secret is invoked by this test. Full worker/reviewer/delegate suite: 106/106 on Linux, zero skips; Mac: 104 passes and two Linux-only skips. Commands: node --test --test-concurrency=1 tools/eve-reviewer/cache-sandbox.spec.mjs tools/eve-reviewer/worker-recovery.spec.mjs tools/eve-reviewer/reviewer.spec.mjs tools/eve-reviewer/delegate.spec.mjs, then node --test tools/eve-reviewer/cache-sandbox.spec.mjs with the runtime XDG_CACHE_HOME. Logs: /api/content/media/9bcacc6c-38ac-8bf5-85ce-5d46ba08ac6e/file and /api/content/media/e43f8fcc-28ba-8b1f-8d6d-4332b7eac347/file. Old-code baseline failures: /api/content/media/65d0ca74-93f5-8df0-94d2-b1a757f3e657/file. The first broad Linux fixture lacked ESLint for two schema-validation tests; installing the same 10.4.0 version used in the repository enabled the complete passing run. Lint, formatting and Eve preflight also passed.
Runtime rollout completed at 21:57:31 UTC. All four unprivileged agent hosts stage the root-owned 0448d50 source for their next poll: product 46.62.222.247, platform 84.247.129.82, reviewer 84.247.174.203 and delegate 84.247.128.180. Imports as the service user, systemd-analyze verify and active timers passed on each. The override sets the corresponding account’s NX_CACHE_DIRECTORY to its own state/nx-cache. Model choices, credentials and resource budgets were preserved. Already running jobs retain their existing process and will receive the new sandbox on their next run.
Recovery archive, ownership and disposition proof: /api/content/media/7c459937-08d7-800a-bcf6-7037ac2bd325/file. The original checkout, two patches, original/retired recovery records and manifest remain at /mnt/volume300/eve-codex/worker/recovery-20261002-operator, owned by eve-codex-worker, protected by mode 0700. All six preserved source files match the original SHA-256 manifest. The staged patch is empty, the working-tree patch is retained, and the preserved checkout contains no pending Eve request files. The record includes checksums, the original base/branch, and the worker’s fresh WEB token 2 and subsequent parking event; the expired token 1 was not revived.
Five of the six preserved test changes were reworked and pushed by Eve Codex Worker in d9e2a8f3b05d5b1fb96c9067e82d38bc607e51cd. The proposed audit-console-errors exceptions were preserved in the archive but not landed: the strict existing audit stays unchanged, while TourPlayer and WorkbenchInspector received actual effect-dependency fixes. WEB remains honestly parked upstream for the cross-app booking adapter repair; it was not marked green. That worker’s later ADMIN, ADMIN-MOBILE and MOBILE submissions are recorded in the attached proof, including MOBILE’s 48ead2091d5dc4b2b9096d227724b0098befa6ab resubmission. Eve’s human-controlled automatic-worker switch remains enabled, with separate product/platform heartbeats. This completes runner recovery evidence, not the separate first-full-application-release acceptance for platform-ops:eight-host-migration-2026-10-02.
Live resource guard correction — 2026-10-02 22:36 UTC. The platform worker paused at 21:43:54 and again at 22:10:30 with “Worker approached its service memory ceiling”. The second supervised run showed inactive filesystem cache, rather than active allocations, dominated memory.current while about 22 GiB remained available on the host. Commit a4bb2f55b0b4d17ca736fa05e1c0b2ca53051b19, pushed to origin/main, now discounts only clean inactive file cache. Dirty/writeback pages, active cache, anonymous and slab memory remain counted. Missing, malformed or inconsistent cache counters retain the total-usage guard. The 2 GiB host reserve, 2 GiB disk reserve, 512 MiB service working-set margin, kernel MemoryHigh/MemoryMax, heap limits and authentication permissions are preserved. Kernel accounting reference: https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html .
Verification: node --test tools/eve-reviewer/resources.spec.mjs tools/eve-reviewer/worker-recovery.spec.mjs tools/eve-reviewer/reviewer.spec.mjs passed 100/100, zero skips; ESLint, Prettier, git diff --check and Eve preflight passed. Full test log: /api/content/media/668d2241-e87b-8bc3-8f65-3ee66a63d81b/file . A native Linux transient cgroup with a 1 GiB kernel ceiling wrote/fsynced a 400 MiB fixture; the real sampler measured 447434752 total bytes and 27992064 working-set bytes. A scaled diagnostic guard threshold rejected the old total-only sample and allowed the corrected sample, while low host memory still refused. It used no model or credentials. Full command/output: /api/content/media/80a676e1-cead-83c5-8377-36c8505a2591/file .
Both worker hosts stage a4bb2f55b0b4d17ca736fa05e1c0b2ca53051b19; service-user imports, systemd validation and active timers passed. Product work already in flight was preserved. The platform worker resumed at 22:25 UTC on R065’s same valid token 1, with gpt-6.1-sol/xhigh, the Nx cache grant and Codex auth read denial verified on the child process. Its four modified/untracked implementation files, patches, recovery record and manifest were preserved under /srv/eve-codex/platform-worker/recovery-cache-guard-20261002 (private, owned by the worker). Across 64 samples over 632 seconds, it remained active; 10 samples exceeded the former stop threshold. Total usage reached 10451075072 bytes while the kernel reclaimed cache, working-set usage peaked at 2854547456, and host available memory stayed above 23291924480. Kernel max/oom/oom_kill counters remained zero. Snapshot and samples: /api/content/media/55266292-cfb8-80c5-9474-4ab6df715641/file . The R065 task remains owned by that worker and is not completed by this recovery.
Reviewer evidence follow-through: commit 933dcc37879cd7dbc5f666d501e73fd0895fb905 lets task reviews download media named in their submission and linked guide through the existing scoped, bounded, hash-checked collector. It adds no reviewer credential access inside the command sandbox. The reviewer/evidence tests passed 91/91 after integration; the reviewer host stages this version for future polls without interrupting its active review. A read-only assistant probe retrieved all eight earlier guide attachments without missing context or media, while correctly retaining the lack of a trusted exact-commit CI receipt as a limitation: /api/content/media/e44b3f5f-cab1-8f14-8374-5f24ccf6c07c/file . No review decision was made by the author.
At 22:28 UTC both automatic workers had fresh independent heartbeats and no attention flags; the reviewer and delegate also had recent heartbeats. All 40 public HTTPS deployment checks passed across Dev and UAT at 22:29:37, including private-path 404s and redirects staying in their environment. The main application release at 34e989b142b56786fd5ae4843c19cd488725de3c is still running and is a separate migration acceptance check. Its broad Nx test and typecheck/lint phases exhausted their configured budgets and are recorded incomplete, not passed. Its main API/database suite completed with 628 passes, 18 skips and no failures. No publication freeze, approval or UAT promotion was performed.
Worker recovery follow-through — 2026-10-03 08:05 UTC. Both approved family testing profiles remain applied; no additional administrator privileges, content approvals, freezes or UAT promotions were granted.
Bounded content-read resilience: 08990280fb665248ead27e1f3fd35f6f802461ad retries only identity_unavailable with HTTP 503 on document, media-metadata, publication and link-resolution JSON reads, at most three attempts with one- and two-second waits. Every attempt reauthenticates. Mutations, 401/403/429 responses, other transport errors and revision/idempotency conditions keep their previous behavior. All four unattended-role hosts stage this client. The 29 transport/preflight/publication tests passed, including revocation during retry and exhausted outages; 20 serial live reads and content preflight passed. The underlying CMS-to-board latency cause is not proven; a persistent outage remains a refusal. Tests: /api/content/media/b268d04c-ba64-8be6-85c0-9cf067eaaacb/file ; live timings: /api/content/media/fe4b0635-9145-8d4f-a426-0b250dc0ae0a/file ; runtime staging: /api/content/media/19d5ac08-71f7-8dbf-b3df-4fd2eda1ebcc/file
Platform worker R362 originally pushed 01ac2ada7e5ad6b001c791606bcdab1420c2da62, but two content identity failures prevented completion before token 1 expired. Its clean checkout, empty pending queue and pushed implementation were verified, and its evidence and recovery record were archived privately at /srv/eve-codex/platform-worker/recovery-r362-20261003. Normal polling claimed fresh token 2. The worker independently refreshed verification, added failed-release coverage and submitted d45f3dd106f30f589e86e0abf707f88093717332 at 07:57:43 UTC, work product 18. It passed 123 Python, 13 recorder/parser and 97 disposable-PostgreSQL API checks; composition independently reran 85 assertions. The runner finished cleanly at 07:59:13 and advanced to the next platform task at 08:00:21. This is submission for independent review, not author approval. Recovery: /api/content/media/7d6a584e-59b3-844f-91ef-870f135aee39/file
Product worker G13.1 stopped at integration because its fresh worktree had no package-local dependencies. Its six-file local commit 5cc5c7e58571d257f09b701f31c4113fbfccf639 was superseded by claude-board-worker’s separately shipped e94ce4b2d451786a39f2859a1a4be2f0de462c86. The original branch, Git bundle, all six files and 91 evidence/recovery entries are preserved under the worker-owned mode-0700 /mnt/volume300/eve-codex/worker/recovery-g131-superseded-20261003. Checksummed manifest: /api/content/media/2e246186-2591-855a-ad63-aef5443dcc8a/file ; verified clean rearm with no pending operations: /api/content/media/7fe71bbf-771a-84c6-86f0-59fd33b900e6/file. The normal runner now works from current main and claimed R12.2 token 7 at 08:03:25; obsolete code was not merged over the completed task.
Fresh integration dependencies: 4e9303f3ba4c3b92b68589b73c4bc55ff9b8dce1 uses the existing composer’s frozen, filtered, lifecycle-disabled, serial and supervised install before integration gates. Install failures retain the worktree and prevent gates/pushes. Lockfile bytes must remain unchanged even if pnpm reports success. Linux cgroup guards are preserved; interactive non-Linux callers use conservative free-memory and disk guards. Both worker runtimes stage this revision; the active ops job was preserved. A real workspace fixture proves a transitive dependency advanced on main is resolved from the rebased worktree while the lane still reads its old version. All 63 integration, resolver, composer and process/fixture tests pass under the repository’s pnpm 10.25.0. Tests: /api/content/media/ada18d31-974f-8bdb-8e4d-2f32c708f574/file ; ops staging: /api/content/media/0c81531d-0e26-8c65-891a-82360843f4d7/file. Initial Mac resource-dependent checks were refused below 2 GiB free; the complete verification ran on the idle worker VM. An unpinned pnpm 12.5.1 test environment changed synthetic lockfile behavior; the final run used the repository version and also exposed and fixed a silent malformed-lockfile rewrite.
At 08:04 UTC both worker heartbeats, reviewer and delegate were fresh with attention cleared: /api/content/media/4398cbdc-fe64-8b82-a647-0c0ff1b5c0cf/file. The full application CI release remains a separate migration acceptance check. Dev and UAT public routes and the new Dev backup repository remain healthy; this recovery did not change the frozen UAT publication.
Bounded identity lookup recovery — 2026-10-03T15:22:41.822Z. Identity implementation 566eeca0ee0f8f4d1f33d4e8e50373d6b3738796 and tested deployment 743a252fc95c547279b25f5524ee5534330b1437 are on origin/main. Read-only diagnosis observed a 6.253535-second PostgreSQL board advisory-lock wait and valid direct identity responses taking 7.1–8.9 seconds. This exceeded the CMS fixed five-second loopback identity deadline and caused identity_unavailable/503 during content reads, evidence uploads and release preflight. The existing bounded read retries mitigated symptoms but could still exhaust their attempts.
The CMS now reads EVE_CONTENT_IDENTITY_TIMEOUT_MS through @oshun/config. Unset configuration retains five seconds; an explicit integer from 1000 through 20000 milliseconds is accepted, and malformed configuration fails closed. Managed CMS units set 20000 milliseconds. Every request still checks the current key and scopes with the existing worker API, including revocation and rate-limit responses. No authorization cache, credential scope change or mutation retry was added. The deadline accommodates measured queue latency; it does not eliminate serialization or promise success beyond the bound.
Focused verification used a real worker API in a separate process and a disposable PostgreSQL database holding the same advisory lock for 6.5 seconds. The default deadline refused, the configured 20-second deadline succeeded, revocation while waiting refused, a shorter configured deadline refused, malformed values failed closed, and the existing 429 behavior remained intact. Three auth/rate-limit tests and 47 service integration cases passed without skips. Ten provisioning and 12 content-runtime checks passed after integration. TypeScript, lint, formatting and commit hooks passed. Logs: /api/content/media/a08fdc96-dbab-8026-91f9-93616a4ff9a1/file ; /api/content/media/35e45084-87bd-83ef-b8d7-02c6453b234c/file ; /api/content/media/aada8f82-38f0-89d4-a6c0-080350ebe0b8/file ; /api/content/media/0385a73b-6c67-8fd8-b189-2c6e5ac1caae/file.
After correcting a measured BFF fixture startup margin, the exact committed source then passed the complete versioned controller CMS verification, including infrastructure, MCP/content tests, typecheck/lint, isolated persistence suites, production Next build, writable browser suite and the separate read-only browser boundary. Its verification receipt and all retained runtime bytes were checked before provisioning Dev. The initial launcher used origin/main in a bare repository that stores FETCH_HEAD; this setup error occurred before verification and was corrected. Writable browser: /api/content/media/75e06016-27dd-8a73-a1b2-617e0df7ac56/file ; read-only browser: /api/content/media/80d63fa4-3488-8e3f-811e-8f610a6d049f/file ; complete log: /api/content/media/5e1b946c-d6ca-832e-8b15-00a6d1161305/file ; resource samples: /api/content/media/a5ce0df7-6714-809c-ab97-e79786d94f92/file ; retained artifact: /api/content/media/ff4e8e3e-7254-875b-ae99-46feda401806/file ; deployment: /api/content/media/c1133912-7cc4-8b54-8e80-fb85dca49c11/file.
CI fixture integration corrections 54dd81db0465a35e1fc81bdf56c07be55e777b8a and 469c8e860fe12e6a098b4ad1d19a41f93d5bd026, included in deployment 743a252fc95c547279b25f5524ee5534330b1437 (platform-ops:mcp-ci-disposable-postgres-2026-10-03) supplies the existing private PostgreSQL fixture to the fast tier and both release MCP gates. PostgreSQL runs under its unprivileged account, and the new standing-decision MCP regression accepts only an explicit loopback database ending in _test. The standalone worker path still starts its own disposable cluster. This keeps all eligibility, refusal, escalation and replay assertions enabled. The original full run failed on the root-only caller assumption before its production build: /api/content/media/7fd1fcab-a83a-86ef-97ab-1d96004b8548/file. After correction, 92 controller/fixture cases and 83 root-owned CI MCP cases passed without skips, followed by all 29 MCP cases under the existing unprivileged CI account: /api/content/media/57e0aa2a-7d84-8579-8fe1-d4a3728a8ee2/file ; /api/content/media/b6c1d6e9-7c60-8d0e-80ea-d5605b7729fe/file. A full-platform follow-through isolated the new fixture in mocked publication-rollout tests; the final 335-case platform suite passed without skips: /api/content/media/a210cd9c-7649-801e-ab1a-f9394b4bb49d/file. The intermediate full-suite failure is preserved at /api/content/media/45c5ba22-0206-80f2-a20e-3ddf74d249a7/file. Both task commits and exact per-file Git identities are recorded at /api/content/media/b63a905d-264f-872b-83bc-6a8e1a126b0f/file. No live database role or service account privilege was changed.
The preceding CMS run at 469c8e860fe12e6a098b4ad1d19a41f93d5bd026 passed through the production build, then the BFF board-filter browser fixture exceeded its 60-second startup window. Two isolated launches took 54.654 and 54.429 seconds, including one with production NODE_ENV. Migration follow-through 82b8e72c5504caf6bcae74d3733a1d20cd849ec4 provides a bounded 180-second startup window while preserving every browser assertion and per-test deadline. Its 14 runtime checks passed. The next full run started the browser successfully but exposed the task-initialization and suite-budget problems recorded below; no failed browser run was converted into a passing receipt. Failure: /api/content/media/0607221d-7e5e-8632-89ae-35ba5b9db30d/file ; measured startup: /api/content/media/4a2f6c44-8322-8013-9a3c-fbdf6b27f5ad/file and /api/content/media/f84d08b7-24ea-8bd3-8b41-c69dc21dd578/file ; focused checks: /api/content/media/2f052055-6ab5-8e51-982f-a03092df31b1/file.
Task initialization correction 9472a98f30e677d7d898eb3303610539ebb15060 (platform-ops:task-workspace-initial-read-order-2026-10-03) loads the selected task before background board reads, shares the task count with the shell, and holds draft controls until the initial context settles, including failure. The earlier complete run measured a 5889 ms task load against the unchanged 5000 ms requirement and exposed a draft form closed by a late initial response. Two new real-browser regressions and all four originally failing scenarios passed against a fresh production build, alongside lint and typecheck. Desktop/mobile layouts were visually inspected. Targeted proof: /api/content/media/9c69ffbb-b2a3-8523-8394-b0fa8d7a31fc/file ; log: /api/content/media/7ed00291-2a38-89e4-9a2c-26c086e66ce4/file ; desktop: /api/content/media/a9969acd-5cb4-8925-9bb1-bc187411a8a1/file ; mobile: /api/content/media/0d5b3e31-6586-8439-b8dc-b928c7b9cb8c/file. The full earlier failure and sanitized request timings are retained: /api/content/media/b1a5adfe-845b-838e-bd95-6a4c069e0b45/file ; /api/content/media/1564bf41-54fc-87e5-a35b-d8d909bfebcf/file.
Migration follow-through 971f05510f9caf1d5f05fb3ddcdd501793775666 permits 45 minutes for the expanded serial browser suite after 114 of 118 cases finished in the previous 30-minute window. Two complete multi-step standing-decision scenarios receive 60 seconds after measured successful runs of 28.0 and 29.6 seconds and earlier timeouts at about 32 and 35 seconds. Assertions, per-step waits, and the five-second task-load requirement remain intact. Fourteen runtime checks passed: /api/content/media/2f052055-6ab5-8e51-982f-a03092df31b1/file. The subsequent integrated run failed before build when concurrent board routing work added an unconditional board index to the standalone API schema: /api/content/media/71da71a3-516e-8b81-ad79-0ffe176f4e8a/file.
API schema correction 743a252fc95c547279b25f5524ee5534330b1437 (platform-ops:api-schema-optional-board-index-2026-10-03) preserves standalone recovery without a board schema and creates the routing index when board events exist. A real PostgreSQL regression proves delayed board initialization, actual index use and idempotent replay. All 12 recovery and 25 restricted-role provisioning checks passed. The first ad hoc provisioning invocation lacked Node on PATH; the corrected command reran all 25 cases successfully. Initial command evidence: /api/content/media/fbed8312-dc99-821c-bc28-56ee1234420b/file ; recovery: /api/content/media/8f855fc3-6596-813e-a44b-7522a32f0943/file ; provisioning: /api/content/media/c88ed022-2b76-807b-838b-d8a79ee0ee48/file ; exact file hashes: /api/content/media/154a8f5e-eaec-8d30-a6b0-fcadc6767897/file. No live role or database privilege changed. The final complete CMS verification referenced above covers these corrections together.
An intermediate run passed all 88 content infrastructure, two backup and 352 platform checks, then lint found an untracked 163-byte macOS AppleDouble metadata file left by the earlier UI overlay transfer. Its magic bytes and untracked status were verified, the sidecar was moved to a private backup, and the exact committed source was rerun from the beginning. No source code or lint rule changed. Intermediate failure: /api/content/media/f6c6b6f9-14e3-8a29-bcdc-dce1d9ac12ed/file ; preserved artifact hash and cleanup: /api/content/media/9e490e31-7593-89d0-98d2-5162221d6a25/file.
Additional worker commissioning evidence addresses the latest needs-evidence review of platform-ops:automated-platform-worker-2026-10-02. A read-only operator-authenticated query of the Dev key metadata selected only id, agent, scopes, dates and revocation state for the existing platform worker key; it confirms exactly board:read, board:work, board:author, content:read and content:write, with no revocation and expiry 2026-12-31. It did not read a secret or digest, use the worker credential, or change privileges: /api/content/media/21e5a176-f26d-8afb-a542-35011e7d23d7/file. Authenticated Eve API reads using the assistant’s own identity recovered the actual platform worker claim and completion under fencing token 2 for R362, commit d45f3dd106f30f589e86e0abf707f88093717332, work product 18: /api/content/media/fc6bbb9a-d798-843b-9817-71df9f266ec2/file. The full application Dev release receipt remains tracked by the migration task and will be linked before requesting renewed independent review.
After installation, 20 sequential authenticated content reads passed without retries while the normal workers remained active; the longest took 4880 milliseconds. Proof: /api/content/media/5710b5c3-551a-8bb0-98a6-a8645d64dff9/file. The evidence attachments in this revision were uploaded through the same scoped content API after recovery. Full application Build-to-Dev acceptance remains the separate migration task until its entire controller run and post-deploy turn pass. UAT code/publication and all existing privileges were preserved; this guide revision is submitted for independent review, not self-approved or published.
Codex subscription recovery — Elikem, 2026-10-03. Both automatic workers, reviewer and delegate now share the 15-minute quota wait introduced in e1514e6d07ebeac21d94e15af5727940a7660589. It applies only to recognized provider usage-limit errors from codex exec error events. Authentication failures, cancellations, unsafe resource pressure and other errors retain their existing recovery policy. Models, reasoning settings, account credentials, scoped privileges and human automation switches are unchanged.
While waiting, codex-quota.json is private, atomic, durable scheduler state in each runner’s state directory. Ordinary one-minute timer polls keep Eve heartbeats current but do not start model work or claim new jobs. Every 900,000 milliseconds after the last attempted probe, a bounded, short-lived stdio Codex app-server connection calls only account/rateLimits/read. It receives no Eve credential and retains no raw account response. All reported primary/secondary windows and limit buckets must indicate usable allowance. Missing metadata, authentication refusal or a failed probe retains the wait until the next 15-minute check. The old resetsAt deadline alone never authorizes a restart.
An early banked reset, or a reset applied externally by Tibo, is detected from the next quota response showing restored allowance, even if the previously advertised reset deadline is still in the future. No callback is required. Detection is normally within 15 minutes plus the runner timer’s scheduling delay; successful probes persist a deterministic per-identity restart stagger of up to 90 seconds. No reset credits are redeemed automatically and there is no model, billing or account fallback. Official protocol reference: https://learn.chatgpt.com/docs/app-server#6-rate-limits-chatgpt .
Workers retain their checkout, uncommitted edits and quota interruption checkpoint. Recovery rechecks identity, assigned family, task definition and dependencies, journal, pending API mutations, Git conflicts, current ownership and fencing. An unchanged live lease can continue; a normally expired lease is reclaimed under a fresh fencing token, with its request ID persisted before transmission and read back after acknowledgement. Eve’s normal sweep increments the task version once and removes the claim; only the matching lease-expired journal entry explains that increment. Another owner, intervening changes, unanswered questions, unresolved API writes or an unverifiable checkpoint require attention. Already completed work is reconciled only to the original actor/token and a clean, pushed checkout; it is never completed twice. Recovery starts a fresh codex exec turn with explicit current fencing and preserved context because these runners use ephemeral sessions.
Reviewer and delegate jobs stop without submitting partial review reports or decisions. Their exact claims are released using durable retry identities; a lost acknowledgement is reconciled before new claims, even while automation is off. After capacity returns and the human switch permits work, a fresh claim supplies the current review/decision snapshot. Turning a role off still prevents new unattended claims; quota recovery does not override this control.
Verification: the serial Linux suite passed 183 tests with zero failures, including disposable PostgreSQL plus the real authenticated worker API, normal expiry, strictly newer fencing, stale-token refusal, lost claim acknowledgement, early resets, multiple limit buckets, durable cooldowns, unknown probe failures, conflicts, dirty-work preservation, no duplicate completion, all role switches and interrupted release recovery. Two native sandbox checks skipped in the isolated Dev fixture because Codex was absent there; the deployed platform-worker host subsequently ran the complete four-assertion cache/sandbox suite with its installed CLI. The 11 installer checks, ESLint, Prettier and Eve content preflight also passed. Tests: /api/content/media/40299ffd-ffba-877e-99fc-da2e1551bdf1/file ; final formatted API recheck (5/5): /api/content/media/9316cbc7-9881-8804-a0c8-5c5e7087b9b2/file ; verified fixture hashes: /api/content/media/1c73d0d6-d643-83c1-b007-64c83c2588c6/file ; tested file hashes: /api/content/media/22e30eb0-6033-8db9-9cd2-436d77320ce3/file ; native sandbox and installation: /api/content/media/0614da25-4899-84bb-87cb-3316bf05293a/file ; installer: /api/content/media/e77a404f-6da5-8801-9cad-18bee5e00e88/file ; lint: /api/content/media/ef70907e-76eb-869d-a20f-5e893e6e14b5/file ; formatting: /api/content/media/6b30c0db-4a31-8314-9066-e6d43cf90141/file ; preflight: /api/content/media/1eb735a1-ae5c-8b06-aa4b-5af33d280721/file. Earlier failed fixtures remain private diagnostic logs: missing retained dependency links and an invalid synthetic expired-lease timestamp were corrected; no production checks were weakened.
Runtime: read-only quota probes succeeded under each dedicated service account on product 46.62.222.247, platform ops 84.247.129.82, reviewer 84.247.174.203 and delegate 84.247.128.180; each reported the 900,000 ms polling constant: /api/content/media/a157d01f-82fe-8445-b3f4-55a13c742bae/file. Root-owned versioned source and systemd next-cycle configuration are verified here: /api/content/media/7e7ec2ea-ffe3-8b9d-acb6-184ccbfaee59/file. The rollout reloaded unit definitions without restarting active work; current jobs retain their previous process until completion, and subsequent timer invocations use e1514e6d07ebeac21d94e15af5727940a7660589. Existing recovery records were not relabelled as quota failures. Eve controls and heartbeats: /api/content/media/9108e7ee-0716-869d-8854-b0efa141f6d2/file. This guide is submitted for independent review; no author approval or publication is performed.