Eve library

@bellona/remote-control-testing

Other

Revision 1

On this page

Bellona remote-control mock harness and protocol fixture utilities for Phase 180 tests.

This package is intentionally test-only. It imports the canonical @bellona/remote-protocol contracts, validates every generated fixture against those schemas, and provides in-memory mock device, adapter, host, and gateway objects without opening network sockets or talking to real desktop, browser, Blender, Unreal, or WebRTC surfaces.

Utilities#

  • createMockRemoteDeviceFixture
  • createMockRemoteAdapterFixture
  • createMockRemoteCapabilityFixture
  • createMockRemotePermissionFixture
  • createRemoteControlCommandFixture
  • createRemoteControlBridgeCoexistencePlan
  • createRemoteControlBridgeRetirementGatePlan
  • createRemoteControlProgressFixture
  • createRemoteControlResultFixture
  • createRemoteControlAuditEntryFixture
  • createRemoteControlSnapshotFixtureSet
  • createRemoteControlDeploymentSmokePlan
  • createRemoteControlMvpAlphaDemoPlan
  • createRemoteControlMvpAlphaReleaseGatePlan
  • createRemoteControlFinalGaReleaseGatePlan
  • createRemoteControlReleaseGateAlignmentPlan
  • parsePhase180Checklist
  • parsePhase180Tracker
  • createControlRoomOperatorUsabilityStudyPlan
  • createPassingControlRoomOperatorUsabilityEvidenceSnapshot
  • createRemoteControlMvpAlphaSoakPlan
  • createRemoteControlMvpAlphaSloPolicy
  • createPassingRemoteControlMvpAlphaEvidenceSnapshot
  • createPassingRemoteControlFinalGaEvidenceSnapshot
  • createPassingRemoteControlBridgeRetirementEvidenceSnapshot
  • createPassingRemoteControlMvpAlphaSloEvidenceSnapshot
  • runRemoteControlDeploymentSmokeScenario
  • runRemoteControlDeploymentSmokeSuite
  • evaluateRemoteControlMvpAlphaReleaseGate
  • evaluateRemoteControlFinalGaReleaseGate
  • evaluateRemoteControlBridgeRetirementGate
  • evaluatePhase180ChecklistTrackerSync
  • evaluatePhase180ChecklistTrackerSyncFiles
  • evaluateControlRoomOperatorUsabilityStudy
  • evaluateRemoteControlMvpAlphaSloPolicy
  • createRemoteControlNetworkImpairmentPlan
  • runRemoteControlNetworkImpairmentScenario
  • runRemoteControlNetworkImpairmentSuite
  • createRemoteControlPerformanceBudgetPlan
  • evaluateRemoteControlPerformanceBudget
  • runRemoteControlPerformanceBudgetScenario
  • runRemoteControlPerformanceBudgetSuite
  • createRemoteControlSecurityRegressionPlan
  • inspectRemoteControlPromptInjection
  • redactRemoteControlSecurityValue
  • runRemoteControlSecurityRegressionScenario
  • runRemoteControlSecurityRegressionSuite
  • createRemoteControlRecordReplaySandbox
  • createRemoteControlReplayCommandKey
  • createRemoteControlProtocolFuzzCases
  • parseRemoteControlReplayCassette
  • runRemoteControlProtocolFuzzingHarness
  • serializeRemoteControlReplayCassette
  • FakeGatewayTransport
  • FakeHostAdapter
  • FakeMcpClientTransport
  • RemoteControlRecordReplaySandbox
  • MockRemoteAdapter
  • MockRemoteDevice
  • MockRemoteHost
  • MockRemoteGateway

Reuse Points#

Future gateway, host, MCP, and adapter work should import this package instead of rebuilding mock transports:

  • Gateway work can use FakeGatewayTransport for outbound host registration, command dispatch, idempotency, policy/approval decisions, audit entries, progress aggregation, and host disconnect behavior.
  • Host and adapter work can use FakeHostAdapter for capability registration, command receipt, injected success/failure/timeout/cancel outcomes, progress events, and disconnect handling.
  • MCP work can use FakeMcpClientTransport for MCP-shaped tools/list and tools/call coverage against the fake gateway.
  • Contract and UI-adjacent tests can use createRemoteControlSnapshotFixtureSet for stable device capability, session timeline, Blender scene query, browser page query, and stream descriptor fixture data.
  • Adapter and agent regression tests can use RemoteControlRecordReplaySandbox to capture protocol-valid command results/progress from live or fake sessions, serialize checksumed replay cassettes, and replay those responses deterministically without requiring Blender, Unreal, Chrome, or a live host.
  • Protocol, gateway, and adapter release gates can use runRemoteControlProtocolFuzzingHarness to exercise malformed envelopes, oversized payloads, Unicode spoofing inputs, schema downgrade attempts, deeply nested JSON, and known high-risk policy-denial paths before shipping.
  • Alpha hardening can use runRemoteControlNetworkImpairmentSuite to cover the deterministic gateway restart, host reconnect, host sleep/wake, packet-loss, high-latency, and TURN-placeholder checks required by 180.C.16.04 before live network shaping and two-MacBook evidence exist.
  • Alpha hardening can use runRemoteControlSecurityRegressionSuite to cover prompt-injection containment, denied path policy, denied app/window redaction, revoked device fail-closed behavior, approval downgrade rejection, and secret redaction required by 180.C.16.05 before live malicious-page and hardware evidence exists.
  • Alpha hardening can use runRemoteControlPerformanceBudgetSuite to evaluate deterministic p50/p95/max measurements for command latency, stream startup, screenshot capture, browser navigation, Blender command time, and gateway-host reconnect required by 180.C.16.06 before live LAN/cloud histograms exist.
  • Alpha hardening can use runRemoteControlDeploymentSmokeSuite to validate the local, LAN, Hetzner, and AWS deployment smoke contracts required by 180.C.16.08 before live two-MacBook and cloud evidence is attached.
  • Alpha hardening can use createRemoteControlMvpAlphaReleaseGatePlan() and evaluateRemoteControlMvpAlphaReleaseGate() to block MVP alpha release until the required stages, docs bundle, demo scenarios, smoke suites, and package/hardware evidence are attached for 180.C.16.09.
  • Alpha hardening can use createRemoteControlMvpAlphaDemoPlan() to keep the scripted LAN happy path, cloud reference path, and safety/resilience evidence pack aligned with the MVP alpha release bundle required by 180.C.16.11.
  • Alpha hardening can use createRemoteControlMvpAlphaSloPolicy() and evaluateRemoteControlMvpAlphaSloPolicy() to keep command acknowledgement, stream startup, approval round-trip, and audit durability SLOs aligned with the MVP alpha release gate required by 180.C.16.12.
  • Alpha hardening can use createRemoteControlMvpAlphaSoakPlan() to define the 72+ hour uptime, sleep/wake, LaunchAgent recovery, reconnect, audit-growth, and memory-leak evidence bundle required by 180.C.16.13.
  • Alpha hardening and later Stage 29 work can use createRemoteControlReleaseGateAlignmentPlan() so the MVP alpha gate and the final GA gate share one required-stage matrix instead of drifting into separate checklist interpretations.
  • Stage 180.C.29.11 audits can use evaluatePhase180ChecklistTrackerSyncFiles() to keep the canonical TODOS/phase-180.md checklist synchronized with the implementation tracker: unchecked checklist items must map to blocked tracker rows with concrete verification, evidence, and blocker notes, and blocked tracker rows must map back to unchecked checklist items.
  • Stage 180.C.21.20 can use createControlRoomOperatorUsabilityStudyPlan() and evaluateControlRoomOperatorUsabilityStudy() to keep the new-operator study scenarios, timing/clarity/error/discoverability metrics, published report, and top-three friction fixes explicit. The evaluator blocks by default until real recruited-operator evidence exists.
  • Stage 180.C.29.08 through .12 and the final fleet gate in 180.C.30.10 can use createRemoteControlFinalGaReleaseGatePlan() and evaluateRemoteControlFinalGaReleaseGate() to keep the final verification matrix, release evidence bundle, product acceptance scenarios, conditional platform scope, and one-or-many host fleet closure evidence explicit. The evaluator intentionally blocks without live hardware, cloud, and operator evidence.
  • Stage 180.C.16.15 and later Stage 180.C.38 work can use createRemoteControlBridgeCoexistencePlan() to keep the legacy bridge-blender and bridge-unreal shim behavior, coexistence rules, cutover date, and final removal gate synchronized.
  • Stage 180.C.38.07 through .10 can use createRemoteControlBridgeRetirementGatePlan() and evaluateRemoteControlBridgeRetirementGate() to keep removal PR evidence, zero-consumer telemetry, package cleanup, post-removal scans, documentation, and final retirement approval fail-closed until the old bridges are actually retired.

Deployment Smoke#

runRemoteControlDeploymentSmokeSuite runs the Stage 16 deterministic deployment smoke gate. It validates the shared deployment contract for local, same-LAN, Hetzner reference, and AWS reference modes using the existing fake gateway and host transports. Each mode records its gateway profile, protocol network mode, documented smoke commands, required probes, event stream, command status, audit entries, and evidence level.

Only the local mode is treated as fixture evidence. LAN, Hetzner, and AWS modes are marked live-hardware-required: the suite proves that the reference smoke contract is complete and testable, but it does not claim a physical two-MacBook, provider edge, TLS, firewall, ALB, target-health, TURN, Blender, Unreal, browser, or desktop deployment has actually been exercised. Each result also exposes liveDeploymentEvidenceRequired and releaseSignoffEligible; live-required modes must remain releaseSignoffEligible=false until the external smoke artifacts are attached.

MVP Alpha Release Gate#

createRemoteControlMvpAlphaReleaseGatePlan() documents the Stage 16 MVP alpha release gate as a concrete checklist over required phases, docs, demo scenarios, and verification surfaces: protocol, gateway, host, policy/audit, Blender, browser, desktop fallback, Control Room, MCP, Unreal, deployment, security, performance, and network resilience.

evaluateRemoteControlMvpAlphaReleaseGate() combines the deterministic Stage 16 deployment, security, performance, and network suites with an evidence snapshot for the broader package/build/Playwright/hardware checks that cannot be inferred from this test package alone. The evaluator defaults to blocked when required evidence is missing, and it only returns passed when every required stage, docs bundle item, demo scenario, and gate check is attached as valid structured evidence. Bare checklist ID summaries are compatibility metadata only; they do not prove MVP alpha release closure without dated records that include unique artifacts, planned commands, full commit SHAs, CI environment IDs, concrete rig IDs, and operator IDs where required.

Release Gate Alignment#

createRemoteControlReleaseGateAlignmentPlan() records which stages belong to the MVP alpha gate versus the final GA gate. The alpha gate stays a strict subset of the GA gate: Stage 180.C.29.08 through .10 inherits the Stage 16 requirements and adds the production-breadth work from 180.C.17 through 180.C.28, the Stage 180.C.29.01 through .10 release checklist, and the conditional Windows or Linux scope decisions in 180.C.36 and 180.C.37.

Checklist Tracker Sync#

evaluatePhase180ChecklistTrackerSyncFiles() reads the canonical Phase 180 checklist and implementation tracker and verifies their completion and remaining-open state is intentional. The gate passes only when tracker rows stay in canonical checklist order, every tracker task starts with the checklist task text, statuses are limited to Complete or Blocked, every checked item maps to a Complete tracker row with concrete owner, changed-files, verification, and evidence cells, every unchecked item maps to a Blocked tracker row with non-placeholder verification, evidence, and blocker notes, and every blocked tracker row maps back to an unchecked checklist item.

This gate does not close the external work. It prevents the audit from drifting by making the current completion evidence and open-item/blocker mapping executable.

Operator Usability Study#

createControlRoomOperatorUsabilityStudyPlan() records the structured 180.C.21.20 Control Room study for new operators: approve under pressure, policy explanation clarity, approve/deny error rate, and takeover discoverability.

evaluateControlRoomOperatorUsabilityStudy() consumes a study evidence snapshot and returns blocked unless at least five new operators participated, every metric meets its threshold, the findings report is published, and the top three friction points have fix evidence. It also rejects duplicate, blank, non-canonical, or placeholder participant identities, including case-variant duplicates, placeholder participant roles, missing or duplicate recruited-participant observations for any scenario, metric-only observations without the scenario's required evidence kinds and unique evidence artifacts, case-variant duplicated observation evidence, stale, missing, or non-canonical UTC millisecond observation timestamps, published reports that omit recruited participant IDs or observation evidence IDs, published reports outside the canonical 180.C.21.20 evidence path, reports published before the observations they summarize, observation artifacts that do not identify their participant/scenario context with concrete non-placeholder evidence suffixes, malformed metric values, stale, non-canonical, or unparseable report timestamps, zero-duration human timing metrics, successful takeover discoveries without a valid time-to-first-action sample, duplicated friction-point IDs including case variants, invalid severity, reused or case-variant duplicated fix evidence, whitespace-only, placeholder, or non-contextual fix evidence without concrete non-placeholder suffixes, and top-three fixes verified before the findings report or without canonical UTC millisecond verification timestamps. This is local gate logic only; it does not replace recruited human-operator observations.

Final GA Release Gate#

createRemoteControlFinalGaReleaseGatePlan() records the final GA release gate as a concrete evidence contract for 180.C.29.08, 180.C.29.09, 180.C.29.10, 180.C.29.12, and 180.C.30.10. It lists the required verification checks, release evidence artifacts, final product acceptance scenarios, conditional platform stages, and fleet-closure requirements.

evaluateRemoteControlFinalGaReleaseGate() consumes an evidence snapshot and returns blocked by default. It only returns passed when every required stage, conditional platform completion or accepted scope-waiver evidence, verification check, release evidence artifact, acceptance scenario, and fleet-closure requirement is attached as a structured evidence record. Bare requirement ID arrays such as passedVerificationCheckIds are compatibility summaries only; they do not satisfy the gate without matching records carrying pass status, a canonical UTC millisecond timestamp on or after 2026-07-02T00:00:00.000Z, unique artifacts, and run context. Each final-GA requirement must have exactly one structured evidence record; duplicate records for the same requirement do not satisfy the gate. Verification records must cite one of the planned commands for that check, automated release-log records must include command, full 40-character commit SHA, and environment context, and rig/evidence lists cannot use duplicate placeholders, case-variant duplicates, or count-only artifact lists to satisfy required counts. Artifact IDs must also be unique across the full final-GA evidence snapshot, including case-variant matches, so one pasted log, capture, bundle, or placeholder cannot satisfy independent verification, release-evidence, acceptance, and fleet requirements. Acceptance and fleet records must name the required evidence kinds covered by the attached artifacts, and their artifact IDs must identify those specific evidence slots instead of generic bundles or placeholder slot context. Artifact IDs must name the final-GA record kind, requirement ID, and concrete evidence suffix, and environment, reviewer, operator, and fleet-run IDs must use contextual final-GA prefixes with non-empty suffixes. Final test-log release evidence must be captured after every verification record it summarizes, operator signoff must be captured after every acceptance scenario record, and fleet closure records must be captured after the Stage 29 verification, release evidence, and acceptance records. Conditional Windows/Linux completion requires structured conditional-platform evidence records whose artifacts name each platform evidence slot, or accepted scope-waiver evidence. Conditional Windows/Linux scope waivers must include canonical macOS-scoped release notes, target-platform scope text, current approval from an operator:final-ga-waiver-approver: identity, residual-risk ownership from an owner:final-ga-waiver-risk: identity, and unique supporting evidence under the remote-control docs tree; bare 180.C.36/180.C.37 stage or waiver IDs do not pass. The evaluator is a guardrail for release evidence assembly; it does not claim that local deterministic tests replace live macOS/Windows, cloud, or operator acceptance runs.

The GA evidence bundle includes the Stage 26 macOS host packaging and acceptance artifact: native menu-bar run, LaunchAgent lifecycle, Developer ID signing and notarization, install QA matrix, and non-developer acceptance evidence. That macOS host packaging release artifact must be backed by a passing evaluateMacOSHostStageGate() result. The record must identify both the MAC-OP-01 non-developer operator rig and MAC-HOST-01 macOS host rig. A contextual final-GA artifact ID, cloud-only rig record, or rig-only record cannot prove the native app, LaunchAgent, signing/notarization, install QA, and non-developer acceptance evidence.

The GA evidence bundle also requires the 180.C.21.20 operator usability study release artifact to be backed by a passing evaluateControlRoomOperatorUsabilityStudy() result. A contextual final-GA artifact ID and operator signature alone do not prove recruited new-operator observations or top-three friction-fix evidence.

Bridge Coexistence#

createRemoteControlBridgeCoexistencePlan() records the legacy Blender and Unreal bridge command surfaces, which commands can already translate into Remote Host protocol envelopes, which commands must stay on the compatibility path until later stages land, the 2026-10-21 hard cutover date, and the final Stage 180.C.38.09 removal gate.

Bridge Retirement Gate#

createRemoteControlBridgeRetirementGatePlan() records the Stage 180.C.38.07 through .10 removal contract for the old Blender and Unreal bridges.

evaluateRemoteControlBridgeRetirementGate() consumes removal evidence and returns blocked unless both bridge removal PRs include migration and release note links plus fresh submission timestamps, zero-consumer telemetry covers exactly every compatibility-only command after cutover with no unknown command entries, the old bridge app packages are removed after merged removal PRs and zero-consumer windows, post-removal scans prove no active app/library/service/script/test/workspace dependencies or findings remain, documentation is updated, removal PRs are merged for roll-up, and final retirement approval metadata is attached after the removal PR, zero-consumer telemetry, cleanup, and post-removal scan evidence. Duplicate PR, bridge telemetry, or command telemetry records, reused per-bridge PR URLs or telemetry sources, scan commands that do not use rg/git grep, scans that omit or only partially cover workspace metadata files, scans that mention required paths only as substrings instead of scanning exact legacy bridge package/path references, non-canonical PR, migration, release-note, cleanup, telemetry, or bridge-retirement approver identities, placeholder telemetry source or bridge-retirement approver suffixes, stale or non-canonical timestamp removal submissions, premature or non-canonical timestamp merges, stale or non-canonical timestamp cleanup records, and stale or non-canonical timestamp final approvals do not satisfy the single authoritative evidence record expected for each bridge or compatibility command. Package cleanup allowed-reference paths must be unique docs or release-note references, including case variants; active app, library, service, script, test, or workspace dependency paths cannot be whitelisted directly or through path traversal as allowed residual bridge references.

MVP Alpha Demo Scenarios#

createRemoteControlMvpAlphaDemoPlan() records the scripted MVP alpha release evidence flows. The primary scenario is the solo-creator two-Mac LAN happy path: pair the host, query Blender scene info, create one primitive, navigate the isolated browser, capture a screenshot, and complete operator approval plus replay. Supporting scenarios cover the reference cloud gateway path and the security/network/release-gate evidence pack.

Each scenario includes exact commands, required evidence, and acceptance criteria so the release bundle can point to one explicit script instead of ad hoc operator notes.

Network Impairment#

runRemoteControlNetworkImpairmentSuite runs the Stage 16 deterministic network impairment gate. It uses FakeGatewayTransport and FakeHostAdapter to prove that command dispatch survives gateway restart, host reconnect, and host sleep/wake reconnect flows, and that packet-loss and high-latency fixtures surface degraded telemetry instead of silently succeeding.

TURN fallback is intentionally represented as a placeholder scenario. The suite records that TURN relay is required for cross-network media reliability, but it does not claim WebRTC or coturn coverage until later signaling, credentials, relay, teardown, and hardware smoke tests exist.

Security Regression#

runRemoteControlSecurityRegressionSuite runs the Stage 16 deterministic security gate. It combines the canonical protocol schema, policy evaluator, and redaction-style evidence checks to prove that prompt-injection content remains untrusted data, denied file paths do not dispatch, denied app/window context is redacted before evidence exposure, revoked devices fail closed, destructive approval metadata cannot be downgraded, and known secret patterns are removed from structured evidence.

The suite is intentionally not a substitute for live browser, macOS TCC, gateway, host, or support-bundle evidence. Those later gates must still prove that real web pages, capture surfaces, host revocation, approval decisions, and production logs enforce the same rules outside the deterministic fixture harness.

Performance Budgets#

runRemoteControlPerformanceBudgetSuite runs the Stage 16 deterministic performance gate. It evaluates fixture measurements for command latency, stream startup, screenshot capture, browser navigation, Blender command time, and gateway-host reconnect using target p95, max p95, and minimum sample-count rules. The evaluator returns passed, warning, or failed so release gates can distinguish target drift from hard release blockers.

The suite records budgets and measurement sources only. It does not claim live LAN, cloud gateway, WebRTC/TURN, macOS capture, managed-browser, or Blender latency until those environments attach hardware/deployment evidence.

MVP Alpha SLOs#

createRemoteControlMvpAlphaSloPolicy() records the Stage 16 SLO targets and error-budget policy for command acknowledgement latency, stream start latency, approval round-trip time, and audit write durability.

evaluateRemoteControlMvpAlphaSloPolicy() combines a live-evidence snapshot with multiwindow burn-rate math. Active burn-rate alerts block release. Low remaining budget without an active alert remains a warning so the release note can call it out explicitly.

MVP Alpha Soak Plan#

createRemoteControlMvpAlphaSoakPlan() records the multi-day host-uptime plan for the Stage 16 alpha hardening work. It defines the minimum 72-hour run, the probe cadence, the planned sleep/wake and LaunchAgent recovery drills, the audit-growth checkpoints, the RSS/memory-pressure evidence, and the fail-fast conditions that should stop the run early.

Record And Replay#

RemoteControlRecordReplaySandbox is an in-memory cassette recorder for adapter responses. It can ingest direct live-adapter responses or existing FakeGatewayTransport dispatches, validates every command/progress/result against @bellona/remote-protocol, computes a checksumed cassette, and replays matching commands with stable result IDs, progress IDs, timestamps, and idempotency keys remapped to the replayed command.

Replay matching defaults to command namespace/name plus structured arguments, so captured Blender, Unreal, browser, or Chrome responses can be reused by offline agent and regression tests even when device IDs or fake adapter IDs differ. Strict target matching is also available when a test must bind to a specific adapter/capability target.

Protocol Fuzzing#

runRemoteControlProtocolFuzzingHarness executes deterministic fuzz cases against the canonical RemoteCommandEnvelopeSchema and policy evaluator. The default case set covers malformed envelopes, oversized IDs and idempotency keys, RTL marks, normalization tricks, surrogate pairs, schema-downgrade payloads, deeply nested argument structures, and policy-bypass attempts for project file writes, shell execution, real browser profiles, and destructive approval downgrades.

The harness returns a RemoteControlProtocolFuzzReport. blockRelease is true when any fuzz case causes a parser/evaluator crash, unexpectedly passes schema validation, bypasses a policy denial, or rejects a known-safe control case. Callers can pass custom fuzz cases to extend the release gate without replacing the baseline corpus.

Envelope Property Tests#

tests/envelope-invariants.test.ts provides a deterministic property-test equivalent for remote command envelope invariants without adding a new package dependency. The generated cases cover idempotency-key determinism, priority ordering, timeout-class monotonicity, risk-class escalation, and actor-chain well-formedness against the canonical protocol schemas.

Verification#

The package is tagged scope:bellona, so the repo's affected Nx CI jobs cover its build, typecheck, lint, and test targets when this harness or its protocol dependency changes. For local pre-merge verification, run the combined harness and Phase 180 closure gate:

  • pnpm bellona:remote-control:verify

The individual commands covered by that gate are:

  • pnpm nx build @bellona/remote-control-testing
  • pnpm nx typecheck @bellona/remote-control-testing
  • pnpm nx lint @bellona/remote-control-testing
  • pnpm nx test @bellona/remote-control-testing
  • pnpm verify:phase-180