Eve library

Bellona Performance Budget Test Plan

Guide

Revision 1

On this page

Status: 180.C.16.06 deterministic fixture coverage.

Implementation date: 2026-04-23.

This plan adds automated fixture coverage for command latency, stream startup, screenshot capture, browser navigation, Blender command time, and gateway-host reconnect. It records target p95, max p95, minimum sample counts, and the local measurement sources that later live LAN/cloud/hardware runs must replace or augment.

Sources Reviewed#

Local implementation references:

  • testing/bellona/remote-control/fake-host-adapter.ts
  • testing/bellona/remote-control/network-impairment.ts
  • testing/bellona/remote-control/snapshot-fixtures.ts
  • apps/bellona/remote-gateway/src/command-dispatcher.ts
  • apps/bellona/remote-gateway/src/telemetry.ts
  • apps/bellona/remote-host/src/command-executor.ts
  • apps/bellona/remote-host/src/blender-adapter.ts
  • apps/bellona/control-room/src/stream-preview.ts
  • apps/bellona/control-room/src/webrtc-loopback.ts

Automated Budgets#

testing/bellona/remote-control/performance-budgets.ts exposes createRemoteControlPerformanceBudgetPlan(), evaluateRemoteControlPerformanceBudget(), runRemoteControlPerformanceBudgetScenario(), and runRemoteControlPerformanceBudgetSuite().

The deterministic suite covers:

Scenario Target p95 Max p95 Fixture measurement source Live evidence still required
command-latency 750 ms 1200 ms Gateway command received-to-result duration telemetry LAN/cloud command histograms for read-only and safe mutation
stream-startup 3000 ms 5000 ms Control Room stream descriptors and WebRTC loopback timing Same-LAN and TURN WebRTC startup measurements
screenshot-capture 1500 ms 2500 ms Desktop/browser screenshot result and artifact timing macOS display/window screenshot capture with redaction timing
browser-navigation 4000 ms 7500 ms Managed-browser navigation and DOM/screenshot readiness Managed profile startup, navigation, page readiness evidence
blender-command-time 2500 ms 5000 ms Blender adapter latency and command result duration Live addon scene query and primitive mutation timings
gateway-host-reconnect 5000 ms 10000 ms Gateway host online/reconnect and fake reconnect events Gateway restart, network drop, and sleep/wake reconnect runs

Release Gate Rules#

The alpha gate can use this suite to enforce deterministic target p95 budgets and hard max p95 limits. A warning means the fixture p95 is above target but below the hard release max. A failed result means the fixture has too few samples or exceeds the hard max and should block release until explained or fixed.

Before MVP alpha can claim production-ready performance, the release evidence must attach:

  • command latency histograms from LAN and cloud gateway modes
  • WebRTC same-LAN and TURN stream startup timings
  • macOS screenshot capture timings for display, window, and redaction paths
  • managed-browser navigation timings for fixture pages and safe production pages
  • Blender scene query and safe primitive mutation timings
  • gateway-host reconnect timings for gateway restart, network drop, and sleep/wake

Verification#

The committed automated gate is:

bash
pnpm exec vitest run --config vitest.config.ts tests/performance-budgets.test.ts --reporter=verbose
pnpm nx run-many -p @bellona/remote-control-testing -t lint,typecheck,test,build --skip-nx-cache --verbose

The first command proves the individual budget scenarios, warning/failure classification, and canonical suite order. The second command keeps the reusable test harness buildable and covered by Nx.