Eve library

Automated Playtesting Bot Authoring Guide

Guide

Revision 1

On this page

The playtesting framework in src/playtesting provides deterministic bot runs for headless game clients. It is designed to run in unit tests, nightly CI, and engine-backed smoke jobs without requiring a renderer.

Core Model#

  • HeadlessPlaytestClient is the adapter boundary for a real game build.
  • SimulatedHeadlessGameClient is the in-memory reference implementation used by unit tests and early content validation.
  • BotHarness runs a BotPolicy, records actions and observations, detects crashes, stuck states, navigation failures, and combat defeats, and captures screenshots when the client supports them.
  • BotResultReporter aggregates runs into markdown plus structured summary data for CI.
  • PlaytestCiIntegration converts report summaries into gating pass/fail results for per-commit, nightly, weekly, or manual suites.

Bot Categories#

The module covers the phase-72 bot families:

  • Random exploration bot for crash, stuck-spot, interactable, door, and item discovery.
  • Coverage mapper for heatmaps and unexplored cell reporting.
  • Navigation stress tester for all known location pairs.
  • Combat bot for defeatable enemy and difficulty sanity checks.
  • Puzzle-solving bot using bounded brute-force and heuristic attempts.
  • Soak test analyzer for long-run memory growth and FPS degradation.
  • Load test bot army analyzer for multiplayer client count, tick rate, RTT, and packet loss.
  • RL exploration agent that trains a small q-table against world coverage rewards.
  • Quest completion bot for objective reachability and blocker detection.
  • Economy bot for accelerated buy/sell behavior, inflation, and suspicious transaction signals.
  • Performance profiling bot for standardized route frame and memory budgets.

Writing a Real Client Adapter#

Implement HeadlessPlaytestClient for the target game runtime:

ts
const client: HeadlessPlaytestClient = {
  reset(initialState) {
    // Launch or reset the game world, then return the first observation.
  },
  observe() {
    // Return the latest world state, available actions, and recent events.
  },
  execute(action) {
    // Translate BotAction into engine input or script command.
  },
  captureScreenshot(label) {
    // Optional: return a screenshot URI when issues are found.
  },
};

Keep the adapter authoritative. If the engine rejects an action, surface that as a PlaytestEvent such as navigation_failed, interaction_failed, or a domain-specific event in recentEvents.

CI Gates#

Recommended suites:

  • per_commit: short deterministic smoke run with low max steps and crash gate.
  • nightly: broader exploration, route profiling, navigation stress, quest reachability, economy simulation, and report generation.
  • weekly: soak/load analysis and RL exploration training with larger episode counts.

Use PlaytestCiIntegration.evaluate to gate on:

  • minimum coverage percent
  • maximum crash count
  • maximum performance regressions
  • minimum issue severity that should fail the suite

Interpreting Reports#

Reports include run counts, pass/fail counts, crashes, severity totals, coverage percent, coverage gaps, performance regressions, screenshots, and all issues with evidence payloads. Keep issue messages specific enough that content authors can reproduce a failure from the run seed and action list.