Fighting Game · Guides & deep dives

V2 RL-Trained Exploration Agent

random-baseline comparison, coverage lift threshold, and bug-discovery lead threshold.

3sections1 minread
Markdown source
On this page

V2RLTrainedExplorationAgent covers TODOS.phase-72.72.19.1.9: it models a reinforcement learning agent maximizing world state coverage and finding novel bugs faster than random exploration.

Runtime Surface#

  • FV2RLExplorationAgentConfig defines training episodes, policy step budget, random-baseline comparison, coverage lift threshold, and bug-discovery lead threshold.
  • BuildRLExplorationTrainingStates creates deterministic world-state rewards with novelty, coverage, risk, and trained Q-value signals.
  • BuildRLExplorationPolicySteps converts the trained policy into harness-compatible actions using the RL policy model path and observation schema.
  • BuildRLExplorationCoverageSamples records the world positions selected by the trained policy for heatmap analysis.
  • RunRLTrainedExplorationAgent executes the headless harness, analyzes RL coverage, runs the random exploration baseline, measures coverage lift over random, and records novel bugs found faster than random.
  • FormatRLExplorationAgentReport emits a compact coverage and bug-discovery report for QA dashboards.

QA Behavior#

The RL-trained exploration agent is deterministic and automation-first. It does not train a live model during the test run; it validates the shipped reinforcement learning policy surface by replaying trained policy decisions, measuring world state coverage, comparing against random exploration, and recording faster novel-bug discovery frames.

Required capabilities: reinforcement learning agent, maximizing world state coverage, random baseline comparison, finds novel bugs faster than random, trained policy Q-values, coverage lift over random, and headless harness execution.

Validation Commands#

bash
python3 V2/ue/Tools/check-v2-rl-trained-exploration-agent.py
python3 V2/ue/Tools/check-v2-load-test-bot-army.py
python3 V2/ue/Tools/check-v2-soak-test-bot.py
python3 V2/ue/Tools/check-v2-puzzle-solving-bot.py
python3 V2/ue/Tools/check-v2-combat-bot.py
python3 V2/ue/Tools/check-v2-navigation-stress-test-bot.py
python3 V2/ue/Tools/check-v2-bot-coverage-mapping.py
python3 V2/ue/Tools/check-v2-random-exploration-bot.py
python3 V2/ue/Tools/check-v2-bot-harness.py
python3 V2/ue/Tools/check-v2-tests-module.py
python3 V2/ue/Tools/check-v2-ci-workflow.py
python3 -m json.tool V2/ue/Content/V2/QA/RLTrainedExplorationAgent_V2_Contract.json
python3 -m json.tool V2/ue/Build/Horde/v2-buildgraph-job.json
python3 V2/tools/validate-v2-docs.py