# V2 RL-Trained Exploration Agent

`V2RLTrainedExplorationAgent` covers `TODOS.phase-72.72.19.1.9`: it models a
reinforcement learning agent maximizing world state coverage and finding novel
bugs faster than random exploration.

## Runtime Surface

- `FV2RLExplorationAgentConfig` defines training episodes, policy step budget,
  random-baseline comparison, coverage lift threshold, and bug-discovery lead
  threshold.
- `BuildRLExplorationTrainingStates` creates deterministic world-state rewards
  with novelty, coverage, risk, and trained Q-value signals.
- `BuildRLExplorationPolicySteps` converts the trained policy into
  harness-compatible actions using the RL policy model path and observation
  schema.
- `BuildRLExplorationCoverageSamples` records the world positions selected by
  the trained policy for heatmap analysis.
- `RunRLTrainedExplorationAgent` executes the headless harness, analyzes RL
  coverage, runs the random exploration baseline, measures coverage lift over
  random, and records novel bugs found faster than random.
- `FormatRLExplorationAgentReport` emits a compact coverage and bug-discovery
  report for QA dashboards.

## QA Behavior

The RL-trained exploration agent is deterministic and automation-first. It does
not train a live model during the test run; it validates the shipped
reinforcement learning policy surface by replaying trained policy decisions,
measuring world state coverage, comparing against random exploration, and
recording faster novel-bug discovery frames.

Required capabilities: reinforcement learning agent, maximizing world state
coverage, random baseline comparison, finds novel bugs faster than random,
trained policy Q-values, coverage lift over random, and headless harness
execution.

## Validation Commands

```bash
python3 V2/ue/Tools/check-v2-rl-trained-exploration-agent.py
python3 V2/ue/Tools/check-v2-load-test-bot-army.py
python3 V2/ue/Tools/check-v2-soak-test-bot.py
python3 V2/ue/Tools/check-v2-puzzle-solving-bot.py
python3 V2/ue/Tools/check-v2-combat-bot.py
python3 V2/ue/Tools/check-v2-navigation-stress-test-bot.py
python3 V2/ue/Tools/check-v2-bot-coverage-mapping.py
python3 V2/ue/Tools/check-v2-random-exploration-bot.py
python3 V2/ue/Tools/check-v2-bot-harness.py
python3 V2/ue/Tools/check-v2-tests-module.py
python3 V2/ue/Tools/check-v2-ci-workflow.py
python3 -m json.tool V2/ue/Content/V2/QA/RLTrainedExplorationAgent_V2_Contract.json
python3 -m json.tool V2/ue/Build/Horde/v2-buildgraph-job.json
python3 V2/tools/validate-v2-docs.py
```