Eve library

nous-muzero

Other

Revision 1

On this page

MuZero-class planning with learned models (Phase 176.4), PyTorch, CPU-runnable with overfit-verified training.

Components#

  • Networks (networks.py, support.py) — representation h, dynamics g, prediction f with min-max-normalised hidden states and categorical value/reward heads over the MuZero scaling support. (176.4.1.1)
  • Training (muzero.py) — joint unrolled training with categorical value/reward cross-entropy, ½-scaled recurrent gradients, n-step value targets, and a Reanalyser that recomputes value/policy targets with the current network via MCTS. (176.4.1.1)
  • MCTS (mcts.py) — PUCT selection (c1=1.25, c2=19652), Dirichlet root noise (α=0.3), min-max Q normalisation, and batched tree search that stacks the per-simulation network calls into single forward passes. (176.4.1.2)
  • Planner kernel (planner.py) — LookaheadPlanner drives MCTS over any PlanningModel (a learned MuZeroNetwork or a SymbolicModel built from explicit tool/code-action transition/reward callbacks), ready to ship to @iris/reasoning-thinking for tool-use look-ahead. (176.4.1.4)

Not included (external — left unchecked in TODOS)#

The local efficientzero.py implementation adds EfficientZero stopped-target latent consistency and value-prefix heads plus Stochastic MuZero afterstate/chance-node dynamics. 176.4.1.3 remains open because training these models on Atari-100k and the Hathor simulator to the published 1.9× human score needs full-scale accelerated training on Atari frames.

Tests#

text
cd libs/nous/muzero && python3 -m pytest tests/ -q

Verifies the support transform inverse + twohot reconstruction, hidden-state normalisation, MCTS visit concentration on a solvable model, the n-step target closed form, an overfit of a deterministic MDP (all three loss heads drop), and the planner selecting the optimal tool sequence in a decision tree.