Self-improvement & self-play reasoning for the Phase-178 autonomous-research stack (TODOS/phase-178 §178.3). Every learner here trains on self-generated, executor-verified signal — nothing fabricates a reward.
Components#
| Module | Summary |
|---|---|
azr.py |
Absolute Zero Reasoner: propose/solve deduction / abduction / induction tasks over an executable integer DSL, verified by execution; learnability reward (favour harder-but-solvable), TRR++ advantages (baselined per role × task-type), and a self-improving synthesis solver whose op-priors sharpen from its own verified solutions. Trained on zero external data, held-out solve-rate rises (fresh 0.14 → trained 0.22 in the test). (§178.3.1.1) |
star.py |
STaR / V-STaR: rationale (skill-choice) bootstrapping with rationalisation for unsolved problems; the V-STaR verifier learns from correct and incorrect attempts and can select a correct skill the policy never sampled. Accuracy climbs from chance to ≥0.9. (§178.3.1.2) |
self_rewarding.py |
Self-Rewarding & Meta-Rewarding: LLM-as-judge → DPO preference pairs → policy improvement; a meta-judge recalibrates the judge against known anchors, provably reducing judgment error under reward drift. (§178.3.1.3) |
spiral.py |
SPIRAL zero-sum self-play RL on a Nim-like subtraction game with a closed-form optimum (Sprague–Grundy) for verification; tabular Monte-Carlo self-play reaches ≥0.99 win-rate vs random. Optional search_policy hook plugs in a @nous/tree-search-reasoning kernel. (§178.3.1.4) |
voyager.py |
Voyager skill library: embedding-indexed executable skills, cosine retrieval, pipeline composition, test-gated iterative addition, and a Phase-47 memory sink. (§178.3.2.1) |
curriculum.py |
Automatic curriculum generator: per-skill mastery, ZPD gap analysis, difficulty escalation as skills saturate, and a quality-gated retraining hook. (§178.3.2.2) |
evoagentx.py |
EvoAgentX agent-workflow evolver: mutate role/tool/prompt/kernel graphs, keep the Pareto front over quality × cost × latency. (§178.3.2.3) |
Scope of local verification#
The substrates are deliberately controllable so learning is provable, not asserted: AZR's induction DSL, STaR's skill-selection tasks, SPIRAL's game-theoretically-solved game, self-rewarding's known-target quality. Swap an LM policy + Python sandbox behind the same interfaces to run the papers' math/code evals and the reasoning-transfer claims (which need the LM and are out of local scope).
Testing#
bash
python3 -m pytest tests/ -q # 24 tests
python3 -m ruff check .