# nous-distillation

Policy distillation & kickstarting (Phase 177.9), in PyTorch, CPU-runnable.

## Components

- **Policy distillation** (`policy_distillation.py`, 177.9.1.1) — Rusu et al.
  Temperature-scaled KL distillation (Hinton, `·T²`) of a teacher policy into a
  student over shared rollouts, and joint distillation of several teachers /
  progressive columns into one deployable student (each on its own input
  region).
- **Kickstarting** (`kickstarting.py`, 177.9.1.2) — Schmitt et al. RL objective
  plus a decayed teacher-distillation auxiliary loss (`λ_t` exponential/linear
  schedule with a floor) and a mixture-of-experts teacher (per-state gating over
  teacher policies) for bootstrapping a new robot from fleet teachers.
- **Data-free distillation** (`data_free.py`, 177.9.1.3) — DistBelief-style
  adversarial ZSKD: a generator maximises teacher/student disagreement (with an
  information-entropy diversity term and a known-input-normalisation moment
  prior to stay on-manifold) while the student minimises it, so the student
  matches the teacher on **real held-out inputs it never saw** (0.43 → 0.91
  agreement).
- **Quality scorecard** (`scorecard.py`, 177.9.1.4) — per-task retention
  (student vs teacher accuracy), mean/min retention, and a ≥ 97 %-mean promotion
  gate.

## Tests

`python3 -m pytest tests/ -q` — 15 tests: zero-loss on identical logits +
student matching the teacher after distillation + multi-column joint
distillation; schedule decay (exp/linear/floor) + convex mixture teacher +
kickstart loss/train decay; data-free transfer to real held-out data + KL
trajectory; and scorecard retention/promotion pass & fail.