Galatea fleet & interpretability-driven safety integration (Phase 177.10), in PyTorch, CPU-runnable. Composes the continual-learning and interpretability libs into fleet-level safety gates.
Components#
- Fleet continual update (
fleet_update.py, 177.10.1.1) — a monotonic- improvement harness that runs a federated round (Phase 85.20.1.3 seam) through privacy replay + EWC, then accepts the rollout only if backward transferBWT ≥ 0on certified skills; otherwise it rolls back to the last certified checkpoint. Certified baselines advance on accept. - Continual-evaluation watchdog (
watchdog.py, 177.10.1.3) — re-runs the regression suite after each update and freezes it if any safety-critical skill regresses beyond tolerance (BWT < −1 %; a looser bound for non-safety skills), naming the violations. - Interpretability-integrated eval (
interp_eval.py, 177.10.2.2) — adds circuit-stability (edge-set Jaccard) and SAE feature-drift (unmatched-cosine) metrics that gate promotion, so internal-structure regressions block a release even when behavioural scores pass. - Red-team interp tooling (
red_team.py, 177.10.2.3) — root-causes an elicited failure to its top contributing SAE features (activation · decoder·failure_dir) and the circuit edges they participate in, in seconds.
Tests#
python3 -m pytest tests/ -q — 15 tests: accept/rollback + advancing baseline,
safety-critical freeze vs benign pass + per-skill BWT, Jaccard stability +
feature-drift (zero on identical/permuted, bounded on rotated) + structure-gated
promotion, and failure root-cause ranking + edge mapping.
Not locally actionable#
177.10.2.1 (per-checkpoint SAE safety report) needs real trained SAEs on real
checkpoints; 177.10.2.4 (wiring SAE features into Kuan Yin precognition, Phase
85.5.2.4) needs that phase — both left unchecked. The fleet dashboard
(177.10.1.2) is delivered separately in @nous/interp-dashboard.