## Phase 177: Interpretability & Continual-Learning Research Track — Sparse Autoencoders, Circuit Discovery, Progressive Networks, EWC, CLEAR Replay, PackNet & Policy Distillation

Sovereign research track covering the two frontier safety-and-robustness
disciplines that the current ML Sovereignty phases (85–96) do not address in
depth: mechanistic interpretability (Anthropic, DeepMind, OpenAI — sparse
autoencoders, circuit discovery, attribution patching, transcoders) and
continual learning (Hadsell et al. — progressive networks, EWC, online EWC,
synaptic intelligence, MAS, CLEAR, PackNet, policy distillation, Kickstarting).
Powers Kuan Yin safety audits, Galatea fleet-wide lifelong learning, and
alignment research defensibility in front of UK AISI / US AISI.

Tech split: Python (~70%) for research-grade experimentation (PyTorch +
TransformerLens + sae-lens); Rust (~10%) for high-throughput SAE inference
kernels and streaming activation capture; TypeScript (~20%) for dashboards
(interp explorer, continual-learning fleet monitor) and serving integration.

Library layout: `libs/nous/interp-core/`, `libs/nous/sae/`,
`libs/nous/circuits/`, `libs/nous/probes/`, `libs/nous/continual-core/`,
`libs/nous/progressive-nets/`, `libs/nous/replay/`, `libs/nous/distillation/`,
`libs/nous/interp-dashboard/`.

---

### 177.1 Interpretability Foundations — `@nous/interp-core`

#### 177.1.1 Activation Capture & Tooling

- [x] **177.1.1.1** Create `libs/nous/interp-core/` with `project.json`,
      `package.json`, `tsconfig.json`. Define `HookedModel<M>` wrapper over any
      `@nous/llm` or `@nous/vision` model exposing streaming hooks at every
      residual-stream layer, MLP pre/post, attention Q/K/V/O, and logits —
      TransformerLens-compatible semantics
- [x] **177.1.1.2** Build `ActivationCapture` recorder writing Zarr-chunked
      activation tensors keyed by
      `(model_rev, dataset, sample_id, layer,     point)` with per-sample
      metadata and deterministic replay
- [x] **177.1.1.3** Build `ActivationStats` online estimator (mean, cov,
      diagonal Fisher, percentile-100 activation magnitudes) used by SAE
      initialisation and Fisher-based EWC (177.7)
- [x] **177.1.1.4** Ship a cross-model adapter covering Llama-4 MoE, Qwen3
      dense + MoE, Gemma-4, Mixtral / DeepSeek V3 MoE routers, and Nous's own
      fine-tunes — guarantees every Oshun-served model is mechanically
      inspectable out of the box

#### 177.1.2 Attribution Primitives

- [x] **177.1.2.1** Implement activation patching (causal intervention: replace
      activation at (layer, point, token) from donor → recipient, measure logit
      shift) with batched single-backward implementation
- [x] **177.1.2.2** Implement Integrated Gradients, SmoothGrad, and LIME
      attribution backends for feature-attribution dashboards and
      safety-relevant model cards
- [x] **177.1.2.3** Implement attribution-patching (gradient-based approximation
      to activation patching) enabling per-token per-layer attribution at
      whole-dataset scale with O(1) forward/backward per prompt

---

### 177.2 Sparse Autoencoders — `@nous/sae`

#### 177.2.1 SAE Training

- [x] **177.2.1.1** Implement JumpReLU SAE (DeepMind 2024) with per-feature
      threshold + straight-through estimator — primary SAE family, target L0 ≈
      60 and reconstruction MSE parity with published Gemma Scope
- [x] **177.2.1.2** Implement TopK SAE (OpenAI 2024) with aux_k dead-feature
      revival loss and 4× expansion factor — complementary family for controlled
      comparisons
- [x] **177.2.1.3** Implement GatedSAE baseline for architecture ablations
- [x] **177.2.1.4** Build SAE training pipeline — stream activations from
      177.1.1.2, shuffle via reservoir sampler, batch 4096, Adam β₂=0.999,
      learning-rate warmup, λ annealing, feature-resampling on dead features
      every 25k steps, EMA checkpointing every 10k
- [ ] **177.2.1.5** Train a full SAE suite on the Gemma-4-9B / Qwen3-8B residual
      stream at every layer (expansion x16/x32/x64) and MLP-post, publishing
      feature dashboards. Needs an owner decision on a compute budget — this is
      large-scale SAE/interpretability training or a circuit investigation over
      frontier checkpoints, not a coding task (stated 2026-09-18).
      `blocked:governance`
- [ ] **177.2.1.6** Train SAEs on the fine-tuned `@iris/code` and
      `@kuanyin/safety` heads and publish per-domain feature dashboards. Needs
      an owner decision on a compute budget — this is large-scale
      SAE/interpretability training or a circuit investigation over frontier
      checkpoints, not a coding task (stated 2026-09-18). It also needs those
      fine-tuned checkpoints, which are themselves frontier-trained.
      `blocked:governance`

#### 177.2.2 Feature Analysis

- [x] **177.2.2.1** Build autointerp pipeline (Bills et al. style) — for each
      SAE feature, sample top-activating contexts, ask a frontier LLM for a
      natural-language label, score label fidelity via held-out-example
      prediction — run at publish-scale over every SAE
- [x] **177.2.2.2** Build feature-splitting detector tracking how features
      divide under scale-ups (×8 → ×32 → ×64) to identify canonical vs
      derivative features
- [x] **177.2.2.3** Build feature-ablation steering — zero or scale specific SAE
      features at inference, measure downstream behavioural shift (refusal rate,
      sentiment, tool-use style) on eval suites — expose as `@nous/interp-core`
      steering API for Kuan Yin safety researchers
- [x] **177.2.2.4** Build concept-dictionary export — feature → label → safety
      category (toxicity, deception, CSAM, self-harm, jailbreak-adjacent) used
      by Kuan Yin classifiers as auxiliary supervision (Phase 85.5 integration)

---

### 177.3 Circuits & Feature Graphs — `@nous/circuits`

#### 177.3.1 Circuit Discovery

- [x] **177.3.1.1** Implement ACDC (Automatic Circuit DisCovery) — iterative
      edge ablation over a computational graph constructed from SAE features +
      attention edges, converging to minimal circuits explaining a task
- [x] **177.3.1.2** Implement EAP-IG (Edge Attribution Patching with Integrated
      Gradients) as the scalable alternative to ACDC for large models and large
      datasets
- [x] **177.3.1.3** Implement transcoder training (per-MLP, predicting MLP
      output from residual) for clean feature-graph edges between residual and
      MLP modules — Anthropic transcoder pattern
- [x] **177.3.1.4** Build feature-circuit visualiser — interactive Reagent
      dashboard showing feature nodes (SAE), attention edges, MLP edges,
      per-prompt activation traces, and downstream token effects

#### 177.3.2 Targeted Interpretability Campaigns

- [ ] **177.3.2.1** Run a refusal-circuit investigation on the safety-aligned
      `@nous/llm` checkpoint — identify the causal features and attention heads.
      Needs an owner decision on a compute budget — this is large-scale
      SAE/interpretability training or a circuit investigation over frontier
      checkpoints, not a coding task (stated 2026-09-18). The investigation
      needs a trained `@nous/llm` checkpoint to run against.
      `blocked:governance`
- [ ] **177.3.2.2** Run a factual-recall circuit investigation (Ghanaian
      history, Yorùbá deities, African philosophy) to characterise failure
      modes. Needs an owner decision on a compute budget — this is large-scale
      SAE/interpretability training or a circuit investigation over frontier
      checkpoints, not a coding task (stated 2026-09-18). `blocked:governance`
- [ ] **177.3.2.3** Run a code-generation circuit investigation on the
      `@iris/code` model to find the features behind common bug-prone patterns.
      Needs an owner decision on a compute budget — this is large-scale
      SAE/interpretability training or a circuit investigation over frontier
      checkpoints, not a coding task (stated 2026-09-18). `blocked:governance`
- [ ] **177.3.2.4** Run a deception / sycophancy circuit investigation on pre-
      and post-RLHF checkpoints to quantify steering targets. Needs an owner
      decision on a compute budget — this is large-scale SAE/interpretability
      training or a circuit investigation over frontier checkpoints, not a
      coding task (stated 2026-09-18). It needs paired pre/post-RLHF
      checkpoints. `blocked:governance`

---

### 177.4 Probing & Representation Analysis — `@nous/probes`

#### 177.4.1 Classical Probing

- [x] **177.4.1.1** Implement linear probing harness (L2-regularised logistic
      regression with bootstrap CIs) over activation tensors, supporting
      per-layer and per-position sweeps
- [x] **177.4.1.2** Implement Representational Similarity Analysis (RSA) and
      Centered Kernel Alignment (CKA) between checkpoints, model families, and
      between model and brain activations (NSD, Algonauts datasets) for
      neuroscience-bridge research
- [x] **177.4.1.3** Implement concept activation vectors (TCAV) + causal concept
      effect, surfaced in the model-card for every published Nous checkpoint
- [x] **177.4.1.4** Build emergent-capability probes — monitor downstream probes
      (truthfulness, multilingual, arithmetic) across training checkpoints so
      scaling-law experiments (Phase 89.8) can track capability emergence, not
      just loss

---

### 177.5 Continual-Learning Foundations — `@nous/continual-core`

#### 177.5.1 Task Abstractions

- [x] **177.5.1.1** Create `libs/nous/continual-core/` with the
      `ContinualTaskStream<T>` abstraction — an ordered stream of tasks emitting
      `(task_id, dataset, eval_fn)` supporting both disjoint task-incremental
      and blurry class-incremental regimes
- [x] **177.5.1.2** Define `ForgettingMetric` — backward transfer (BWT), forward
      transfer (FWT), average accuracy (ACC), and per-task learning curve —
      computed automatically after every task in the stream
- [x] **177.5.1.3** Build a standard CL benchmark suite (Permuted MNIST, Split
      CIFAR-100, CLEAR-100, Continual World 20, LORA-CL, and an Oshun synthetic
      stream combining Iris intent + Isis style + Kuan Yin harms)
- [x] **177.5.1.4** Build a per-domain Oshun continual stream — Iris
      conversations by week, Kuan Yin emerging harms by month, Galatea teleop by
      fleet cohort, Veritas news by cycle — wired to Phase 85 flywheel via
      retrograde-stream sampling

---

### 177.6 Progressive Networks & Parameter Expansion

#### 177.6.1 Progressive Networks (Rusu & Hadsell 2016)

- [x] **177.6.1.1** Implement Progressive Network column architecture — for each
      new task, instantiate a new column, freeze all prior columns, add lateral
      adapters from prior columns' activations at each layer; forward pass
      concatenates lateral contributions; backward flows only through the new
      column
- [x] **177.6.1.2** Integrate progressive columns with Galatea VLA (Phase
      176.7.1) — each new deployment fleet / new task class gets its own column,
      preserving prior proficiency exactly while reusing lateral features — no
      forgetting by construction
- [x] **177.6.1.3** Build column-pruning pass that distils rarely-used
      progressive columns into a shared base (177.9) when parameter budget
      tightens, preserving measured skill retention

#### 177.6.2 Dynamically Expandable Networks & Supermasks

- [x] **177.6.2.1** Implement DEN (Dynamically Expandable Networks) — grow the
      hidden width on demand guided by loss-threshold heuristics, with
      task-selective retraining
- [x] **177.6.2.2** Implement PackNet — iterative magnitude pruning and
      task-mask packing, allowing many tasks to share a single dense backbone
      with per-task bitmasks
- [x] **177.6.2.3** Implement supermask subnetwork discovery (Zhou et al.) for
      inference-time routing between continual tasks without weight mutation

---

### 177.7 Regularisation-Based Continual Learning

#### 177.7.1 Fisher / Path-Integral / Importance-Based

- [x] **177.7.1.1** Implement Elastic Weight Consolidation (EWC) — per-task
      diagonal Fisher estimate F_θ, quadratic penalty Σ λ_i (θ_i − θ\*\_i)² F_i
      on subsequent task losses; validated on Permuted MNIST (target BWT ≥ −2%)
      and Atari-continual
- [x] **177.7.1.2** Implement Online EWC with running Fisher accumulation,
      avoiding per-task Fisher storage, used in long streams (177.5.1.4)
- [x] **177.7.1.3** Implement Synaptic Intelligence (Zenke et al.) — path
      integral of parameter movement weighted by loss contribution — cheaper
      than Fisher, validated against EWC on Oshun synthetic stream
- [x] **177.7.1.4** Implement Memory-Aware Synapses (MAS) — importance from
      output-gradient magnitude, supports unsupervised continual streams
- [x] **177.7.1.5** Build automatic λ-tuning — Bayesian optimisation over
      held-out BWT/ACC frontier to avoid hand-tuned EWC coefficients per task

---

### 177.8 Replay-Based Continual Learning — `@nous/replay`

#### 177.8.1 Experience & Generative Replay

- [x] **177.8.1.1** Implement CLEAR (Continual Learning with Experience And
      Replay) — off-policy behavioural cloning + on-policy RL mixed replay with
      V-trace corrections, per Rolnick et al. (Hadsell group)
- [x] **177.8.1.2** Implement Gradient Episodic Memory (GEM) and A-GEM — project
      gradients to avoid increasing prior-task loss (GEM quadprog, A-GEM
      projection); evaluate on Permuted MNIST + Split CIFAR-100
- [x] **177.8.1.3** Implement Dark Experience Replay (DER++) — logit
      distillation across the episodic memory, SOTA baseline for many
      class-incremental benchmarks
- [x] **177.8.1.4** Implement Generative Replay with a small diffusion / VQ-GAN
      model producing prior-task pseudo-samples when storage of real replay
      violates consent or retention policy (Phase 85.15)
- [x] **177.8.1.5** Build privacy-preserving replay — encrypted, k-anonymised,
      differentially-private replay buffer integrated with Phase 85.15.1.1

---

### 177.9 Policy Distillation & Kickstarting — `@nous/distillation`

#### 177.9.1 Distillation

- [x] **177.9.1.1** Implement policy distillation (Rusu et al.) — teacher policy
      → student policy KL distillation over shared env rollouts, compressing
      multiple progressive columns into a single deployable model
- [x] **177.9.1.2** Implement Kickstarting (Schmitt et al.) — mixture-of-
      experts teacher auxiliary loss added to student RL objective, smoothly
      decayed — used to bootstrap new Galatea robots from fleet teachers
- [x] **177.9.1.3** Implement DistBelief-style data-free distillation producing
      a compact `@nous/llm-edge` continual student consuming teacher logits from
      the progressive stack
- [x] **177.9.1.4** Build distillation-quality scorecard — per-task retention
      after distillation vs pre-distillation progressive stack, promotion gated
      at ≥ 97% mean retention

---

### 177.10 Galatea Fleet & Safety Integration

#### 177.10.1 Fleet-Wide Continual Learning

- [x] **177.10.1.1** Wire Phase 85.20.1.3 federated-learning rounds through
      Phase 177.8.1.5 privacy-preserving replay and Phase 177.7.1.1 EWC so that
      fleet rollouts strictly monotonically improve (BWT ≥ 0) on previously
      certified skills
- [x] **177.10.1.2** Build fleet dashboard (`@nous/interp-dashboard`) showing
      per-robot per-skill retention, forgetting alerts, and roll-back
      recommendations — integrated with Phase 92 MLOps governance
- [x] **177.10.1.3** Build continual-evaluation watchdog — every fleet continual
      update re-runs a regression suite on 177.5.1.4 streams; any BWT < −1% on
      safety-critical skills freezes the update

#### 177.10.2 Interpretability-Driven Safety

- [ ] **177.10.2.1** Publish an "SAE safety report" per major Nous checkpoint
      (feature inventory, refusal circuit, steering vulnerabilities). Needs an
      owner decision on a compute budget — this is large-scale
      SAE/interpretability training or a circuit investigation over frontier
      checkpoints, not a coding task (stated 2026-09-18). It rolls up the SAE
      runs of 177.2/177.3, which are themselves budget-gated.
      `blocked:governance`
- [x] **177.10.2.2** Build interpretability-integrated eval (Phase 91) — in
      addition to behavioural scores, publish circuit-stability and
      feature-drift metrics per checkpoint so regressions in internal structure,
      not just outputs, gate promotion
- [x] **177.10.2.3** Build red-team interp tooling — given an elicited failure,
      automatically surface top contributing SAE features and circuits for
      root-cause analysis within minutes, not weeks
- [x] **177.10.2.4** Wire SAE feature activations as side-channel signals into
      Kuan Yin precognition (Phase 85.5.2.4) — early-warning features (deception
      / coercion / CSAM-adjacent) fire detectors before behavioural
      manifestation

---
