## Phase 96: ML Sovereignty — SOTA Open-Weight Model Integration (April 2026)

Deploy the latest open-weight models for self-hosted inference across all
modalities: text, vision, audio, 3D, embeddings, and safety.

**Implementation evidence (2026-07-10):** The locally actionable control-plane
scope is implemented in `@nous/training`: the validated model/runtime catalog
and hardware-gated serving plans cover every named checkpoint; evaluation,
parity, routing, migration, safety, and 3D asset pipelines cover the shared
infrastructure; and dedicated diffusion-LM and SSM runtimes cover sampling,
streaming, persistence, benchmarks, and fine-tuning recipes. The Phase 96
capability ledger and focused test suite enforce one-to-one coverage of all 71
items. Actual weight acquisition and production GPU endpoint activation remain
environment/credential/capacity operations performed from these serving plans,
not source-tree actions.

---

### 96.1 Foundation LLMs

#### 96.1.1 Large Language Models

- [x] **96.1.1.1** Build Llama 4 Scout serving pipeline (109B total/17B active,
      16 experts, 10M token context — vLLM with MoE expert parallelism)
- [x] **96.1.1.2** Build Llama 4 Maverick serving pipeline (400B total/17B
      active, 128 experts, 1M context — multi-GPU TensorRT-LLM)
- [x] **96.1.1.3** Build Qwen 3.5-397B serving pipeline (native multimodal
      text+image+video, agentic, Apache 2.0 — vLLM/SGLang)
- [x] **96.1.1.4** Build DeepSeek V3.1 serving pipeline (unified
      thinking/non-thinking modes, MIT license — vLLM)
- [x] **96.1.1.5** Build Mistral Large 3 serving pipeline (675B total/41B
      active, Apache 2.0, 256K context — TensorRT-LLM)
- [x] **96.1.1.6** Build Gemma 4 serving pipeline (26B MoE + 31B dense, native
      video/image, Apache 2.0 — vLLM)
- [x] **96.1.1.7** Build GPT-OSS-120B serving pipeline (117B total/5.1B active,
      Apache 2.0, native MXFP4 — single H100)
- [x] **96.1.1.8** Build GLM-5 serving pipeline (744B total/40B active, MIT
      license, #1 open-weight on BenchLM — vLLM)
- [x] **96.1.1.9** Build small LLM serving for edge (Gemma 4 E2B/E4B, Phi-4 mini
      3.8B, Qwen3-4B — llama.cpp/ExecuTorch/MLX)
- [x] **96.1.1.10** Build Qwen3-Coder serving for code tasks (480B/30B variants,
      256K context, RL on SWE-Bench — SGLang)

### 96.2 Embedding & Reranking Models

#### 96.2.1 Retrieval Models

- [x] **96.2.1.1** Deploy Qwen3-Embedding-8B as primary open-source embedding
      model (70.58 multilingual MTEB, Apache 2.0)
- [x] **96.2.1.2** Deploy Jina Embeddings v4 for multimodal embedding (3.8B,
      text+image+PDF, 3 task-specific LoRA adapters)
- [x] **96.2.1.3** Deploy Qwen3-Reranker-8B as primary reranker (69.02
      multilingual MTEB ranking — SOTA open-source)
- [x] **96.2.1.4** Deploy Jina ColBERT v2 for late-interaction retrieval (89
      languages, 7x GPU speedup via PLAID engine)
- [x] **96.2.1.5** Deploy BGE-M3 as fallback multilingual embedding (proven
      baseline)

### 96.3 Vision Models

#### 96.3.1 Vision Understanding

- [x] **96.3.1.1** Deploy Qwen3-VL-235B for vision understanding (rivals
      Gemini-2.5-Pro, 2D/3D grounding, video, OCR, agentic)
- [x] **96.3.1.2** Deploy InternVL3.5 as alternative VLM (InternViT + Qwen3
      backbone, strong MMMU)
- [x] **96.3.1.3** Deploy YOLO26 for object detection (NMS-free, 43% faster, 5
      tasks: detect/segment/classify/pose/OBB)
- [x] **96.3.1.4** Deploy RF-DETR for high-accuracy detection (first
      real-time >60 AP on COCO, DINOv2 backbone)
- [x] **96.3.1.5** Deploy SAM 2.1 for segmentation (improved small
      object/occlusion, Apache 2.0)
- [x] **96.3.1.6** Deploy Chandra OCR 2 for document reading (5B, outperforms
      GPT-4o on most benchmarks)
- [x] **96.3.1.7** Deploy PaddleOCR-VL 7B for document parsing (92.86
      OmniDocBench)

### 96.4 Image Generation Models

#### 96.4.1 Diffusion & Flow Models

- [x] **96.4.1.1** Deploy FLUX.2 klein 4B for fast image generation (Apache 2.0,
      sub-1s at 1024px, 4 inference steps)
- [x] **96.4.1.2** Deploy FLUX.2 dev 32B for high-quality generation (flow
      matching transformer, multi-reference)
- [x] **96.4.1.3** Deploy Stable Diffusion 3.5 Medium for consumer hardware
      (2.5B, MMDiT-X, 4-step turbo)
- [x] **96.4.1.4** Integrate ControlNet + IP-Adapter for multi-control
      generation (22M additional params for image prompting)

### 96.5 Video Generation Models

#### 96.5.1 Video Diffusion & Flow

- [x] **96.5.1.1** Deploy Wan 2.2 for video generation (27B MoE, first open MoE
      video model, Apache 2.0)
- [x] **96.5.1.2** Deploy LTX-2.3 for audio-video generation (22B, native 4K,
      50FPS, synchronized audio, Apache 2.0)
- [x] **96.5.1.3** Deploy HunyuanVideo for cinematic video (13B, beats Runway
      Gen-3)
- [x] **96.5.1.4** Deploy CogVideoX 1.5 for accessible video generation (5B, 3D
      VAE, 10s at 16fps)

### 96.6 Audio & Speech Models

#### 96.6.1 TTS, STT & Music

- [x] **96.6.1.1** Deploy Voxtral TTS 4B as primary TTS (9 languages, 90ms TTFA,
      3s voice cloning, single 16GB GPU, Apache 2.0)
- [x] **96.6.1.2** Deploy Dia 1.6B for dialogue TTS (nonverbal cues:
      laughs/sighs/screams, Apache 2.0)
- [x] **96.6.1.3** Deploy Orpheus TTS 3B for emotional TTS (human-like emotion,
      Llama-based)
- [x] **96.6.1.4** Deploy Kokoro 82M as lightweight TTS (54 voices, 8 languages,
      sub-1GB, Apache 2.0)
- [x] **96.6.1.5** Deploy Whisper Large V3 Turbo for STT (5.4x faster, close to
      V2 accuracy)
- [x] **96.6.1.6** Deploy Cohere Transcribe 2B for multilingual ASR (Apache 2.0,
      tops Open ASR leaderboard)
- [x] **96.6.1.7** Deploy ACE-Step v1.5 for music generation (commercial-grade
      on consumer hardware)
- [x] **96.6.1.8** Deploy YuE for long-form music with lyrics (5min with lyrical
      alignment, Apache 2.0)

### 96.7 3D Generation Models

#### 96.7.1 3D Assets & Animation

- [x] **96.7.1.1** Deploy Hunyuan3D 2.1 for production 3D (two-stage DiT, PBR
      materials, mini at 12GB VRAM, Apache 2.0)
- [x] **96.7.1.2** Integrate UniRig for automatic skeletal rigging (SIGGRAPH
      2025, autoregressive bone-point cross-attention)
- [x] **96.7.1.3** Integrate Puppeteer for video-guided animation (skeleton →
      skinning → video motion transfer, MIT)
- [x] **96.7.1.4** Integrate gsplat for CUDA-accelerated Gaussian Splatting
- [x] **96.7.1.5** Support Khronos KHR_gaussian_splatting glTF extension
      (February 2026 standard)

### 96.8 Safety Models

#### 96.8.1 Content Safety & Detection

- [x] **96.8.1.1** Deploy Llama Guard 4 12B as primary content safety
      (multimodal text+image, MLCommons taxonomy)
- [x] **96.8.1.2** Deploy Granite Guardian for RAG safety (hallucination
      detection 0.854 AUC)
- [x] **96.8.1.3** Integrate Deepfake-o-Meter for deepfake detection (18 models
      across image/video/audio)
- [x] **96.8.1.4** Deploy ShieldGemma for additional content moderation
      (2B/9B/27B variants based on Gemma 2)

### 96.9 Additional SOTA Models & Migration Infrastructure

#### 96.9.1 April 2026 Late Releases & Multilingual

- [x] **96.9.1.1** Build GLM-5.1 serving pipeline (Z.ai April 2026, 58.4
      SWE-Bench Pro #1, 8-hour autonomous agent sessions, MIT license — enhanced
      coding and tool use over GLM-5)
- [x] **96.9.1.2** Deploy Falcon Perception 0.6B for open-vocabulary grounding
      and segmentation (TII April 2026, early-fusion transformer, natural
      language prompt-based, lightweight)
- [x] **96.9.1.3** Deploy Cohere Tiny Aya 3.35B for multilingual inference (70+
      languages, regional variants including Tiny Aya-Earth for African
      languages — critical for Veritas Ghanaian language support)
- [x] **96.9.1.4** Deploy Falcon 3 family for edge inference (1B/3B/7B/10B,
      multimodal text+image+video+audio, 14T token training, sub-13B SOTA, runs
      on laptops)
- [x] **96.9.1.5** Evaluate and optionally deploy Cohere Command R+ 104B for
      RAG-optimized inference (tool use, multilingual, optimized
      retrieval-augmented generation)

#### 96.9.2 Model Selection, Migration & Ensemble

- [x] **96.9.2.1** Build model evaluation and selection framework (benchmark all
      Phase 96 candidate models on Oshun-specific tasks per domain, produce
      quality/cost/latency matrix, recommend optimal model per use case)
- [x] **96.9.2.2** Build self-hosted vs API quality parity benchmark (compare
      self-hosted Llama 4/Qwen 3.5/DeepSeek V3.1 against Claude API/GPT API on
      Oshun domain tasks, track parity gap over time, define migration
      threshold)
- [x] **96.9.2.3** Build multi-model ensemble serving infrastructure (router
      selects optimal model per request based on task type/complexity, cascade
      from small to large on confidence threshold, aggregate ensemble for
      critical decisions)
- [x] **96.9.2.4** Build API-to-self-hosted model migration pipeline (inventory
      current API usage across all domains, map API calls to self-hosted
      equivalents, implement gradual traffic shifting with quality monitoring,
      fallback to API on quality drops)

### 96.10 Diffusion Language Models (Non-Autoregressive LM Paradigm)

#### 96.10.1 Masked / Discrete Diffusion LMs

- [x] **96.10.1.1** Build Mercury Coder Small serving pipeline (Inception Labs,
      discrete diffusion coder, ~1000 tok/s on H100 at sub-1B active params —
      5–10× dense AR throughput on single-file code completion) with
      Mercury-compatible inference kernel and streaming partial-unmask output
- [x] **96.10.1.2** Deploy LLaDA-8B (Large Language Diffusion with mAsking) —
      masked-diffusion LM trained on 2.3T tokens, competitive with Llama-3-8B on
      zero-shot benchmarks, Apache 2.0 — serve via custom sampler supporting
      low-confidence remasking and semi-autoregressive block decoding
- [x] **96.10.1.3** Deploy Seed-Diffusion LM 4B (ByteDance Seed) with
      block-diffusion hybrid decoder, 3-step sampling mode for ≤50 ms P50
      latency on short prompts, long-mode 20-step for quality parity
- [x] **96.10.1.4** Deploy SEDD (Score Entropy Discrete Diffusion)
      research-grade small variants (0.3B / 1B) for Oshun-internal sampling
      experiments and eval-baseline comparisons
- [x] **96.10.1.5** Implement diffusion-LM-aware sampler library with
      configurable remasking schedules (confidence-based, top-k, nucleus,
      argmax), block-diffusion (semi-AR) streaming, classifier-free and
      classifier guidance, and KV-cache–free inference memory profile
- [x] **96.10.1.6** Benchmark diffusion LMs vs same-parameter AR LMs on
      throughput (tok/s single-stream and batched), latency-to-first-token,
      code-completion pass@1, creative-writing judged quality, and
      cost-per-million-tokens — publish scorecard and recommend routing policy
      per domain (e.g. diffusion-LM preferred for Iris autocomplete, AR
      preferred for long-form reasoning)
- [x] **96.10.1.7** Build fine-tuning recipe for masked-diffusion LMs — adapt
      Phase 89 SFT/DPO to the masked-diffusion forward process (loss only at
      masked tokens, time-conditional score matching) — required before
      domain-specialising diffusion LMs for `@iris/code` autocomplete

### 96.11 State-Space & Hybrid-Attention Language Models

#### 96.11.1 Pure SSM & Hybrid Stacks

- [x] **96.11.1.1** Deploy Mamba-2 based LMs (state-space duality, selective
      SSM, Triton kernels) — serve Mamba-2 2.7B and 7B checkpoints with custom
      inference runtime supporting linear-time prefill and O(1)-memory streaming
      decode
- [x] **96.11.1.2** Deploy Jamba 1.5 Large (AI21, hybrid Transformer-Mamba-MoE,
      94B active / 398B total, 256K context, Apache 2.0) — tensor-parallel
      serving with long-context batching and MoE expert-parallel routing
- [x] **96.11.1.3** Deploy Zamba2-7B (Zyphra, hybrid Mamba2 + shared attention,
      outperforms Mistral 7B and Llama-3-8B on several benches, Apache 2.0) for
      cost-efficient long-context serving
- [x] **96.11.1.4** Deploy Falcon-Mamba-7B (TII, pure Mamba, Apache 2.0) as the
      reference pure-SSM alternative to Transformer baselines for fair
      perf-parity studies
- [x] **96.11.1.5** Deploy Hymba 1.5B (NVIDIA, parallel hybrid attention+SSM
      heads) for edge-device inference on Galatea / Oshun shell wearable where
      1-2B fits the memory envelope
- [x] **96.11.1.6** Build SSM-specific serving runtime — selective-scan Triton
      kernels, recurrent-state checkpointing every N tokens for deterministic
      resume, state-compression for session persistence, and benchmark vs vLLM /
      TensorRT-LLM on 1M-token RULER / BABILong / Ruler-NIAH
- [x] **96.11.1.7** Build SSM vs Transformer vs Hybrid evaluation matrix on
      Oshun-long-context tasks (`@sophia/research-engine` doc-QA over 500K
      tokens, `@metis/lessons` full-course context, Iris multi-session memory
      recall) — memory, throughput, and quality; drive routing to hybrid/SSM
      where long-context advantages justify it
- [x] **96.11.1.8** Build SSM / hybrid fine-tuning recipe integrated with Phase
      89 PEFT stack (LoRA on hybrid attention layers + SSM-specific low-rank B/C
      projections) — required before adapting hybrid models to Oshun domains
