Eve library

phase-96.md

Tracker

Revision 1

On this page

Historical task source. The Eve task board holds current task state.

Phase 96: ML Sovereignty — SOTA Open-Weight Model Integration (April 2026)#

Deploy the latest open-weight models for self-hosted inference across all modalities: text, vision, audio, 3D, embeddings, and safety.

Implementation evidence (2026-07-10): The locally actionable control-plane scope is implemented in @nous/training: the validated model/runtime catalog and hardware-gated serving plans cover every named checkpoint; evaluation, parity, routing, migration, safety, and 3D asset pipelines cover the shared infrastructure; and dedicated diffusion-LM and SSM runtimes cover sampling, streaming, persistence, benchmarks, and fine-tuning recipes. The Phase 96 capability ledger and focused test suite enforce one-to-one coverage of all 71 items. Actual weight acquisition and production GPU endpoint activation remain environment/credential/capacity operations performed from these serving plans, not source-tree actions.


96.1 Foundation LLMs#

96.1.1 Large Language Models#

  • 96.1.1.1 Build Llama 4 Scout serving pipeline (109B total/17B active, 16 experts, 10M token context — vLLM with MoE expert parallelism)
  • 96.1.1.2 Build Llama 4 Maverick serving pipeline (400B total/17B active, 128 experts, 1M context — multi-GPU TensorRT-LLM)
  • 96.1.1.3 Build Qwen 3.5-397B serving pipeline (native multimodal text+image+video, agentic, Apache 2.0 — vLLM/SGLang)
  • 96.1.1.4 Build DeepSeek V3.1 serving pipeline (unified thinking/non-thinking modes, MIT license — vLLM)
  • 96.1.1.5 Build Mistral Large 3 serving pipeline (675B total/41B active, Apache 2.0, 256K context — TensorRT-LLM)
  • 96.1.1.6 Build Gemma 4 serving pipeline (26B MoE + 31B dense, native video/image, Apache 2.0 — vLLM)
  • 96.1.1.7 Build GPT-OSS-120B serving pipeline (117B total/5.1B active, Apache 2.0, native MXFP4 — single H100)
  • 96.1.1.8 Build GLM-5 serving pipeline (744B total/40B active, MIT license, #1 open-weight on BenchLM — vLLM)
  • 96.1.1.9 Build small LLM serving for edge (Gemma 4 E2B/E4B, Phi-4 mini 3.8B, Qwen3-4B — llama.cpp/ExecuTorch/MLX)
  • 96.1.1.10 Build Qwen3-Coder serving for code tasks (480B/30B variants, 256K context, RL on SWE-Bench — SGLang)

96.2 Embedding & Reranking Models#

96.2.1 Retrieval Models#

  • 96.2.1.1 Deploy Qwen3-Embedding-8B as primary open-source embedding model (70.58 multilingual MTEB, Apache 2.0)
  • 96.2.1.2 Deploy Jina Embeddings v4 for multimodal embedding (3.8B, text+image+PDF, 3 task-specific LoRA adapters)
  • 96.2.1.3 Deploy Qwen3-Reranker-8B as primary reranker (69.02 multilingual MTEB ranking — SOTA open-source)
  • 96.2.1.4 Deploy Jina ColBERT v2 for late-interaction retrieval (89 languages, 7x GPU speedup via PLAID engine)
  • 96.2.1.5 Deploy BGE-M3 as fallback multilingual embedding (proven baseline)

96.3 Vision Models#

96.3.1 Vision Understanding#

  • 96.3.1.1 Deploy Qwen3-VL-235B for vision understanding (rivals Gemini-2.5-Pro, 2D/3D grounding, video, OCR, agentic)
  • 96.3.1.2 Deploy InternVL3.5 as alternative VLM (InternViT + Qwen3 backbone, strong MMMU)
  • 96.3.1.3 Deploy YOLO26 for object detection (NMS-free, 43% faster, 5 tasks: detect/segment/classify/pose/OBB)
  • 96.3.1.4 Deploy RF-DETR for high-accuracy detection (first real-time >60 AP on COCO, DINOv2 backbone)
  • 96.3.1.5 Deploy SAM 2.1 for segmentation (improved small object/occlusion, Apache 2.0)
  • 96.3.1.6 Deploy Chandra OCR 2 for document reading (5B, outperforms GPT-4o on most benchmarks)
  • 96.3.1.7 Deploy PaddleOCR-VL 7B for document parsing (92.86 OmniDocBench)

96.4 Image Generation Models#

96.4.1 Diffusion & Flow Models#

  • 96.4.1.1 Deploy FLUX.2 klein 4B for fast image generation (Apache 2.0, sub-1s at 1024px, 4 inference steps)
  • 96.4.1.2 Deploy FLUX.2 dev 32B for high-quality generation (flow matching transformer, multi-reference)
  • 96.4.1.3 Deploy Stable Diffusion 3.5 Medium for consumer hardware (2.5B, MMDiT-X, 4-step turbo)
  • 96.4.1.4 Integrate ControlNet + IP-Adapter for multi-control generation (22M additional params for image prompting)

96.5 Video Generation Models#

96.5.1 Video Diffusion & Flow#

  • 96.5.1.1 Deploy Wan 2.2 for video generation (27B MoE, first open MoE video model, Apache 2.0)
  • 96.5.1.2 Deploy LTX-2.3 for audio-video generation (22B, native 4K, 50FPS, synchronized audio, Apache 2.0)
  • 96.5.1.3 Deploy HunyuanVideo for cinematic video (13B, beats Runway Gen-3)
  • 96.5.1.4 Deploy CogVideoX 1.5 for accessible video generation (5B, 3D VAE, 10s at 16fps)

96.6 Audio & Speech Models#

96.6.1 TTS, STT & Music#

  • 96.6.1.1 Deploy Voxtral TTS 4B as primary TTS (9 languages, 90ms TTFA, 3s voice cloning, single 16GB GPU, Apache 2.0)
  • 96.6.1.2 Deploy Dia 1.6B for dialogue TTS (nonverbal cues: laughs/sighs/screams, Apache 2.0)
  • 96.6.1.3 Deploy Orpheus TTS 3B for emotional TTS (human-like emotion, Llama-based)
  • 96.6.1.4 Deploy Kokoro 82M as lightweight TTS (54 voices, 8 languages, sub-1GB, Apache 2.0)
  • 96.6.1.5 Deploy Whisper Large V3 Turbo for STT (5.4x faster, close to V2 accuracy)
  • 96.6.1.6 Deploy Cohere Transcribe 2B for multilingual ASR (Apache 2.0, tops Open ASR leaderboard)
  • 96.6.1.7 Deploy ACE-Step v1.5 for music generation (commercial-grade on consumer hardware)
  • 96.6.1.8 Deploy YuE for long-form music with lyrics (5min with lyrical alignment, Apache 2.0)

96.7 3D Generation Models#

96.7.1 3D Assets & Animation#

  • 96.7.1.1 Deploy Hunyuan3D 2.1 for production 3D (two-stage DiT, PBR materials, mini at 12GB VRAM, Apache 2.0)
  • 96.7.1.2 Integrate UniRig for automatic skeletal rigging (SIGGRAPH 2025, autoregressive bone-point cross-attention)
  • 96.7.1.3 Integrate Puppeteer for video-guided animation (skeleton → skinning → video motion transfer, MIT)
  • 96.7.1.4 Integrate gsplat for CUDA-accelerated Gaussian Splatting
  • 96.7.1.5 Support Khronos KHR_gaussian_splatting glTF extension (February 2026 standard)

96.8 Safety Models#

96.8.1 Content Safety & Detection#

  • 96.8.1.1 Deploy Llama Guard 4 12B as primary content safety (multimodal text+image, MLCommons taxonomy)
  • 96.8.1.2 Deploy Granite Guardian for RAG safety (hallucination detection 0.854 AUC)
  • 96.8.1.3 Integrate Deepfake-o-Meter for deepfake detection (18 models across image/video/audio)
  • 96.8.1.4 Deploy ShieldGemma for additional content moderation (2B/9B/27B variants based on Gemma 2)

96.9 Additional SOTA Models & Migration Infrastructure#

96.9.1 April 2026 Late Releases & Multilingual#

  • 96.9.1.1 Build GLM-5.1 serving pipeline (Z.ai April 2026, 58.4 SWE-Bench Pro #1, 8-hour autonomous agent sessions, MIT license — enhanced coding and tool use over GLM-5)
  • 96.9.1.2 Deploy Falcon Perception 0.6B for open-vocabulary grounding and segmentation (TII April 2026, early-fusion transformer, natural language prompt-based, lightweight)
  • 96.9.1.3 Deploy Cohere Tiny Aya 3.35B for multilingual inference (70+ languages, regional variants including Tiny Aya-Earth for African languages — critical for Veritas Ghanaian language support)
  • 96.9.1.4 Deploy Falcon 3 family for edge inference (1B/3B/7B/10B, multimodal text+image+video+audio, 14T token training, sub-13B SOTA, runs on laptops)
  • 96.9.1.5 Evaluate and optionally deploy Cohere Command R+ 104B for RAG-optimized inference (tool use, multilingual, optimized retrieval-augmented generation)

96.9.2 Model Selection, Migration & Ensemble#

  • 96.9.2.1 Build model evaluation and selection framework (benchmark all Phase 96 candidate models on Oshun-specific tasks per domain, produce quality/cost/latency matrix, recommend optimal model per use case)
  • 96.9.2.2 Build self-hosted vs API quality parity benchmark (compare self-hosted Llama 4/Qwen 3.5/DeepSeek V3.1 against Claude API/GPT API on Oshun domain tasks, track parity gap over time, define migration threshold)
  • 96.9.2.3 Build multi-model ensemble serving infrastructure (router selects optimal model per request based on task type/complexity, cascade from small to large on confidence threshold, aggregate ensemble for critical decisions)
  • 96.9.2.4 Build API-to-self-hosted model migration pipeline (inventory current API usage across all domains, map API calls to self-hosted equivalents, implement gradual traffic shifting with quality monitoring, fallback to API on quality drops)

96.10 Diffusion Language Models (Non-Autoregressive LM Paradigm)#

96.10.1 Masked / Discrete Diffusion LMs#

  • 96.10.1.1 Build Mercury Coder Small serving pipeline (Inception Labs, discrete diffusion coder, ~1000 tok/s on H100 at sub-1B active params — 5–10× dense AR throughput on single-file code completion) with Mercury-compatible inference kernel and streaming partial-unmask output
  • 96.10.1.2 Deploy LLaDA-8B (Large Language Diffusion with mAsking) — masked-diffusion LM trained on 2.3T tokens, competitive with Llama-3-8B on zero-shot benchmarks, Apache 2.0 — serve via custom sampler supporting low-confidence remasking and semi-autoregressive block decoding
  • 96.10.1.3 Deploy Seed-Diffusion LM 4B (ByteDance Seed) with block-diffusion hybrid decoder, 3-step sampling mode for ≤50 ms P50 latency on short prompts, long-mode 20-step for quality parity
  • 96.10.1.4 Deploy SEDD (Score Entropy Discrete Diffusion) research-grade small variants (0.3B / 1B) for Oshun-internal sampling experiments and eval-baseline comparisons
  • 96.10.1.5 Implement diffusion-LM-aware sampler library with configurable remasking schedules (confidence-based, top-k, nucleus, argmax), block-diffusion (semi-AR) streaming, classifier-free and classifier guidance, and KV-cache–free inference memory profile
  • 96.10.1.6 Benchmark diffusion LMs vs same-parameter AR LMs on throughput (tok/s single-stream and batched), latency-to-first-token, code-completion pass@1, creative-writing judged quality, and cost-per-million-tokens — publish scorecard and recommend routing policy per domain (e.g. diffusion-LM preferred for Iris autocomplete, AR preferred for long-form reasoning)
  • 96.10.1.7 Build fine-tuning recipe for masked-diffusion LMs — adapt Phase 89 SFT/DPO to the masked-diffusion forward process (loss only at masked tokens, time-conditional score matching) — required before domain-specialising diffusion LMs for @iris/code autocomplete

96.11 State-Space & Hybrid-Attention Language Models#

96.11.1 Pure SSM & Hybrid Stacks#

  • 96.11.1.1 Deploy Mamba-2 based LMs (state-space duality, selective SSM, Triton kernels) — serve Mamba-2 2.7B and 7B checkpoints with custom inference runtime supporting linear-time prefill and O(1)-memory streaming decode
  • 96.11.1.2 Deploy Jamba 1.5 Large (AI21, hybrid Transformer-Mamba-MoE, 94B active / 398B total, 256K context, Apache 2.0) — tensor-parallel serving with long-context batching and MoE expert-parallel routing
  • 96.11.1.3 Deploy Zamba2-7B (Zyphra, hybrid Mamba2 + shared attention, outperforms Mistral 7B and Llama-3-8B on several benches, Apache 2.0) for cost-efficient long-context serving
  • 96.11.1.4 Deploy Falcon-Mamba-7B (TII, pure Mamba, Apache 2.0) as the reference pure-SSM alternative to Transformer baselines for fair perf-parity studies
  • 96.11.1.5 Deploy Hymba 1.5B (NVIDIA, parallel hybrid attention+SSM heads) for edge-device inference on Galatea / Oshun shell wearable where 1-2B fits the memory envelope
  • 96.11.1.6 Build SSM-specific serving runtime — selective-scan Triton kernels, recurrent-state checkpointing every N tokens for deterministic resume, state-compression for session persistence, and benchmark vs vLLM / TensorRT-LLM on 1M-token RULER / BABILong / Ruler-NIAH
  • 96.11.1.7 Build SSM vs Transformer vs Hybrid evaluation matrix on Oshun-long-context tasks (@sophia/research-engine doc-QA over 500K tokens, @metis/lessons full-course context, Iris multi-session memory recall) — memory, throughput, and quality; drive routing to hybrid/SSM where long-context advantages justify it
  • 96.11.1.8 Build SSM / hybrid fine-tuning recipe integrated with Phase 89 PEFT stack (LoRA on hybrid attention layers + SSM-specific low-rank B/C projections) — required before adapting hybrid models to Oshun domains