## Phase 85: ML Sovereignty — Data Flywheel Core (Passive Training Data from Core AI Domains)

Every interaction in the Oshun system should produce training data as a byproduct
of normal operation. This phase wires up passive data collectors at each core AI
domain touchpoint. No new events need to be created — the events already exist;
these tasks extract training-relevant data from them.

---

### 85.1 Iris Conversation Data Collection

#### 85.1.1 Conversation & Preference Data Extraction

- [x] **85.1.1.1** Build conversation-to-training-pair exporter that converts completed Iris turns (user message + assistant response + turn metadata from ConversationEngineEvent) into SFT training examples in ShareGPT/OpenAI format
- [x] **85.1.1.2** Build preference pair collector from Iris feedback system (memory/personalization/feedback FeedbackProcessor: positive/negative/correction signals with sentiment scores and aspect categories → chosen/rejected pairs for DPO/SimPO/KTO)
- [x] **85.1.1.3** Build implicit preference detector from Iris conversation analytics (message regeneration detection from streaming events, context summarization as quality signal, tool failures as correction signal → implicit preference training data)
- [x] **85.1.1.4** Build conversation quality scorer that assigns multi-dimensional quality labels (helpfulness, accuracy, safety, style) to completed conversations for reward model training
- [x] **85.1.1.5** Build model routing outcome logger that records which model was selected by @iris/model-routing, the response quality, and user satisfaction signal → routing optimization training data
- [x] **85.1.1.6** Build RAG retrieval feedback collector from Iris knowledge interactions (which retrieved documents were used in final response vs ignored → retrieval relevance labels for reranker fine-tuning)
- [x] **85.1.1.7** Build tool-use training data collector from Iris agent tool calls (tool.called + tool.completed events → function calling training examples in OpenAI tool format)
- [x] **85.1.1.8** Build conversation summarization training data from Iris context.summarized events (original messages + generated summary → summarization training pairs)
- [x] **85.1.1.9** Build embedding quality feedback from Iris memory/embeddings (similar items retrieved and used → positive pairs; irrelevant items retrieved and ignored → hard negative pairs for contrastive training)
- [x] **85.1.1.10** Build user personalization training data from Iris adaptation rules (which rules improved user satisfaction metrics → personalization model training data)

#### 85.1.2 Ensemble, Emotional Intelligence & Advanced Signal Extraction

- [x] **85.1.2.1** Build ensemble multi-model comparison data from @iris/ensemble (member responses with roles primary/validator/critic/synthesizer + confidence scores + ConsensusResult agreement levels + DisagreementPoint severity + VotingResult + ResolutionResult → model comparison and consensus training data)
- [x] **85.1.2.2** Build emotional recognition training data from @iris/emotional (48 Hume emotion classifications from voice-analysis + ProsodicFrame features: pitch, energy, jitter, shimmer, HNR, tension + text emotion signals → emotion model fine-tuning data)
- [x] **85.1.2.3** Build multimodal emotion fusion data from @iris/emotional/multimodal-fusion (FusedEmotionResult with text+voice+context fusion weights + agreement scores → multimodal emotion fusion model training data)
- [x] **85.1.2.4** Build emotional response effectiveness data from @iris/emotional/response (response calibration decisions + tone matching outcomes + de-escalation success/failure + crisis support interventions + user satisfaction after → empathetic response model training data)
- [x] **85.1.2.5** Build rapport-building training data from @iris/emotional/rapport (trust indicator changes over sessions + relationship progression stages + personalized interaction outcomes → social AI training data)
- [x] **85.1.2.6** Build code understanding training data from @iris/code (code explanation quality + architecture analysis accuracy + bug fix pattern success + refactoring suggestion acceptance/rejection → code reasoning fine-tuning data)
- [x] **85.1.2.7** Build code generation quality data from @iris/code/generation (contextual autocomplete + test generation + generated code acceptance/rejection by user → code generation preference pairs)
- [x] **85.1.2.8** Build reasoning chain training data from @iris/reasoning-thinking and @iris/conversation/reasoning (reasoning steps + validation outcomes from thought-validator + metacognitive confidence calibration + knowledge gap detection → chain-of-thought fine-tuning data)
- [x] **85.1.2.9** Build intent classification training data from @iris/conversation-intent (detected intents + question types from question-detector + urgency levels + slot fills + user correction signals → intent classifier fine-tuning data)
- [x] **85.1.2.10** Build uncertainty calibration data from @iris/conversation-uncertainty (confidence estimates from confidence-estimator + actual correctness outcomes + knowledge boundary detections + clarification trigger patterns → calibrated confidence model training data)
- [x] **85.1.2.11** Build accessibility preference training data from @iris/accessibility (visual/hearing/motor/cognitive adaptive UI selections + user capability profiles + interface adapter outcomes → accessibility model training data)
- [x] **85.1.2.12** Build multi-agent collaboration data from @iris/a2a (task-negotiation outcomes + result-sharing quality + agent capability matching from agent-card → multi-agent orchestration training data)

### 85.2 Psyche Avatar & Interaction Data Collection

#### 85.2.1 Emotion, Expression & Voice Data Extraction

- [x] **85.2.1.1** Build emotion recognition training data pipeline from Psyche EmotionTrack (detected emotions with confidence + FACS action units + micro-expressions → emotion classifier training data, filtered by quality: face visibility > 0.8, landmark stability > 0.7)
- [x] **85.2.1.2** Build avatar expression training data from Psyche avatar-expressions (FLAME parameters: 100 shape + 50 expression + pose coefficients per frame → expression generation model training data)
- [x] **85.2.1.3** Build lip-sync training data from Psyche voice-engine (phoneme sequences + prosody data + LipSyncFrame viseme weights → audio-to-face training pairs)
- [x] **85.2.1.4** Build voice quality training data from Psyche voice-engine (synthesized audio + user engagement metrics → TTS quality ranking data for preference learning)
- [x] **85.2.1.5** Build behavior coordination training data from Psyche behavior-coordinator (behavior sequences executed + transitions + user engagement response → behavior policy training data)
- [x] **85.2.1.6** Build gaze tracking calibration data from Psyche perception-engine (tracked gaze coordinates + baseline calibration per user → gaze model fine-tuning data)
- [x] **85.2.1.7** Build avatar rendering quality data from Psyche avatar-core (render quality presets selected + FPS achieved + user quality complaints → adaptive quality model training data)

#### 85.2.2 Advanced Perception, Engagement & Computer-Use Data

- [x] **85.2.2.1** Build head pose and movement training data from @psyche/head-pose-estimation (nod/shake detection with intensity + fatigue indicators: head dropping, reduced movement + posture stability: jitter, drift + HeadVelocity/HeadAcceleration → attention/engagement model training data)
- [x] **85.2.2.2** Build gaze behavior training data from @psyche/gaze-estimation (Fixation/Saccade/SmoothPursuit/Blink events + AOIMetrics: dwell percentage, revisits, time to first fixation → visual attention model training data)
- [x] **85.2.2.3** Build computer-use agent training data from @psyche/computer-use-core (AgentStep: reasoning + action + result + screenshot sequences + AgentExecutionResult success/failure + total steps → computer-use model fine-tuning data)
- [x] **85.2.2.4** Build screen understanding training data from @psyche/computer-use-core (ScreenAnalysisResult: UI elements + text blocks + document type classification → screen analysis model training data)
- [x] **85.2.2.5** Build micro-expression training data from @psyche/micro-expressions (rapid expression changes + temporal windows + onset/offset timing + ground truth correlations → micro-expression classifier training data)
- [x] **85.2.2.6** Build face detection quality data from @psyche/face-detection + @psyche/face-analysis (detection confidence distributions + landmark precision + face quality metrics across lighting/angle conditions → face model fine-tuning data)
- [x] **85.2.2.7** Build engagement detection training data from @psyche/engagement-detector + @psyche/engagement-recovery (multimodal engagement signals + disengagement events + recovery intervention outcomes + re-engagement success rates → engagement model training data)
- [x] **85.2.2.8** Build attention tracking training data from @psyche/attention-tracker (attention scores from scoring-engine + lapse events from lapse-detector + shift patterns from shift-detector + topic correlations → attention prediction model training data)
- [x] **85.2.2.9** Build Tavus avatar pipeline data from @psyche/tavus-* (turn-taking decisions + expression mapping quality + persona consistency scores + perception outputs + objective completion rates → avatar conversation model training data)
- [x] **85.2.2.10** Build proactive behavior training data from @psyche/behavior-prediction + @psyche/anticipatory-responses (predicted behaviors + actual outcomes + anticipatory response effectiveness → proactive assistance model training data)
- [x] **85.2.2.11** Build user satisfaction training data from @psyche/satisfaction + @psyche/nps (satisfaction signals + NPS scores + interaction context → satisfaction prediction model training data)
- [x] **85.2.2.12** Build screen analysis training data from @psyche/screen-analysis (screen element detections + layout analysis + interaction predictions → screen understanding model training data)

### 85.3 Isis Generation Data Collection

#### 85.3.1 Generation Quality & Preference Data Extraction

- [x] **85.3.1.1** Build generation prompt-quality pairing pipeline from Isis job-envelope + outputs (text prompt + negative prompt + parameters + output files + ProvenanceRecord → prompt engineering training data with quality labels)
- [x] **85.3.1.2** Build user selection/rejection collector from Isis outputs (which generated outputs users selected vs discarded/regenerated → image/video/audio preference pairs for diffusion alignment via DPO)
- [x] **85.3.1.3** Build generation parameter optimization data from Isis operation-orchestrator (parameter combinations + quality tier assignments → parameter recommendation model training data)
- [x] **85.3.1.4** Build LoRA effectiveness tracker from Isis model-fine-tuning (which LoRA adapters produce higher-quality outputs per style/subject → LoRA routing training data)
- [x] **85.3.1.5** Build ComfyUI workflow outcome data from Isis workflows (workflow configurations + execution results + quality scores → workflow recommendation training data)
- [x] **85.3.1.6** Build 3D generation quality data from Isis 3d-inference-local (model used + input type + output quality metrics from 3d-quality-gates → 3D model quality prediction training data)
- [x] **85.3.1.7** Build video enhancement before/after pairs from Isis video-enhancement (original + enhanced frames with RIFE interpolation or upscaling → super-resolution training pairs)
- [x] **85.3.1.8** Build texture generation quality data from Isis ai-texturing (input mesh + generated texture + quality score → texture generation training data)

#### 85.3.2 Post-Production, Specialized Generation & Orchestration Data

- [x] **85.3.2.1** Build dailies analysis training data from @isis/post-production-ai (camera angle/movement classifications + shot type classifications + color palette extractions + lighting analysis + depth-of-field estimation → film analysis model training data)
- [x] **85.3.2.2** Build editorial consistency training data from @isis/post-production-ai (per-production editorial decisions + consistency learner outputs + continuity error detections → editorial AI training data)
- [x] **85.3.2.3** Build VFX pipeline training data from @isis/post-production-ai (wire/rig removal inputs/outputs + clean plate generation + background replacement + crowd augmentation quality → VFX automation model training data)
- [x] **85.3.2.4** Build video generation style preference data from @isis/seedance-provider (SeedanceTransitionArc choices + SeedanceSpliceBlendMode selections + quality presets + content types + user selections → video style preference training data)
- [x] **85.3.2.5** Build audio generation preference data from @isis/audio-generation (MusicGenre + MusicMood parameters + generation prompts + outputs + user selections → music/audio generation model training data)
- [x] **85.3.2.6** Build 3D scene assembly training data from @isis/3d-scene-assembly (scene configurations + asset placements + spatial relationships + quality assessments → 3D scene composition model training data)
- [x] **85.3.2.7** Build gaussian splatting quality data from @isis/gaussian-splatting (input captures + splat quality metrics + viewer preferences → 3DGS optimization training data)
- [x] **85.3.2.8** Build LLM generation orchestration data from @isis/llm-orchestrator + @isis/batch-llm-processing (prompt engineering decisions + batch processing outcomes + quality correlations → generation prompt optimization training data)

### 85.4 Sophia Knowledge & RAG Data Collection

#### 85.4.1 Search, Retrieval & Citation Data Extraction

- [x] **85.4.1.1** Build search query-result relevance data from Sophia semantic-search (query text + filters + returned results + clicked results vs ignored → relevance training data for reranker fine-tuning)
- [x] **85.4.1.2** Build citation verification training data from Sophia evaluation (citation text + verification status + citation checks → citation quality classifier training data)
- [x] **85.4.1.3** Build RAG faithfulness training data from Sophia evaluation (question + retrieved context + generated answer + faithfulness/relevancy scores → RAG quality training data)
- [x] **85.4.1.4** Build document chunking quality data from Sophia document-parser (document + chunking strategy used + downstream retrieval quality → optimal chunking model training data)
- [x] **85.4.1.5** Build embedding model comparison data from Sophia embeddings (same query against multiple embedding models + retrieval quality per model → embedding model selection training data)
- [x] **85.4.1.6** Build knowledge graph entity extraction training data from Sophia knowledge-base (document + extracted entities + verified relationships → NER and relation extraction training data)

#### 85.4.2 Advanced Knowledge, Credibility & Research Data

- [x] **85.4.2.1** Build citation network training data from @sophia/citation-graph + @sophia/citation-analysis (citation networks + influence paths + citation quality scores → citation recommendation model training data)
- [x] **85.4.2.2** Build source credibility training data from @sophia/credibility (source assessments + credibility score history + verification outcomes → credibility scoring model training data)
- [x] **85.4.2.3** Build content ingestion quality data from @sophia/crawling + @sophia/corpus (crawl outcomes + content extraction quality + deduplication decisions → content quality classifier training data)
- [x] **85.4.2.4** Build cultural research training data from @sophia/cultural-research (research queries + cultural context analysis + cross-cultural findings → cultural intelligence model training data)
- [x] **85.4.2.5** Build prediction accuracy data from @sophia/predictions (predictions made + actual outcomes + confidence calibration → prediction model training data)
- [x] **85.4.2.6** Build research agent training data from @sophia/research-engine + @sophia/agents (research queries + agent search strategies + information synthesis quality → research agent fine-tuning data)
- [x] **85.4.2.7** Build theory evaluation data from @sophia/theory (proposed theories + supporting evidence + verification outcomes → theory construction model training data)

### 85.5 Kuan Yin Safety & Moderation Data Collection

#### 85.5.1 Moderation Decision & Intervention Data Extraction

- [x] **85.5.1.1** Build moderation decision training data from Kuan Yin foundation (content + HarmCategory detected + severity + confidence → harm classifier training data, with PII scrubbed before storage)
- [x] **85.5.1.2** Build intent classification training data from Kuan Yin foundation (content + detected Intent code + valence + confidence → intent classifier training data)
- [x] **85.5.1.3** Build appeal outcome training data from Kuan Yin rehabilitation (original decision + appeal reason + appeal outcome → moderation quality improvement data for false positive reduction)
- [x] **85.5.1.4** Build emotional state modeling data from Kuan Yin foundation (content + EmotionalState with Plutchik dimensions + arousal/valence/dominance → emotion model training data)
- [x] **85.5.1.5** Build intervention effectiveness data from Kuan Yin foundation (intervention category + urgency + applied + user response + escalation outcome → intervention policy training data)

#### 85.5.2 Advanced Safety Signals

- [x] **85.5.2.1** Build mindful friction effectiveness data from @kuanyin/mindful-friction (cognitive reframing prompts + pause-breathe triggers + compassion nudge presentations + perspective-shift suggestions + user behavioral response after intervention → friction intervention model training data)
- [x] **85.5.2.2** Build performer protection training data from @kuanyin/performer-protection (parasocial behavior detections + boundary violation attempts + NCII/deepfake detection outcomes + performer wellness signals → performer safety model training data)
- [x] **85.5.2.3** Build community health training data from @kuanyin/community-harmony (temperature monitor readings + conflict detections + raid defense activations + post-incident healing outcomes + culture cultivation metrics → community health prediction model training data)
- [x] **85.5.2.4** Build predictive moderation data from @kuanyin/precognition (pre-harm behavioral pattern detections + cascade predictions + typing dynamic signals + intent analysis + prediction accuracy outcomes → predictive safety model training data)
- [x] **85.5.2.5** Build community wellness training data from @kuanyin/dharma-analytics (community health trends + individual dharma paths + moderation transparency outcomes + predictive wellness alerts → community wellness model training data)
- [x] **85.5.2.6** Build reputation system training data from @kuanyin/merit-karma (karma score trajectories + merit accumulation patterns + behavioral correlations → reputation prediction model training data)

### 85.6 Hathor NPC & Worldbuilding Data Collection

#### 85.6.1 NPC Dialogue, Emotion & World Simulation Data Extraction

- [x] **85.6.1.1** Build NPC dialogue training data from Hathor llm-npc (NPCPersonality Big Five traits + player query + world state + generated dialogue + quality assessment → character-consistent dialogue fine-tuning data)
- [x] **85.6.1.2** Build NPC emotional response data from Hathor llm-npc (interaction context + NPCEmotionalState transitions + player engagement → emotion-aware dialogue model training data)
- [x] **85.6.1.3** Build NPC memory relevance data from Hathor llm-npc (retrieved memories + access count + memory importance + dialogue quality → memory retrieval model training data)
- [x] **85.6.1.4** Build world simulation outcome data from Hathor simulation (economy transactions + political events + player decisions + downstream consequences → world model training data)
- [x] **85.6.1.5** Build NPC personality consistency data from Hathor llm-npc (Big Five traits + speech patterns + generated responses over time → personality consistency evaluation data)

#### 85.6.2 Granular Simulation & Narrative Data

- [x] **85.6.2.1** Build economy simulation training data from Hathor simulation/economy (MarketType dynamics + TradeRouteType activity + supply/demand fluctuations + guild economic interactions + economic event cascades → economic model training data)
- [x] **85.6.2.2** Build political simulation training data from Hathor simulation/politics (GovernmentType transitions + FactionType relations + PowerSource dynamics + alliance formations + conflict outcomes + election results + treaty negotiations → political AI model training data)
- [x] **85.6.2.3** Build cultural evolution training data from Hathor simulation/culture (TraitCategory diffusion patterns + TraitTransmission modes + DiffusionMechanism outcomes + AssimilationLevel dynamics + InnovationType adoption curves → cultural simulation model training data)
- [x] **85.6.2.4** Build NPC social network training data from Hathor llm-npc (RelationshipMemory trust evolution + interaction patterns across NPC network + social influence propagation → social AI model training data)
- [x] **85.6.2.5** Build narrative generation training data from @hathor/narrative + @hathor/lore-compiler (generated narratives + player engagement metrics + branching decisions + lore consistency checks → narrative AI model training data)
- [x] **85.6.2.6** Build scenario effectiveness data from Hathor simulation/scenario (scenario configurations + player engagement outcomes + outcome diversity → scenario generation model training data)

### 85.7 Veritas/Yaa News Data Collection

#### 85.7.1 News NLP, Fact-Checking & Bias Data Extraction

- [x] **85.7.1.1** Build article classification training data from Yaa NLP pipeline (articles + NER results + topic classifications → NER and topic classifier fine-tuning data, especially for Ghanaian entities)
- [x] **85.7.1.2** Build fact-checking training data from Yaa fact-verification (claims + evidence + verification outcomes → fact-checking model training data)
- [x] **85.7.1.3** Build bias detection training data from Yaa bias analysis (articles + GPL bias scores + political lean annotations → bias classifier training data)
- [x] **85.7.1.4** Build editorial decision training data from Yaa newsroom agents (agent decisions + story selections + placement choices → editorial AI training data)
- [x] **85.7.1.5** Build multilingual NLP training data from Yaa Ghana NLP integration (Twi/Ewe/Ga/Hausa text processing results → low-resource language model fine-tuning data)

#### 85.7.2 Advanced News Signals

- [x] **85.7.2.1** Build headline optimization training data from @veritas/ab-testing + @veritas/headline-service (headline variants + click-through rates + engagement duration + statistical significance → headline generation model training data)
- [x] **85.7.2.2** Build article revision training data from @veritas/archive (article versions + correction history + editorial amendment decisions → article improvement model training data)
- [x] **85.7.2.3** Build story clustering training data from @veritas/story-clustering (story groupings + deduplication decisions + cluster quality signals + merge/split editorial corrections → story similarity model training data)
- [x] **85.7.2.4** Build news knowledge graph training data from @veritas/knowledge-graph (entity extractions + event relations + temporal connections → news graph model training data)
- [x] **85.7.2.5** Build news recommendation training data from @veritas/recommendations (recommended articles + user engagement + preference signals → news recommendation model training data)
- [x] **85.7.2.6** Build news media production training data from @veritas/video-production + @veritas/audio-production (production decisions + quality metrics + audience engagement → news production AI training data)
- [x] **85.7.2.7** Build source reliability training data from @veritas/risk (source accuracy history + reliability scoring + retraction patterns → source trust model training data)

### 85.8 Euterpe & Aphrodite Content Data Collection

#### 85.8.1 Music Preference & Content Engagement Data Extraction

- [x] **85.8.1.1** Build music preference data from Euterpe analytics (plays + skips + favorites + completion percentage + sharing actions → music generation quality ranking data)
- [x] **85.8.1.2** Build audio generation quality data from Euterpe genesis (generation prompts + parameters + outputs + user selections → audio generation preference data)
- [x] **85.8.1.3** Build content recommendation feedback from Aphrodite recommendation-engine (recommendations shown + clicked + ignored + engagement duration → recommendation model training data)
- [x] **85.8.1.4** Build content moderation training data from Aphrodite content-safety (content items + moderation decisions + appeals → content classifier training data)

#### 85.8.2 Euterpe Advanced Music Data

- [x] **85.8.2.1** Build music composition training data from @euterpe/theory + @euterpe/score + @euterpe/accompany (harmony analysis + chord progressions + arrangement decisions + composition quality ratings → music theory model training data)
- [x] **85.8.2.2** Build audio mastering preference data from @euterpe/master (mastering parameter choices + A/B test results + quality ratings → mastering AI model training data)
- [x] **85.8.2.3** Build lyrics generation training data from @euterpe/lyrics (prompts + generated lyrics + user edits + quality ratings → lyrics model training data)
- [x] **85.8.2.4** Build spatial audio training data from @euterpe/spatial (spatial configurations + listener preferences + immersion ratings → spatial audio model training data)

#### 85.8.3 Aphrodite Advanced Platform Data

- [x] **85.8.3.1** Build streaming engagement training data from @aphrodite/streaming-core + @aphrodite/analytics (viewer behavior patterns + engagement metrics + retention signals + interaction types → streaming optimization model training data)
- [x] **85.8.3.2** Build payment behavior training data from @aphrodite/payment + @aphrodite/tip-* (tipping patterns + goal completion correlations + payment timing + revenue sharing outcomes → monetization prediction model training data)
- [x] **85.8.3.3** Build device interaction training data from @aphrodite/haptics + @aphrodite/device-* (haptic patterns + device response mappings + user satisfaction signals → haptic interaction model training data)
- [x] **85.8.3.4** Build consent and safety training data from @aphrodite/consent-engine + @aphrodite/safety-intelligence (consent verification patterns + safety intervention outcomes + autonomy boundary effectiveness → consent/safety model training data)
- [x] **85.8.3.5** Build virtual environment preference data from @aphrodite/virtual-rooms + @aphrodite/scene-* (room configurations + scene selections + aesthetic preferences → virtual environment generation model training data)

### 85.9 Minerva/Metis Education Data Collection

#### 85.9.1 Lecture Quality & Learning Data Extraction

- [x] **85.9.1.1** Build lecture quality training data from Minerva lecture generation (generated lectures + student engagement metrics + assessment performance → lecture generation quality data)
- [x] **85.9.1.2** Build assessment effectiveness data from Minerva tutoring (questions + student answers + time to answer + follow-up needed → assessment quality training data)
- [x] **85.9.1.3** Build learning progression data from Minerva learning-system (student knowledge states from DKT + skill acquisition rates → adaptive learning model training data)
- [x] **85.9.1.4** Build VLM diagram quality data from Minerva VLM pipeline (diagram descriptions + generated diagrams + educational effectiveness → VLM fine-tuning data)

#### 85.9.2 Advanced Educational Pipeline Data

- [x] **85.9.2.1** Build multi-agent generation trace data from @metis/agents-types (ReAct iterations: Thought + Action + Observation per agent + agent debates with DEBATE_POSITION/COUNTER_PROPOSAL messages + task delegation decisions + conflict resolutions → multi-agent orchestration model training data)
- [x] **85.9.2.2** Build linter violation training data from @metis/linters-types (FindingSeverity + FindingType + FindingCategory distributions per LinterKind + 15 finding types across content/quiz/richness/learning-objective/fact-claim/code/multi-modal/evaluation/terminology/bias/citation + auto-fix outcomes → content quality model training data)
- [x] **85.9.2.3** Build prompt effectiveness training data from @metis/prompt-management (prompt versions + generation quality correlation + A/B comparison results → prompt optimization model training data)
- [x] **85.9.2.4** Build learning path training data from @metis/knowledge-graph (prerequisite relationships + student traversal patterns + learning outcome correlations → adaptive learning path model training data)
- [x] **85.9.2.5** Build generation cost-quality correlation data from @metis/cost-tracking (LLM costs per generation + quality scores + model selections → cost-efficient generation model training data)
- [x] **85.9.2.6** Build content quality assessment training data from @metis/quality (quality scores + human evaluator feedback + quality dimension breakdowns → content quality scoring model training data)
- [x] **85.9.2.7** Build educational A/B testing data from @metis/ab-testing (experiment configurations + variant performance + statistical outcomes → educational content optimization model training data)

### 85.10 Cross-Domain Data Collection Infrastructure

#### 85.10.1 Centralized Training Data Pipeline

- [x] **85.10.1.1** Build centralized training data sink service that subscribes to all domain event buses and routes training-relevant events to the dataset registry with consistent schema (domain, data type, timestamp, quality score, format)
- [x] **85.10.1.2** Build privacy-first data collection pipeline with automatic PII detection and scrubbing (using Presidio/GLiNER) before any training data is stored
- [x] **85.10.1.3** Build consent management system that tracks user opt-in/opt-out for training data collection per domain and data type
- [x] **85.10.1.4** Build data collection rate limiter to prevent storage bloat (sample strategically: high-quality examples always stored, average examples sampled at configurable rate)
- [x] **85.10.1.5** Build training data deduplication service (near-duplicate detection using MinHash/SimHash) running continuously as new data arrives
- [x] **85.10.1.6** Build data collection health dashboard showing ingest rates per domain, quality distribution, storage usage, and coverage gaps
- [x] **85.10.1.7** Build automated data quality scoring for all collected training data (multi-dimensional: completeness, consistency, diversity, toxicity, usefulness)
- [x] **85.10.1.8** Build data collection audit trail for compliance (what data was collected, when, from whom, for what purpose, retention policy applied)

#### 85.10.2 Advanced Collection Infrastructure

- [x] **85.10.2.1** Build training data feature store that transforms raw collected data into model-ready feature vectors (text tokenization, embedding pre-computation, numerical normalization, categorical encoding) with versioning and point-in-time correctness
- [x] **85.10.2.2** Build dual-mode collection pipeline with streaming ingestion for real-time signals (conversation events, emotion/gaze/attention streams, moderation decisions) and batch ingestion for post-hoc labels (quality scores, A/B test results, human evaluations)
- [x] **85.10.2.3** Build dataset versioning and lineage tracking (dataset snapshots with content hashes, training job → dataset version mapping, data provenance chain for reproducibility)
- [x] **85.10.2.4** Build cross-domain data correlation pipeline that joins training signals from multiple domains on shared interaction/session IDs for multi-task and multi-signal model training
- [x] **85.10.2.5** Build annotation infrastructure for human labeling of passively collected data (annotation queues, inter-annotator agreement tracking, active learning sample selection)
- [x] **85.10.2.6** Build training data bias detection pipeline (demographic representation analysis, topic coverage analysis, temporal drift detection, poisoning/adversarial example detection)

### 85.11 Streaming Infrastructure & Data Transport

The event-bus uses Redis pub/sub which is at-most-once delivery (messages lost if
no subscriber listening). Kafka is provisioned in Docker but has zero integration
code. These tasks build the durable, replayable data transport layer.

#### 85.11.1 Streaming & CDC Foundation

- [x] **85.11.1.1** Build @oshun/streaming library wrapping KafkaJS with typed producers/consumers, schema registry integration (Avro/JSON Schema), consumer group management, exactly-once semantics, and backpressure handling
- [x] **85.11.1.2** Build CDC (Change Data Capture) infrastructure using Debezium to stream PostgreSQL WAL changes from all domain databases (yemaya, lilith, isis, sophia, hathor, bellona, veritas, aphrodite + others) to Kafka topics — the foundational mechanism for passive data collection without application-level event emission
- [x] **85.11.1.3** Define signal latency tier classification system: Tier 1 sub-second (emotion/gaze/attention, moderation decisions) via Kafka low-latency consumers; Tier 2 minutes (conversation/generation completion) via event-bus→Kafka bridge; Tier 3 daily (A/B results, quality scores, human evaluations) via scheduled batch jobs
- [x] **85.11.1.4** Migrate training data collection from Redis pub/sub to Redis Streams (XADD/XREADGROUP) or Kafka for durable, replayable, consumer-group-based delivery — prevent message loss during sink downtime
- [x] **85.11.1.5** Define canonical TrainingDataEnvelope schema: {dataId, domain, dataType, format, timestamp, qualityScore, consentId, interactionId, schemaVersion, payload} — all collectors MUST emit this
- [x] **85.11.1.6** Build backpressure mechanism for data ingestion pipeline (consumer lag monitoring, producer rate throttling based on sink capacity, overflow-to-S3 for burst traffic)
- [x] **85.11.1.7** Build pipeline-specific observability metrics: training_data_ingested_total{domain,type}, processing_latency_seconds, quality_score{domain}, sink_lag_seconds, errors_total{domain,type,error_class} via Prometheus

### 85.12 Feature Store

#### 85.12.1 Online & Offline Feature Serving

- [x] **85.12.1.1** Build online feature store service backed by Redis for low-latency feature retrieval at inference time (<10ms) — entity-key lookups (user_id, session_id, content_id), TTL-based feature expiry, feature freshness tracking
- [x] **85.12.1.2** Build offline feature store backed by PostgreSQL/TimescaleDB with timestamp-indexed feature rows, point-in-time query API (get features as-of timestamp T), integration with dataset registry (Phase 87) — prevents data leakage from future features
- [x] **85.12.1.3** Build feature computation engine with declarative feature definitions (e.g., "user's average session length over last 7 days"), batch (scheduled) and streaming (on-event) computation modes, automatic materialization to both online and offline stores
- [x] **85.12.1.4** Build cross-domain feature catalog with domain ownership, access policies, discovery API, and usage tracking — Iris-computed features available to Kuan Yin/Psyche without recomputation
- [x] **85.12.1.5** Evaluate and integrate Feast or Tecton as feature store backend (Feast v0.10: sub-ms Redis serving, Python/SQL/Spark transformations; Tecton: declarative DSL, GitOps management, scheduling)

### 85.13 Data Quality & Observability

#### 85.13.1 Ingestion Validation & Monitoring

- [x] **85.13.1.1** Build ingestion-time schema validation using Zod/JSON Schema — every training data type gets a registered schema; incoming data validated; failures counted, logged, routed to quarantine queue
- [x] **85.13.1.2** Build data distribution monitor tracking statistical distributions (mean, variance, percentiles, categorical frequencies) per data source, computing KL divergence / KS statistics against baseline windows, alerting on significant drift
- [x] **85.13.1.3** Build data freshness dashboard tracking last-received timestamp and event rate per domain/data-type combination — alert when any source goes stale beyond expected cadence
- [x] **85.13.1.4** Build volume anomaly detector using rolling window comparisons (current hour vs same hour last week) with configurable sensitivity — detect 10x surges (duplicating bug) or 0.1x drops (broken collector)
- [x] **85.13.1.5** Integrate Evidently AI for embedding drift detection (5 methods: Euclidean, cosine, MMD, model-based classifier, per-component) — monitor embedding quality for all domains producing vector data
- [x] **85.13.1.6** Build data completeness monitor tracking per-field null rates and alerting on degradation

### 85.14 Feedback Loop Closure (Model Performance → Smarter Collection)

#### 85.14.1 Adaptive Collection Strategies

- [x] **85.14.1.1** Build uncertainty-aware data collection router — when any model serves a prediction with confidence in the uncertain zone (0.4-0.6), automatically route interaction to: priority annotation queue, enriched data collection (capture more context), A/B variant serving (expose to multiple models for comparison data)
- [x] **85.14.1.2** Build difficulty-aware collection strategy using evaluation results (Phase 91) to identify model weakness areas → increase collection priority for signals matching weakness patterns
- [x] **85.14.1.3** Build error-case mining pipeline connecting model serving (Phase 88) error/correction signals to data collection with priority routing — wrong predictions are goldmines for training
- [x] **85.14.1.4** Build evaluation-to-collection bridge mapping benchmark regressions per domain to increased collection quotas for corresponding data types
- [x] **85.14.1.5** Build Tesla-style trigger classifier system — lightweight classifiers (SVMs, small neural networks) running on production outputs to identify high-value edge cases for priority data collection, rather than collecting everything uniformly
- [x] **85.14.1.6** Build self-improvement data loop — model generates candidate training data (self-instruct pattern) → quality-filter using existing models → add to training set if above threshold → retrain → repeat (Absolute Zero Reasoner / ALAS pattern)

### 85.15 Privacy & Compliance at Scale

#### 85.15.1 Production Privacy Infrastructure

- [x] **85.15.1.1** Build differential privacy integration for training data aggregation (DP-SGD noise injection, per-sample gradient clipping budgets, privacy budget epsilon tracking per dataset version)
- [x] **85.15.1.2** Build k-anonymity enforcement for training data with quasi-identifiers (generalization and suppression to ensure k>=5 for all quasi-identifier combinations before storage)
- [x] **85.15.1.3** Build GDPR deletion propagation service — on deletion request: identify all training data entries via user-ID index → remove from all dataset versions → invalidate affected features → log deletion for audit → flag affected model versions for retraining evaluation
- [x] **85.15.1.4** Extend consent management to include revocation workflows — revocation event → find all data collected under that consent → delete from all downstream stores → recompute affected dataset versions
- [x] **85.15.1.5** Build domain privacy classification system (Tier 1: public data, Tier 2: user-contributed, Tier 3: behavioral/interaction, Tier 4: biometric/health/financial/adult) — progressively stronger privacy controls per tier

### 85.16 Cross-Domain Signal Correlation

#### 85.16.1 Unified Identity & Temporal Alignment

- [x] **85.16.1.1** Define and enforce unified interaction context schema: {interactionId, sessionId, userId, timestamp, domainOrigin} — MUST be attached to every cross-domain event, with middleware that auto-injects from request scope
- [x] **85.16.1.2** Build temporal alignment buffer that collects signals for a given interaction ID within configurable window, then emits unified multi-signal record once window closes or all expected signals arrive — aligns Psyche gaze (<50ms), Iris response (~2s), Kuan Yin safety (~5s)
- [x] **85.16.1.3** Build cross-domain signal join engine with configurable join semantics (inner vs left outer), window duration per signal type, late-arrival grace period, watermarking, and output schema per join combination

### 85.17 Data Lifecycle & Training Orchestration

#### 85.17.1 Lifecycle Management & Retraining Triggers

- [x] **85.17.1.1** Build training data tiering system (hot: MinIO SSD <1 day actively used, warm: standard 1-30 days available for retraining, cold: Glacier 30-365 days audit access, archive: 365+ regulatory retention)
- [x] **85.17.1.2** Build data freshness weighting system assigning decay weights to training examples based on collection timestamp, configurable per domain (news data ages fast, medical knowledge slowly)
- [x] **85.17.1.3** Build dedicated data poisoning detection pipeline — anomaly detection on individual samples (outlier embedding distance), influence function estimation (which examples most affect outputs), population-level detection (sudden distribution shifts from single source)
- [x] **85.17.1.4** Build retention policy engine with per-domain and per-data-type rules, automated enforcement, and audit trail
- [x] **85.17.1.5** Build data-volume-based retraining trigger — track new sample counts per dataset per training recipe; when count exceeds configurable threshold, emit training request to Phase 89 pipeline
- [x] **85.17.1.6** Build quality-gated retraining trigger — before full training run, run fast evaluation (subset train on new data, evaluate on held-out set) to estimate expected improvement; only trigger if gain exceeds threshold
- [x] **85.17.1.7** Build retraining strategy configuration with both cron-scheduled (weekly/daily per model) and event-driven (threshold-based) modes, with conflict resolution
- [x] **85.17.1.8** Build ML pipeline orchestrator (Prefect/Dagster/custom) defining full lifecycle: data trigger → dataset assembly (Phase 87 recipe) → training launch (Phase 89) → evaluation (Phase 91) → model registry promotion → serving deployment (Phase 88) — the critical glue between all ML sovereignty phases

### 85.18 Implicit Behavioral Signal Extraction

These tasks extract higher-order behavioral patterns from sequences of events,
temporal relationships, and cross-domain correlations — the highest-value,
lowest-friction training signals.

#### 85.18.1 Temporal & Sequential Signals

- [x] **85.18.1.1** Build inter-message dwell time collector for all conversational interfaces (Iris, Psyche, Hathor NPC) — time between assistant response delivery and user's next keypress is a direct response quality signal
- [x] **85.18.1.2** Build prompt refinement diff tracker across all prompt-accepting domains (Iris, Isis, Sophia) — the diff between original and refined prompt reveals user intent more precisely than either prompt alone
- [x] **85.18.1.3** Build content consumption speed curve extractor (Minerva/Metis per-section time, Veritas per-paragraph time, Nyx per-lesson section) — acceleration indicates easy/boring content, deceleration indicates hard/engaging content
- [x] **85.18.1.4** Build search session reformulation chain collector (Sophia, Veritas, Maya) — the sequence of query1→query2→query3 within a session reveals intent evolution, not just individual queries
- [x] **85.18.1.5** Build screen-to-screen navigation sequence collector across all domains — ordered page/screen visit sequences as training data for navigation optimization and user journey modeling
- [x] **85.18.1.6** Build cross-domain navigation journey tracker — when user goes from Iris→Sophia→Isis in a session, capture the cross-domain workflow as multi-domain intent training data

#### 85.18.2 Abandonment & Recovery Signals

- [x] **85.18.2.1** Build mid-conversation abandonment detector for Iris (user stops responding mid-thread, distinct from conversation end) — silence duration + context = dissatisfaction signal
- [x] **85.18.2.2** Build user error-recovery behavioral pattern tracker across all domains — what users DO after encountering an error (retry? rephrase? give up?) as error-recovery sequence training data
- [x] **85.18.2.3** Build funnel drop-off position collector from Aphrodite + Tara + onboarding flows — exact step where users abandon as UX optimization training data
- [x] **85.18.2.4** Build goal/task abandonment causal context collector from Arete (AbandonmentPattern interface already exists in codebase) + Maya (quest abandonment) + Meditation (session early termination with content trigger identification)

#### 85.18.3 Comparison & Preference Signals

- [x] **85.18.3.1** Build implicit comparison signal from sequential generation in Isis — when user generates multiple variations and keeps one, capture the full sequence + parameter deltas between attempts + final selection as preference training data
- [x] **85.18.3.2** Build positional skip-in-context signal for Euterpe — when user picks one song over adjacent ones in queue/playlist, the skipped songs with their position context are implicit negatives
- [x] **85.18.3.3** Build universal save/bookmark/export/copy signal collector across ALL domains — content that users save, export, copy to clipboard, or share externally is the strongest positive quality signal; content ignored is implicit negative
- [x] **85.18.3.4** Build proactive suggestion accept/reject training data from Psyche (suggestion-engine already tracks accepted/dismissed/ignored with responseTimeMs) — faster acceptance = stronger positive signal

#### 85.18.4 Speed & Rhythm Signals

- [x] **85.18.4.1** Build breathing pattern deviation tracker for Meditation — delta between guided breathing rate and user's actual breathing as comfort/difficulty signal
- [x] **85.18.4.2** Build Maya player movement speed as zone interest signal — walking speed variations across game zones (slow=exploring/engaged, fast=rushing/disinterested)
- [x] **85.18.4.3** Build back-button/undo frequency tracker as universal dissatisfaction signal across all web/app domains
- [x] **85.18.4.4** Build typing hesitation and rewrite detector for Iris — pauses mid-typing or delete/rewrite patterns reveal user uncertainty about how to phrase requests

### 85.19 SOTA Self-Improvement & Shadow Mode Patterns

#### 85.19.1 Autonomous Improvement Infrastructure

- [x] **85.19.1.1** Build shadow mode evaluation system — run candidate improved models alongside production model on real traffic without affecting users; compare outputs; use disagreements as high-value training data (Tesla shadow mode pattern)
- [x] **85.19.1.2** Build TikTok Monolith-style real-time parameter update infrastructure — separate training-PS that updates from real-time event stream and inference-PS that syncs periodically, closing the feedback loop at minute-level intervals for recommendation models
- [x] **85.19.1.3** Build Midjourney-style implicit preference collection — integrate pair ranking into generation UX (show 2+ variants, user picks preferred) to create RLHF preference data from normal product usage at massive scale
- [x] **85.19.1.4** Build ALAS-style autonomous learning loop — curriculum generation → retrieval & QA synthesis → SFT → LLM-as-judge evaluation → DPO on errors → re-evaluation; converge when accuracy improvement drops below delta or all topics exceed mastery threshold
- [x] **85.19.1.5** Build Meta UTIS-style user satisfaction survey integration — lightweight alignment layer trained on periodic user satisfaction surveys (How well does this match your interest? 1-5) to calibrate implicit signal weights
- [x] **85.19.1.6** Build knowledge distillation flywheel (Meta GEM pattern) — foundation model trains on full data → distills to production models via Student Adapter using most recent ground-truth → Student Adapter continuously refreshed

### 85.20 Missing Domain Coverage

Domains with zero or near-zero data flywheel coverage that have substantial
data-producing code.

#### 85.20.1 Galatea (Robotics) Data Collection

- [x] **85.20.1.1** Build robot teleoperation episode data pipeline from @galatea/ai/data-collection (joint angles + contact forces + scene objects + task progress + quality scores + SAM2/DINO auto-labels → VLA imitation learning training data)
- [x] **85.20.1.2** Build RL environment data pipeline from @galatea/ai/reinforcement-learning (reward breakdowns: progress/stability/energy/noise/slip + domain randomization + curriculum stage transitions → locomotion policy training data)
- [x] **85.20.1.3** Build federated learning round data from @galatea/fleet/federated-learning (gradient updates + differential privacy + low-confidence scenarios + model aggregation results → cross-robot model improvement data)
- [x] **85.20.1.4** Build perception data pipeline from @galatea/perception (person detection + garment recognition + fit analysis + bounding boxes + confidence scores → fashion/retail vision model training data)
- [x] **85.20.1.5** Build robot operational analytics from @galatea/sdk/analytics-sdk (engagement metrics + fleet performance + safety events + OTA update results → robot operations prediction data)

#### 85.20.2 Nous (Core AI Platform) Data Collection

- [x] **85.20.2.1** Build Nous safety classifier feedback loop from @nous/safety (25+ safety detection system outputs: toxicity/hate/harassment/PII/jailbreak/deepfake classification decisions with confidence → safety classifier improvement training data, the classifiers should improve from their own production decisions)
- [x] **85.20.2.2** Build Nous vision inference feedback from @nous/vision (video classification/captioning/editing/upscaling/QA results + downstream user acceptance → vision model improvement training data)
- [x] **85.20.2.3** Build Nous audio inference feedback from @nous/audio (ASR/classification/enhancement/fingerprinting/forensics results + downstream quality signals → audio model improvement training data)
- [x] **85.20.2.4** Build Nous LLM reasoning trace data from @nous/llm (chain-of-thought traces + classification decisions + agent coordination logs + code extraction results → reasoning model fine-tuning data)
- [x] **85.20.2.5** Build Nous training meta-data from @nous/training (loss curves, convergence data, comparison results across training runs → meta-learning for training efficiency optimization)

#### 85.20.3 Oshun Shell (Cross-Domain Application Layer) Data Collection

- [x] **85.20.3.1** Build Oshun shell assistant interaction data from @oshun/shell-assistant (intent resolution results + action routing decisions + response quality signals → assistant model training data)
- [x] **85.20.3.2** Build Oshun cross-domain usage correlation data from @oshun/domain-registry + @oshun/navigation (domain switching patterns + feature discovery paths + cross-domain workflow sequences → user journey model training data)
- [x] **85.20.3.3** Build Oshun wearable interaction data from @oshun/shell-wearable (complication selections + haptic pattern effectiveness + summary companion usage patterns → wearable UX model training data)
- [x] **85.20.3.4** Build Oshun routine effectiveness data from @oshun/shell-routines (routine recommendations + step execution completion + routine abandonment patterns → routine recommendation model training data)
- [x] **85.20.3.5** Build Oshun achievement/gamification data from @oshun/shell-achievements (achievement unlocking patterns + social accountability interactions + engagement correlation → engagement prediction model training data)

---
