Date: 2026-06-16 Context: METIS_GAP_CLOSURE_TODOS_2026-06-16.md Phase
0; gap analysis METIS_CORRECTNESS_AND_MEDIA_SOTA_GAP_ANALYSIS_2026-06-16.md.
Decision#
Build a metis-local @metis/verification library that REUSES the shared
@oshun/content-quality-judge engine for the generic, already-verified
machinery, and ADDS the education-specific verifiers the shared engine does not
provide.
What we reuse from @oshun/content-quality-judge (do not re-implement)#
The shared package is real and tested (312+ tests; see
V1_V9_AGENTIC_CONTENT_SOTA_ASSESSMENT_2026-06-14.md §2). Metis consumes:
JudgePanel/judge-engine— calibrated, position-bias-mitigated LLM-judge panel (Task 1.8 pedagogical-quality dimensions).calibration— κ/Pearson/MAE vs human gold.grounding-gate(ClaimExtractor,GroundingRetriever,createGroundingGate) — claim extraction + grounding primitive (Tasks 1.1–1.2 build on this).best-of-n,self-refine,reflexion— verifier-guided generation (Task 3.4).active-learning,drift,quality-champion-challenger— HITL + drift (Tasks 0.6, 6.2).compute-budget, budget/kill-switch — budgets (Tasks 0.7, 6.1).
What is metis-specific and lives here (the gap)#
Education is the verifiable domain (math/code/facts have ground truth), so metis adds verifiers the creative-content engine has no reason to:
- Atomic claim decomposition + per-claim factuality vs the metis knowledge graph + research corpus (FActScore / SAFE F1@K). (Tasks 1.1–1.2)
- RAG faithfulness / TRACe adherence and citation sufficiency (entailment, not format). (Tasks 1.3–1.4)
- STEM correctness: CAS/symbolic math, sandboxed code execution, and a process reward model for worked-solution steps. (Tasks 1.5–1.7)
- Contradiction detection, calibration/abstention, pedagogical/curriculum correctness. (Tasks 3.1, 3.3, 5.1)
Integration boundary#
@metis/verificationexposes aVerificationGatecomposingVerifiers; education-specific verifiers are implemented here, the pedagogical-quality judge is the sharedJudgePanelwrapped as aVerifier.- The gate is fail-loud: a required verifier that cannot run
(
notConfigured) blocks release — it never silently passes. - Heavy models (NLI/DeBERTa faithfulness, CAS, code sandbox, PRM, Manim) run in
the Python
apps/metis/service; the TS lib calls them through typed clients that fail loud when an endpoint is unconfigured. - The gate plugs into
@metis/llm-clientcontent generation (Task 1.9) and the agentic media loop (Phase 2), mirroring the shared engine'sgenerate → judge-gate → refineloop.
Rejected alternative#
Adopt @oshun/content-service wholesale. Rejected: it is tuned for creative
artifacts without ground-truth verifiers; metis needs domain verifiers as
first-class gate members. We reuse its judge/calibration/loop primitives rather
than its creative pipeline shell.