Eve library

Metis Operator Training And Launch Criteria

Guide

Revision 1

On this page

Status: Canonical
Last updated: 2026-04-11
Owners: Metis Operations, Metis Product, Metis Safety, Metis Support

Purpose#

This document defines the minimum operator training package and the launch criteria required before Metis publication workflows are treated as launch- ready.

It is paired with:

  • docs/domains/metis/METIS_PUBLICATION_RELEASE_GATES_RUNBOOK.md

Operator Training Curriculum#

Every operator with publish, review, rights, or incident authority must finish the following modules:

  1. Release-gate model Learn the six publication gates, the enforced thresholds, and the fail- closed behavior.
  2. Workspace routing Practice moving between /reviews, /rights, /operations, /monitoring, and /incidents without losing ownership context.
  3. Rights and provenance review Learn how restricted, unknown, and missing source-license signals block publication.
  4. Grounded build review Practice interpreting evidence bundles, low-confidence output, and abstention-required states.
  5. Publication package review Learn how to sign off only when learner-facing lessons, assessments, and packaging metadata are complete.
  6. Incident escalation Practice when to route a publish blocker into incident command instead of continuing queue-local review.

Required Training Drills#

Each operator must complete and pass:

  1. A blocked-rights simulation
  2. A low-groundedness publication failure simulation
  3. An incomplete-assessment automation-coverage simulation
  4. A policy-sensitive tutoring spillover escalation simulation
  5. A clean publication approval walkthrough with audit evidence capture

Launch Criteria#

Metis publication is launch-ready only when all statements below are true:

  • publication attempts are blocked automatically when any release gate fails
  • admin approval cannot bypass the same gates that block instructor publication
  • operators have one canonical runbook for gate triage and escalation
  • operators know which workspace resolves each gate family
  • launch reviewers can explain the release readiness score and its blockers
  • rights, groundedness, and assessment support are treated as launch-blocking signals rather than advisory warnings
  • every publish or deny path writes auditable evidence

Role-Based Readiness#

Instructors#

Must know:

  • which course changes affect groundedness, provenance, and assessment support
  • that direct publish attempts fail closed when requirements are missing
  • where to look when Metis returns a publication conflict

Review Operators#

Must know:

  • how to clear grounded build and publication package blockers
  • how to distinguish low-confidence evidence issues from rights issues
  • when a course can return to queue versus when it needs incident command

Rights Operators#

Must know:

  • how source-license signals are classified
  • what counts as unknown-rights carryover
  • why publication cannot proceed with unresolved allowed-use posture

Incident Command#

Must know:

  • how release-gate failures map to incident categories
  • when learner impact turns a queue blocker into an incident
  • how to route back into the owning workspace after stabilization

Signoff Checklist#

  • Engineering signoff: release-gate enforcement is live on both publish paths
  • Operations signoff: runbook and training package published
  • Safety signoff: grounding and policy-sensitive escalation posture reviewed
  • Rights signoff: rights-clearing workflow and unknown-rights handling accepted
  • Product signoff: launch criteria match learner-facing release expectations
  • recorded walkthrough or live shadow signoff
  • drill completion checklist per operator
  • one successful blocked-publication remediation exercise
  • one successful publish approval exercise with audit evidence

Verification#

  • docs/domains/metis/METIS_PUBLICATION_RELEASE_GATES_RUNBOOK.md exists and matches the enforced gate model
  • pytest services/metis/tests/test_admin_review_gates.py services/metis/tests/test_backend_service_coverage.py passes
  • ruff check services/metis/src/metis/services/publication_release_gates.py services/metis/src/metis/services/course_service.py services/metis/src/metis/api/courses.py services/metis/src/metis/services/admin_service.py services/metis/src/metis/api/admin.py services/metis/tests/test_backend_service_coverage.py services/metis/tests/test_admin_review_gates.py passes