# Metis — Technical Specifications

Metis is the education and course-generation domain of the Oshun platform. The
production implementation is a Python backend service located at
`services/metis/`. This document specifies what is actually implemented in that
service — every table, every endpoint, every enumeration, and every
configuration variable — grounded in the real source code.

- **Language / runtime:** Python 3.11+, FastAPI, SQLAlchemy 2.0 (async), Celery
- **Package name:** `metis-backend` (`pyproject.toml`), version `0.1.0`
- **Application factory:** `metis.main:create_app`
  (`services/metis/src/metis/main.py`)
- **API mount prefix:** all domain routers mount under `/api`
- **OpenAPI artifact:** `services/metis/openapi/metis.openapi.json` (OpenAPI 3,
  149 path items, 174 operations, 519 component schemas at the time of writing)

> Note: a separate, large TypeScript library tree exists at `libs/metis/*`. This
> specification documents the deployed Python service under `services/metis/`,
> which is the runnable Metis backend.

---

## 1. Service Architecture

### 1.1 Process Topology

Metis runs as three cooperating process types, all built from the same Docker
image (`services/metis/docker-compose.yml`). The split between API, worker, and
beat ensures that long-running background jobs cannot starve synchronous HTTP
request handling.

| Process        | Command                                              | Role                           |
| -------------- | ---------------------------------------------------- | ------------------------------ |
| `metis-api`    | `gunicorn metis.main:app` with `UvicornWorker` (×4)  | FastAPI REST API               |
| `metis-worker` | `celery -A metis.tasks.celery_app:celery_app worker` | Celery background task worker  |
| `metis-beat`   | `celery -A metis.tasks.celery_app:celery_app beat`   | Celery periodic-task scheduler |

The production container (`services/metis/Dockerfile`) is a two-stage Python
3.11-slim build: `uv` installs dependencies plus `gunicorn`; the runtime stage
adds `curl`, `tesseract-ocr`, and `tini`, runs as a non-root `metis` user, and
exposes port 8000 with a `/health` HEALTHCHECK.

### 1.2 Application Lifecycle

The FastAPI application is constructed by a factory function rather than at
module level, which makes it easy to instantiate with different settings in
tests. `create_app()` configures the FastAPI app with: CORS middleware, an
`X-Request-Time` timing middleware, structured-logging configuration via
`structlog`, optional OpenTelemetry instrumentation, exception handlers, health
endpoints, and all API routers. The lifespan context manager initializes the
async database engine and the Redis connection pool on startup and closes them
on shutdown (`metis.deps`).

`docs_url` / `redoc_url` / `openapi_url` are served in non-production
environments only.

### 1.3 Infrastructure Dependencies

The table below lists every external service Metis depends on and the library
used to connect to it. All defaults are calibrated for the local
`docker-compose` stack; production deployments override each variable via
environment.

| Dependency   | Library / driver         | Local default (docker-compose)                         |
| ------------ | ------------------------ | ------------------------------------------------------ |
| PostgreSQL   | SQLAlchemy 2.0 + asyncpg | `postgresql+asyncpg://oshun:oshun_dev@.../oshun_dev`   |
| Redis        | `redis.asyncio`          | `redis://localhost:6379/0`                             |
| Celery       | `celery[redis]`          | broker `redis://.../1`, result backend `redis://.../2` |
| Object store | `boto3` (S3 / MinIO)     | `http://localhost:9000` (`minioadmin`)                 |
| Email (dev)  | `smtplib` → Mailpit      | development notification delivery                      |

The root `docker-compose.dev.yml` supplies PostgreSQL, Redis, MinIO, and
Mailpit. There is no RunPod, no managed GPU fleet, no AWS ECS/CloudFront
infrastructure, and no Metis-specific Terraform in the repository.

---

## 2. Persistence — Relational Schema

SQLAlchemy ORM models live in `services/metis/src/metis/models/`. The schema is
created by the Alembic migration `alembic/versions/001_initial.py`
(`revision = "001"`, create date 2026-03-05), which defines 12 tables.
PostgreSQL is used; pgvector is not used by this service.

### 2.1 TimestampMixin

Every table inherits from `metis.models.base.TimestampMixin`, which provides
three universal columns. Using a mixin rather than repeating these columns
ensures consistent types and default values across the entire schema.

| Column       | Type                      | Notes                                     |
| ------------ | ------------------------- | ----------------------------------------- |
| `id`         | `UUID` (PK, indexed)      | default `uuid.uuid4`                      |
| `created_at` | `DateTime(timezone=True)` | set on insert; server default `now()`     |
| `updated_at` | `DateTime(timezone=True)` | set on insert; `onupdate` on every UPDATE |

### 2.2 `users`

Platform user accounts (`metis.models.user.User`). The `role` column controls
which API endpoints a user can reach; `preferences` stores the full learner
preferences struct as a JSON-encoded string, keeping the schema simple while
still allowing rich, typed preference access in application code.

| Column            | Type           | Required | Notes                                                    |
| ----------------- | -------------- | -------- | -------------------------------------------------------- |
| `email`           | `String(255)`  | yes      | unique, indexed                                          |
| `name`            | `String(255)`  | yes      | display name                                             |
| `hashed_password` | `String(512)`  | yes      | bcrypt hash (`passlib`)                                  |
| `role`            | `String(50)`   | yes      | default `student`; one of `student`/`instructor`/`admin` |
| `avatar_url`      | `String(1024)` | no       |                                                          |
| `bio`             | `Text`         | no       |                                                          |
| `is_active`       | `Boolean`      | yes      | default `true`                                           |
| `is_verified`     | `Boolean`      | yes      | default `false`                                          |
| `preferences`     | `Text`         | no       | JSON-encoded `LearnerPreferences` (see §6.2)             |

Relationships: `authored_courses`, `enrollments`, `submissions`,
`tutoring_sessions`, `achievements`.

### 2.3 `courses`

Top-level course container (`metis.models.course.Course`). The `slug` field
provides a human-readable, URL-safe identifier for courses. The `status` column
implements the three-state lifecycle (`draft` → `published` → `archived`) that
controls which courses appear in public listings.

| Column              | Type              | Required | Notes                                                             |
| ------------------- | ----------------- | -------- | ----------------------------------------------------------------- |
| `title`             | `String(500)`     | yes      | indexed                                                           |
| `slug`              | `String(500)`     | yes      | unique, indexed                                                   |
| `description`       | `Text`            | no       |                                                                   |
| `short_description` | `String(500)`     | no       |                                                                   |
| `category`          | `String(100)`     | yes      | default `general`, indexed                                        |
| `difficulty`        | `String(50)`      | yes      | default `beginner`; `beginner`/`intermediate`/`advanced`/`expert` |
| `status`            | `String(50)`      | yes      | default `draft`; `draft`/`published`/`archived`                   |
| `thumbnail_url`     | `String(1024)`    | no       |                                                                   |
| `tags`              | `Text`            | no       | comma-separated tag list                                          |
| `estimated_hours`   | `Float`           | no       |                                                                   |
| `is_featured`       | `Boolean`         | yes      | default `false`                                                   |
| `enrollment_count`  | `Integer`         | yes      | default `0`                                                       |
| `average_rating`    | `Float`           | no       |                                                                   |
| `author_id`         | `UUID` FK→`users` | yes      | `ON DELETE CASCADE`, indexed                                      |

Relationships: `author`, `modules` (ordered by `Module.order`), `assessments`,
`enrollments`. Modules/assessments/enrollments cascade-delete with the course.

### 2.4 `modules`

Ordered section within a course (`metis.models.course.Module`). Modules exist to
give instructors a way to group related lessons and publish them incrementally —
a module can be unpublished while its lessons are being drafted.

| Column              | Type                | Required | Notes                        |
| ------------------- | ------------------- | -------- | ---------------------------- |
| `title`             | `String(500)`       | yes      |                              |
| `description`       | `Text`              | no       |                              |
| `order`             | `Integer`           | yes      | default `0`                  |
| `is_published`      | `Boolean`           | yes      | default `false`              |
| `estimated_minutes` | `Integer`           | no       |                              |
| `course_id`         | `UUID` FK→`courses` | yes      | `ON DELETE CASCADE`, indexed |

Relationship: `lessons` (ordered by `Lesson.order`, cascade delete).

### 2.5 `lessons`

Atomic learning unit within a module (`metis.models.course.Lesson`). The
`content_type` column drives how the `content` field is interpreted by the
client: markdown text for `text` type, a video URL for `video`, and JSON for
`interactive`, `quiz`, and `code_exercise` types. The `is_free_preview` flag
allows a subset of lessons to be publicly visible before enrollment, which is
used for marketing and learner onboarding.

| Column             | Type                | Required | Notes                                                               |
| ------------------ | ------------------- | -------- | ------------------------------------------------------------------- |
| `title`            | `String(500)`       | yes      |                                                                     |
| `description`      | `Text`              | no       |                                                                     |
| `content_type`     | `String(50)`        | yes      | default `text`; `text`/`video`/`interactive`/`quiz`/`code_exercise` |
| `content`          | `Text`              | no       | markdown, video URL, or JSON per type                               |
| `order`            | `Integer`           | yes      | default `0`                                                         |
| `duration_minutes` | `Integer`           | no       |                                                                     |
| `is_published`     | `Boolean`           | yes      | default `false`                                                     |
| `is_free_preview`  | `Boolean`           | yes      | default `false`; available without enrollment                       |
| `resource_urls`    | `Text`              | no       | JSON array of supplementary resource URLs                           |
| `module_id`        | `UUID` FK→`modules` | yes      | `ON DELETE CASCADE`, indexed                                        |

The content hierarchy is **three levels: Course → Module → Lesson.** There are
no `Section` or `ContentBlock` tables.

### 2.6 `assessments`

Evaluation instrument attached to a course
(`metis.models.assessment.Assessment`). The `max_attempts` column set to `0`
means unlimited attempts, which is the correct default for `practice` type
assessments used for self-study.

| Column                 | Type                | Required | Notes                                                          |
| ---------------------- | ------------------- | -------- | -------------------------------------------------------------- |
| `title`                | `String(500)`       | yes      |                                                                |
| `description`          | `Text`              | no       |                                                                |
| `assessment_type`      | `String(50)`        | yes      | default `quiz`; `quiz`/`exam`/`assignment`/`practice`, indexed |
| `duration_minutes`     | `Integer`           | no       | time limit; NULL = untimed                                     |
| `passing_score`        | `Float`             | yes      | default `70.0`; percentage 0–100                               |
| `max_attempts`         | `Integer`           | yes      | default `3`; `0` = unlimited                                   |
| `is_published`         | `Boolean`           | yes      | default `false`                                                |
| `shuffle_questions`    | `Boolean`           | yes      | default `false`                                                |
| `show_correct_answers` | `Boolean`           | yes      | default `true`                                                 |
| `available_from`       | `DateTime(tz)`      | no       |                                                                |
| `available_until`      | `DateTime(tz)`      | no       |                                                                |
| `course_id`            | `UUID` FK→`courses` | yes      | `ON DELETE CASCADE`, indexed                                   |

Relationships: `questions` (ordered by `Question.order`), `submissions`.

### 2.7 `questions`

Single question within an assessment (`metis.models.assessment.Question`).
`correct_answer` is stored as JSON and is `NULL` for essay and code questions,
which are left to instructor manual grading.

| Column           | Type                    | Required | Notes                                                                                   |
| ---------------- | ----------------------- | -------- | --------------------------------------------------------------------------------------- |
| `content`        | `Text`                  | yes      | question prompt (markdown)                                                              |
| `question_type`  | `String(50)`            | yes      | default `multiple_choice`; `multiple_choice`/`true_false`/`short_answer`/`essay`/`code` |
| `options`        | `Text`                  | no       | JSON array of options                                                                   |
| `correct_answer` | `Text`                  | no       | JSON-encoded; NULL for manually-graded types                                            |
| `explanation`    | `Text`                  | no       | shown after answering                                                                   |
| `points`         | `Float`                 | yes      | default `1.0`                                                                           |
| `order`          | `Integer`               | yes      | default `0`                                                                             |
| `difficulty`     | `String(50)`            | yes      | default `medium`; `easy`/`medium`/`hard`                                                |
| `hint`           | `Text`                  | no       |                                                                                         |
| `assessment_id`  | `UUID` FK→`assessments` | yes      | `ON DELETE CASCADE`, indexed                                                            |

### 2.8 `submissions`

A student's attempt at an assessment (`metis.models.assessment.Submission`). The
`academic_integrity_verdict_json` column stores a full
`AcademicIntegrityVerdict` from the Themis domain as a JSON string, enabling
after-the-fact audits without requiring Metis to join against a remote Themis
database.

| Column                            | Type                    | Required | Notes                                                  |
| --------------------------------- | ----------------------- | -------- | ------------------------------------------------------ |
| `answers`                         | `Text`                  | yes      | JSON map of `question_id` → answer                     |
| `score`                           | `Float`                 | no       | percentage 0–100; NULL until graded                    |
| `points_earned`                   | `Float`                 | no       |                                                        |
| `points_possible`                 | `Float`                 | no       |                                                        |
| `passed`                          | `Boolean`               | no       | meets passing threshold                                |
| `attempt_number`                  | `Integer`               | yes      | default `1`                                            |
| `started_at`                      | `DateTime(tz)`          | no       |                                                        |
| `submitted_at`                    | `DateTime(tz)`          | no       |                                                        |
| `graded_at`                       | `DateTime(tz)`          | no       |                                                        |
| `grader_notes`                    | `Text`                  | no       | instructor manual-grading notes                        |
| `academic_integrity_verdict_json` | `Text`                  | no       | JSON `AcademicIntegrityVerdict` from Themis (see §7.4) |
| `is_auto_graded`                  | `Boolean`               | yes      | default `true`                                         |
| `assessment_id`                   | `UUID` FK→`assessments` | yes      | `ON DELETE CASCADE`, indexed                           |
| `user_id`                         | `UUID` FK→`users`       | yes      | `ON DELETE CASCADE`, indexed                           |

### 2.9 `tutoring_sessions`

Tutoring conversation (`metis.models.tutoring.TutoringSession`). The optional
`course_id` and `lesson_id` foreign keys allow a tutoring session to be
contextualised within the content hierarchy — the AI tutor can reference the
specific lesson the learner is working on — but they are nullable so that
open-ended tutoring sessions unrelated to any course are also supported.

| Column                | Type                | Required | Notes                                                                   |
| --------------------- | ------------------- | -------- | ----------------------------------------------------------------------- |
| `topic`               | `String(500)`       | yes      |                                                                         |
| `status`              | `String(50)`        | yes      | default `active`; `active`/`completed`/`abandoned`/`escalated`, indexed |
| `tutor_type`          | `String(50)`        | yes      | default `ai`; `ai`/`human`/`hybrid`                                     |
| `ai_model`            | `String(100)`       | no       | model used (e.g. `gpt-4o`, `claude-sonnet-4-20250514`)                  |
| `summary`             | `Text`              | no       | AI-generated post-session summary                                       |
| `satisfaction_rating` | `Integer`           | no       | 1–5                                                                     |
| `tokens_used`         | `Integer`           | yes      | default `0`                                                             |
| `estimated_cost`      | `Float`             | yes      | default `0.0`; USD                                                      |
| `started_at`          | `DateTime(tz)`      | no       |                                                                         |
| `ended_at`            | `DateTime(tz)`      | no       |                                                                         |
| `course_id`           | `UUID` FK→`courses` | no       | `ON DELETE SET NULL`, indexed                                           |
| `lesson_id`           | `UUID` FK→`lessons` | no       | `ON DELETE SET NULL`                                                    |
| `user_id`             | `UUID` FK→`users`   | yes      | `ON DELETE CASCADE`, indexed                                            |

Relationship: `messages` (ordered by `created_at`).

### 2.10 `tutoring_messages`

One message in a tutoring session (`metis.models.tutoring.TutoringMessage`). The
`sender_type` distinguishes who produced the message, which is important for
replay and moderation: a `human_tutor` message must be preserved as-is during an
escalation handoff, while an `ai` message may be subject to quality evaluation.

| Column          | Type                          | Required | Notes                                                         |
| --------------- | ----------------------------- | -------- | ------------------------------------------------------------- |
| `content`       | `Text`                        | yes      | markdown                                                      |
| `sender_type`   | `String(50)`                  | yes      | `student`/`ai`/`system`/`human_tutor`                         |
| `sender_name`   | `String(255)`                 | no       |                                                               |
| `message_type`  | `String(50)`                  | yes      | default `text`; `text`/`code`/`image`/`suggestion`/`feedback` |
| `tokens_used`   | `Integer`                     | yes      | default `0`                                                   |
| `model_used`    | `String(100)`                 | no       |                                                               |
| `metadata_json` | `Text`                        | no       | JSON metadata                                                 |
| `session_id`    | `UUID` FK→`tutoring_sessions` | yes      | `ON DELETE CASCADE`, indexed                                  |

### 2.11 `enrollments`

User ↔ course binding (`metis.models.progress.Enrollment`). The unique
constraint `uq_enrollment_user_course` prevents a learner from enrolling in the
same course twice, which would produce ambiguous progress records.

| Column             | Type                | Required | Notes                                                              |
| ------------------ | ------------------- | -------- | ------------------------------------------------------------------ |
| `status`           | `String(50)`        | yes      | default `active`; `active`/`completed`/`dropped`/`paused`, indexed |
| `progress_percent` | `Float`             | yes      | default `0.0`; overall completion 0–100                            |
| `enrolled_at`      | `DateTime(tz)`      | yes      |                                                                    |
| `completed_at`     | `DateTime(tz)`      | no       |                                                                    |
| `last_accessed_at` | `DateTime(tz)`      | no       |                                                                    |
| `certificate_url`  | `String(1024)`      | no       | S3 URL of completion certificate PDF                               |
| `rating`           | `Integer`           | no       | 1–5                                                                |
| `review`           | `Text`              | no       |                                                                    |
| `user_id`          | `UUID` FK→`users`   | yes      | `ON DELETE CASCADE`, indexed                                       |
| `course_id`        | `UUID` FK→`courses` | yes      | `ON DELETE CASCADE`, indexed                                       |

Unique constraint `uq_enrollment_user_course` on (`user_id`, `course_id`).

### 2.12 `progress`

Per-lesson completion within an enrollment (`metis.models.progress.Progress`).
`last_position` stores an opaque bookmark string — a video timestamp, scroll
offset, or similar — that clients interpret. The unique constraint ensures
exactly one progress row exists per (enrollment, lesson) pair.

| Column               | Type                    | Required | Notes                              |
| -------------------- | ----------------------- | -------- | ---------------------------------- |
| `completed`          | `Boolean`               | yes      | default `false`                    |
| `completed_at`       | `DateTime(tz)`          | no       |                                    |
| `time_spent_seconds` | `Integer`               | yes      | default `0`; cumulative            |
| `last_position`      | `String(255)`           | no       | bookmark (video timestamp, scroll) |
| `notes`              | `Text`                  | no       | learner's private notes            |
| `enrollment_id`      | `UUID` FK→`enrollments` | yes      | `ON DELETE CASCADE`, indexed       |
| `lesson_id`          | `UUID` FK→`lessons`     | yes      | `ON DELETE CASCADE`, indexed       |

Unique constraint `uq_progress_enrollment_lesson` on (`enrollment_id`,
`lesson_id`).

### 2.13 `achievements`

Gamification achievement (`metis.models.progress.Achievement`). The unique
constraint `uq_achievement_user_type_name` ensures each achievement can only be
earned once — a learner cannot accumulate duplicate "course completion" badges
for the same course.

| Column             | Type              | Required | Notes                                                                                |
| ------------------ | ----------------- | -------- | ------------------------------------------------------------------------------------ |
| `achievement_type` | `String(100)`     | yes      | `course_completion`/`streak`/`perfect_score`/`first_enrollment`/`milestone`, indexed |
| `name`             | `String(255)`     | yes      |                                                                                      |
| `description`      | `Text`            | no       |                                                                                      |
| `icon_url`         | `String(1024)`    | no       |                                                                                      |
| `points`           | `Integer`         | yes      | default `10`                                                                         |
| `earned_at`        | `DateTime(tz)`    | yes      |                                                                                      |
| `metadata_json`    | `Text`            | no       | JSON additional context                                                              |
| `user_id`          | `UUID` FK→`users` | yes      | `ON DELETE CASCADE`, indexed                                                         |

Unique constraint `uq_achievement_user_type_name` on (`user_id`,
`achievement_type`, `name`).

### 2.14 File-Backed Persistence

Several feature areas — source ingestion, lecture packages, credentials, and
agent-runtime artifacts — produce large JSON documents that are better stored in
object storage rather than relational tables. Relational tables are optimised
for querying and joining; these documents are written once, read by ID, and
rarely queried by field values.

`SourceIngestionService` writes to a configurable `source_ingestion_storage_dir`
(default `/tmp/metis-source-ingestion`) and can issue presigned S3 upload URLs
against the `metis-uploads` bucket.

---

## 3. API Surface

All routers are aggregated in `metis.api.__init__.api_router` and mounted at
`/api`. Two unprefixed infrastructure endpoints exist on the app root and are
intentionally outside the `/api` prefix so that health checks can be invoked
without API authentication:

| Method | Path      | Purpose                                            |
| ------ | --------- | -------------------------------------------------- |
| GET    | `/health` | Liveness; returns service name/version/environment |
| GET    | `/ready`  | Readiness; checks PostgreSQL + Redis connectivity  |

The sixteen domain routers and their prefixes are:

| Prefix                  | Router module             | Tag                |
| ----------------------- | ------------------------- | ------------------ |
| `/api/auth`             | `api/auth.py`             | `auth`             |
| `/api/courses`          | `api/courses.py`          | `courses`          |
| `/api/assessments`      | `api/assessments.py`      | `assessments`      |
| `/api/tutoring`         | `api/tutoring.py`         | `tutoring`         |
| `/api/analytics`        | `api/analytics.py`        | `analytics`        |
| `/api/admin`            | `api/admin.py`            | `admin`            |
| `/api/curriculum`       | `api/curriculum.py`       | `curriculum`       |
| `/api/source-ingestion` | `api/source_ingestion.py` | `source-ingestion` |
| `/api/credentials`      | `api/credentials.py`      | `credentials`      |
| `/api/agent-runtime`    | `api/agent_runtime.py`    | `agent-runtime`    |
| `/api/lti`              | `api/lti.py`              | `lti`              |
| `/api/caliper`          | `api/caliper.py`          | `caliper`          |
| `/api/qti`              | `api/qti.py`              | `qti`              |
| `/api/scorm`            | `api/scorm.py`            | `scorm`            |
| `/api/xapi`             | `api/xapi.py`             | `xapi`             |
| `/api/oneroster`        | `api/oneroster.py`        | `oneroster`        |

### 3.1 Authentication — `/api/auth`

These five endpoints handle the complete authentication lifecycle. Registration
and login both return a token pair; subsequent requests use the `access_token`
as a bearer header. The refresh endpoint lets clients extend their session
without re-entering credentials.

| Method | Path        | Auth   | Purpose                                      |
| ------ | ----------- | ------ | -------------------------------------------- |
| POST   | `/register` | none   | Register a user; returns a token pair (201)  |
| POST   | `/login`    | none   | Authenticate; returns a token pair           |
| POST   | `/refresh`  | none   | Exchange a refresh token for a new pair      |
| GET    | `/me`       | bearer | Current user profile with parsed preferences |
| PATCH  | `/me`       | bearer | Update name/bio/avatar/preferences           |

### 3.2 Courses — `/api/courses`

The courses router is the core of the Metis content hierarchy. Public listing
and detail endpoints require no authentication so that unenrolled visitors can
browse the catalog; mutation endpoints require the instructor role.

| Method | Path                                                   | Auth       | Purpose                                          |
| ------ | ------------------------------------------------------ | ---------- | ------------------------------------------------ |
| GET    | `` (root)                                              | none       | List/filter/search/paginate courses              |
| POST   | `` (root)                                              | instructor | Create a course (201)                            |
| GET    | `/{course_id}`                                         | none       | Get a course with nested modules + concept graph |
| PATCH  | `/{course_id}`                                         | instructor | Update course fields                             |
| DELETE | `/{course_id}`                                         | instructor | Delete a course (204)                            |
| GET    | `/{course_id}/concept-graph`                           | none       | Learner-facing course-linked concept graph       |
| POST   | `/{course_id}/lecture-package`                         | instructor | Generate a prerecorded lecture package (201)     |
| GET    | `/{course_id}/lecture-package`                         | bearer     | Latest lecture package                           |
| GET    | `/{course_id}/lecture-package/assets/{asset_path}`     | bearer     | Fetch a rendered lecture asset file              |
| POST   | `/{course_id}/lecture-package/evaluation`              | instructor | Evaluate lecture media quality                   |
| GET    | `/{course_id}/lecture-package/evaluation`              | instructor | Latest lecture media evaluation                  |
| POST   | `/{course_id}/modules`                                 | instructor | Add a module (201)                               |
| PATCH  | `/{course_id}/modules/{module_id}`                     | instructor | Update a module                                  |
| DELETE | `/{course_id}/modules/{module_id}`                     | instructor | Delete a module (204)                            |
| POST   | `/{course_id}/modules/{module_id}/lessons`             | instructor | Add a lesson (201)                               |
| PATCH  | `/{course_id}/modules/{module_id}/lessons/{lesson_id}` | instructor | Update a lesson                                  |
| DELETE | `/{course_id}/modules/{module_id}/lessons/{lesson_id}` | instructor | Delete a lesson (204)                            |
| POST   | `/{course_id}/enroll`                                  | bearer     | Enroll the current user (201)                    |
| POST   | `/{course_id}/progress/{lesson_id}`                    | bearer     | Update lesson progress                           |

### 3.3 Assessments — `/api/assessments`

Assessment endpoints mirror the course pattern: public read, instructor-only
mutation, and learner-scoped submission. The calibration and fairness endpoints
are further restricted to the course author or an admin because psychometric
data about individual learners must not leak to other students.

| Method | Path                                         | Auth         | Purpose                                         |
| ------ | -------------------------------------------- | ------------ | ----------------------------------------------- |
| GET    | `` (root)                                    | none         | List assessments (filter course/type/published) |
| POST   | `` (root)                                    | instructor   | Create an assessment (201)                      |
| GET    | `/{assessment_id}`                           | none         | Get an assessment with nested questions         |
| PATCH  | `/{assessment_id}`                           | instructor   | Update an assessment                            |
| DELETE | `/{assessment_id}`                           | instructor   | Delete an assessment (204)                      |
| POST   | `/{assessment_id}/questions`                 | instructor   | Add a question (201)                            |
| PATCH  | `/{assessment_id}/questions/{question_id}`   | instructor   | Update a question                               |
| DELETE | `/{assessment_id}/questions/{question_id}`   | instructor   | Delete a question (204)                         |
| POST   | `/{assessment_id}/submit`                    | bearer       | Submit answers; auto-grades (201)               |
| GET    | `/{assessment_id}/submissions`               | bearer       | Current user's submissions                      |
| GET    | `/{assessment_id}/calibration`               | author/admin | Item-calibration + mastery-threshold report     |
| POST   | `/{assessment_id}/calibration/recompute`     | author/admin | Recompute calibration from live data            |
| GET    | `/{assessment_id}/fairness-review`           | author/admin | Fairness + DIF review                           |
| POST   | `/{assessment_id}/fairness-review/recompute` | author/admin | Recompute fairness/DIF review                   |

### 3.4 Tutoring — `/api/tutoring`

All tutoring endpoints require at least bearer authentication because tutoring
sessions are private to the learner. The voice-runtime endpoints
(`voice-runtime`, `voice-runtime/events`, `voice-runtime/resume`) interact with
the Psyche live- voice runtime and carry additional Psyche-specific payload
fields.

| Method | Path                                          | Auth   | Purpose                                       |
| ------ | --------------------------------------------- | ------ | --------------------------------------------- |
| GET    | `/sessions`                                   | bearer | List the user's sessions (filter status)      |
| POST   | `/sessions`                                   | bearer | Start a session (201)                         |
| GET    | `/sessions/{session_id}`                      | bearer | Get a session with messages                   |
| PATCH  | `/sessions/{session_id}`                      | bearer | Update status/rating/summary                  |
| POST   | `/sessions/{session_id}/messages`             | bearer | Send a student message (201)                  |
| POST   | `/sessions/{session_id}/generate-response`    | bearer | Generate an AI tutor reply (201)              |
| POST   | `/sessions/{session_id}/voice-runtime`        | bearer | Provision/refresh a live voice bridge         |
| POST   | `/sessions/{session_id}/voice-runtime/events` | bearer | Record turn-taking/interruption events        |
| POST   | `/sessions/{session_id}/voice-runtime/resume` | bearer | Resume a paused/interrupted voice runtime     |
| POST   | `/sessions/{session_id}/delivery-health`      | bearer | Refresh delivery fallback from health signals |
| POST   | `/sessions/{session_id}/quality-evaluation`   | bearer | Evaluate live tutoring embodiment quality     |
| GET    | `/sessions/{session_id}/quality-evaluation`   | bearer | Latest tutoring quality evaluation            |

### 3.5 Analytics — `/api/analytics`

Analytics endpoints operate on different scopes: the learner dashboard and
reminders are scoped to the authenticated user; course analytics are scoped to
the course author or admin; platform analytics are admin-only. Redis caching is
applied to the two most expensive aggregations (dashboard and platform).

| Method | Path                  | Auth         | Purpose                                        |
| ------ | --------------------- | ------------ | ---------------------------------------------- |
| GET    | `/dashboard`          | bearer       | Personalized learner dashboard (Redis-cached)  |
| GET    | `/reminders`          | bearer       | Learner reminders / continuation notifications |
| GET    | `/achievements`       | bearer       | The user's achievements                        |
| GET    | `/course/{course_id}` | author/admin | Per-course analytics (enrollment, completion)  |
| GET    | `/platform`           | admin        | Platform-wide analytics (Redis-cached)         |

### 3.6 Admin — `/api/admin`

The admin router contains 30 operations, all requiring the `admin` role. It
covers user management, course approval, content moderation, review queues,
incident and complaint workspaces, monitoring and alerting, and source-rights
decisions. Operations include: `/dashboard`, `/analytics`,
`/metis-v1-launch-gate`, `/audit-trail`, `/incidents/workspace`,
`/complaints/workspace`, `/complaints/{id}/action`, `/users` (list/detail),
`/users/{id}/status` (PATCH), `/users/{id}/role` (PATCH),
`/users/{id}/learner-actions` (POST), `/courses` (list/detail),
`/courses/{id}/approval` (POST), `/content/flags` and
`/content/flags/{id}/moderate`, `/moderation/incidents` and
`/moderation/incidents/{id}/decision`, `/monitoring`, `/operator-inspections`,
`/alerts` and `/alerts/{id}` (PATCH), `/review-queues` (overview / list / detail
/ decision / workflow-action), and `/source-rights` (workspace / decision).

### 3.7 Curriculum — `/api/curriculum`

Read-mostly curriculum-discovery endpoints backed by in-code catalogs. Because
the curriculum taxonomy is versioned and edited deliberately (not generated at
runtime), these endpoints serve static or near-static data and do not need
database writes: `/subject-taxonomy` and `/subject-taxonomy/{subject_id}`,
`/subject-taxonomy/lookup` (POST), `/subject-routes` (POST), `/seed-packs`
(+`/{discipline_id}`), `/standards-mappings` (+`/{discipline_id}`),
`/reviewer-pools` (+`/{discipline_id}`), `/gold-sets` (+
`/release-gate-dashboard`, +`/{discipline_id}`), `/cross-core-interlocks`
(+`/{interlock_id}`), `/safety-policies`
(+`/{discipline_id}`, +`/{discipline_id}/evaluate` POST).

### 3.8 Source Ingestion — `/api/source-ingestion`

The source-ingestion router exposes the full lifecycle of a source package —
upload, extract, review, generate, maintain. Each stage builds on the previous
one; generation endpoints reject requests until the package has been extracted
and approved. The full endpoint set: `/uploads` (POST presigned targets),
package CRUD
(``POST/GET,`/{package_id}`GET),`/{package_id}/extract`(POST/GET),`/{package_id}/review`(POST),`/{package_id}/evaluate`(POST),`/{package_id}/outline`(POST/GET),`/{package_id}/generate`(POST/GET),`/{package_id}/concepts/graph`(POST/GET),`/concepts/explore`(POST),`/concepts/graph/validate`(POST),`/concepts/graph/repair`(POST),`/{package_id}/refresh`(POST/GET),`/{package_id}/retrieval-benchmark`(POST/GET),`/{package_id}/connectors/preview`(POST/GET),`/study-workspace`(POST),`/retrieval/algorithm-docs`
(GET).

### 3.9 Credentials — `/api/credentials`

Credential endpoints issue, verify, and revoke Open Badges 3.0 and CLR
documents. The `/verify` POST endpoint accepts a full credential document body,
which enables QR-code scan flows where the verifier has the credential but no
persistent URL: `/badges/issue` (POST, 201), `/badges` (GET list),
`/badges/{credential_id}` (GET), `/badges/{credential_id}/verify` (GET),
`/badges/{credential_id}/revoke` (POST), `/verify` (POST a presented
credential), `/clr/{learner_id}` (GET).

### 3.10 Agent Runtime — `/api/agent-runtime`

All agent-runtime endpoints require the instructor role. The router manages the
full lifecycle of durable agent orchestration plans, from registration and
scheduling through evaluation and safety screening: `/registry` (GET),
`/orchestration-plans` (POST/GET by `build_id`), `/events` (GET),
`/graph-indexes` (POST/GET), `/output-verifications` (POST/GET), `/rollouts`
(POST/GET), `/evaluation/gold-sets` (GET), `/evaluation/rubric-scores`
(POST/GET), `/safety-screenings` (POST/GET), `/research-integrity/adjudications`
(POST/GET).

### 3.11 LMS Interoperability

These six routers implement the standard protocols that allow Metis to
interoperate with institutional LMS platforms. The auth pattern differs by
protocol — LTI uses signed JWTs, SCORM uses token-protected runtime state, xAPI
and Caliper use learner-scoped bearer tokens.

| Prefix           | Endpoints                                                                                                                                                                                                           |
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `/api/lti`       | LTI 1.3 / Advantage: `/platforms` (POST), `/launch` (POST), `/token` (POST), `/deep-linking/response` (POST), `/names-and-roles/{context_id}` (GET), `/assignments/line-items` (POST), `/assignments/scores` (POST) |
| `/api/qti`       | QTI 3 item bank: `/item-bank/import` (POST), `/item-bank/export` (POST)                                                                                                                                             |
| `/api/scorm`     | SCORM 1.2/2004 fallback: `/packages` (POST), `/packages/{id}/manifest` (GET), `/runtime/commit` (POST), `/runtime/{package_id}/attempts/{learner_id}` (GET)                                                         |
| `/api/xapi`      | xAPI / cmi5: `/statements` (POST/GET), `/replay` (GET)                                                                                                                                                              |
| `/api/caliper`   | IMS Caliper: `/events` (POST/GET), `/export` (GET)                                                                                                                                                                  |
| `/api/oneroster` | OneRoster: `/sync` (POST, admin) — dry-run or apply a rostering payload                                                                                                                                             |

---

## 4. Authentication & Authorization

Authentication is defined in `metis.deps`. A new engineer should read this
section alongside the API tables above: every endpoint annotated `bearer` uses
`get_current_user`, `instructor` uses `get_current_instructor_user`, and `admin`
uses `get_current_admin_user`.

- **Scheme:** HTTP bearer token. JWT decoding uses `python-jose`; the algorithm
  and secret come from settings (`jwt_algorithm` default `HS256`).
- **Token claims read:** `sub` (user id, required), `email`, `role`, `name`,
  `exp`. Missing `sub` or an expired `exp` yields HTTP 401.
- **`CurrentUser`:** lightweight object with `id`, `email`, `role`, `name` and
  helpers `is_admin()` (role `admin`) and `is_instructor()` (role `instructor`
  or `admin`).
- **Role dependencies:** `get_current_user` (any authenticated),
  `get_current_instructor_user` (instructor/admin, else 403),
  `get_current_admin_user` (admin only, else 403).
- **Token issuance:** `AuthService` issues an access token
  (`jwt_access_token_expire_minutes`, default 30) and refresh token
  (`jwt_refresh_token_expire_days`, default 7). `TokenResponse` returns
  `access_token`, `refresh_token`, `token_type` (`bearer`), and `expires_in`.
- **Resource-level checks:** several endpoints additionally enforce ownership
  (e.g. tutoring sessions are accessible only by the owning user; assessment
  calibration/fairness require the course author or an admin;
  credentials/xAPI/Caliper restrict learner scope to the caller unless the
  caller is an instructor).

---

## 5. Background Tasks (Celery)

Celery configuration is built by `Settings.get_celery_config()` (`metis.config`)
and applied in `metis.tasks.celery_app`. All task modules and their queue
assignments are declared in `task_routes` so that the worker can be scaled
per-queue independently — for example, running more `metis.export` workers
during a large course-export batch without adding capacity to the `metis.ai`
queue.

### 5.1 Queues

`task_routes` map task modules onto named queues: `metis.export`,
`metis.notifications`, `metis.analytics`, `metis.cleanup`, `metis.ai`. The
default queue is `metis.default`. `task_default_retry_delay` is 60s and
`task_max_retries` is 3. The `metis-worker` container consumes all six queues.

### 5.2 Periodic Tasks (`beat_schedule`)

These three tasks run on a fixed schedule driven by `metis-beat` and require no
HTTP trigger. The hourly analytics aggregation keeps the Redis-cached dashboard
data fresh; the daily cleanup tasks prevent orphaned data from accumulating.

| Task                                           | Interval        |
| ---------------------------------------------- | --------------- |
| `metis.tasks.analytics.aggregate_analytics`    | hourly (3600 s) |
| `metis.tasks.cleanup.cleanup_expired_sessions` | daily (86400 s) |
| `metis.tasks.cleanup.cleanup_orphaned_uploads` | daily (86400 s) |

### 5.3 Task Modules

Each task module is routed to its own named queue to allow independent scaling.
The descriptions below summarize the work each module performs:

- `tasks/export.py` — `export_course` exports a full course to S3 in `json` or
  `scorm` format (unknown formats fall back to JSON).
- `tasks/notifications.py` — `send_notification` delivers via `email` (SMTP →
  Mailpit in dev), `push`, or `inapp` channels; plus `send_enrollment_welcome`
  and `send_grade_notification`.
- `tasks/analytics.py` — `aggregate_analytics`.
- `tasks/cleanup.py` — `cleanup_expired_sessions`, `cleanup_orphaned_uploads`,
  `cleanup_expired_cache`.
- `tasks/celery_app.py` — Celery app construction and prerun/postrun/failure
  signal handlers, including a dead-letter routing handler for permanently
  failed tasks.

---

## 6. Key Domain Objects & Enumerations (Pydantic Schemas)

Pydantic schemas live in `services/metis/src/metis/schemas/`. There are 511
`BaseModel` classes across the package — the full set of request bodies,
response shapes, and domain value objects. The subsections below document the
most important enumerations and structural types. Field-level validation is
expressed via Pydantic `Field` constraints and regex patterns.

### 6.1 Validation Patterns (request schemas)

Every request schema enforces domain constraints at the HTTP boundary using
Pydantic `Field` constraints and regex patterns, so invalid values are rejected
before any service code runs. The table below lists the most important
constraints:

| Schema field                         | Constraint                         |
| ------------------------------------ | ---------------------------------- | ------------- | ------------ | ----------- | ---------------- |
| `CourseCreate.slug`                  | regex `^[a-z0-9]+(?:-[a-z0-9]+)*$` |
| `CourseCreate.difficulty`            | regex `^(beginner                  | intermediate  | advanced     | expert)$`   |
| `CourseUpdate.status`                | regex `^(draft                     | published     | archived)$`  |
| `LessonCreate.content_type`          | regex `^(text                      | video         | interactive  | quiz        | code_exercise)$` |
| `AssessmentCreate.assessment_type`   | regex `^(quiz                      | exam          | assignment   | practice)$` |
| `AssessmentCreate.passing_score`     | `ge=0, le=100`                     |
| `QuestionCreate.question_type`       | regex `^(multiple_choice           | true_false    | short_answer | essay       | code)$`          |
| `QuestionCreate.difficulty`          | regex `^(easy                      | medium        | hard)$`      |
| `RegisterRequest.password`           | `min_length=8, max_length=128`     |
| `RegisterRequest.role`               | regex `^(student                   | instructor)$` |
| `EnrollmentUpdate.status`            | regex `^(active                    | completed     | dropped      | paused)$`   |
| `EnrollmentUpdate.rating`            | `ge=1, le=5`                       |
| `TutoringSessionCreate.tutor_type`   | regex `^(ai                        | human         | hybrid)$`    |
| `TutoringMessageCreate.message_type` | regex `^(text                      | code          | image        | suggestion  | feedback)$`      |

### 6.2 `LearnerPreferences` (`schemas/user.py`)

`LearnerPreferences` is serialized to JSON and stored in `users.preferences`. It
captures every dimension of how a learner wants to be taught and supervised,
from learning pace to oversight context. The `model_post_init` validator
enforces cross-field invariants: for example, the `independent` oversight
context automatically clears all oversight roles and sharing flags so a learner
cannot accidentally appear supervised.

The key typed unions within `LearnerPreferences` are:

- `LearnerStudyGoal` — `foundational-mastery`, `exam-prep`,
  `career-advancement`, `project-delivery`, `curiosity`
- `LearnerPace` — `light`, `steady`, `accelerated`
- `LearnerContentType` — `video`, `reading`, `exercise`, `quiz`, `tutoring`
- `LearnerTutorStyle` — `step-by-step`, `socratic`, `direct`,
  `direct-explanation`, `scaffolded-hints`, `exam-practice`, `mentor-coaching`
- `LearnerExplanationDepth` — `concise`, `balanced`, `deep`
- `LearnerOversightContext` — `independent`, `minor-supervised`,
  `managed-program`
- `LearnerOversightRole` — `teacher`, `guardian`, `institution`
- `LearnerReminderDay` — `monday`…`sunday`

Bounded numeric fields: `weekly_hours_target` (1–40), `session_minutes`
(10–180), `study_reminder_hour_local` (0–23), `assessment_reminder_lead_hours`
(1–168). Field validators normalize aliases and clamp list lengths.

### 6.3 Tutoring Delivery & Live Voice (`schemas/tutoring.py`)

The tutoring schema layer is the most complex in Metis because it models three
different delivery channels (text, voice, avatar), a multi-stage fallback state
machine, a full voice turn-taking lifecycle, and teacher representation safety
governance — all as typed Pydantic unions.

The key literal-union types are:

- `TutoringIntegrityMode` — `teach`, `hint`, `practice`,
  `do-not-complete-for-me`
- `TutoringDeliveryMode` — `text`, `live_voice`, `live_avatar`
- `TutoringDeliveryFallbackState` — `native`, `fallback`
- `TutoringDeliveryFallbackReason` — six reasons covering avatar/voice runtime
  unavailability and latency/fidelity budget failures
- `TutoringLiveVoiceTransportState` — `not_requested`, `provisioning`, `ready`,
  `active`, `paused`, `ended`, `failed`
- `TutoringLiveVoiceTurnState` — `idle`, `learner_listening`,
  `learner_speaking`, `processing`, `tutor_speaking`, `interrupted`,
  `resume_ready`
- `TutoringLiveVoiceRuntimeEventType` — 10 runtime event types (speech
  start/stop, response start/stop, interruption, connection loss, pause/resume,
  partial/final transcript)
- `TutoringTeacherRepresentationReleaseStatus` — `draft`, `review`, `approved`,
  `restricted`, `revoked`, `retired`
- `TutoringTeacherRepresentationConsentStatus` — `not-required`, `pending`,
  `granted`, `expired`, `revoked`, `blocked`

`TutoringSessionResponse` is a rich response object carrying the session's
`delivery_fallback` (`TutoringDeliveryFallbackResponse`), `live_voice`
(`TutoringLiveVoiceRuntimeResponse`), `teacher_representation`
(`TutoringTeacherRepresentationSafetyResponse`), an append-only `consent_log`,
oversight context/roles, and optional escalation/handoff fields.

### 6.4 Source Ingestion (`schemas/source_ingestion.py`)

Source ingestion schemas model the four ways a package can be created and the
four scopes to which it can be attached. The discriminated union on
`SourceScope` uses the `level` field as the discriminator so that Pydantic can
deserialize the correct subtype without ambiguity.

- Ingestion input kinds: `FileIngestionInput`, `UrlIngestionInput`,
  `FeedIngestionInput` (`feed_format`: `rss`/`atom`/`jsonfeed`/`unknown`),
  `PackageIngestionInput` (`package_type`: `scorm`/`imscc`/`lti_export`/
  `institutional_archive`/`zip_bundle`).
- `SourceScope` is a discriminated union over `CourseSourceScope`,
  `WorkspaceSourceScope`, `NotebookSourceScope`, `LearnerSessionSourceScope`
  (discriminator `level`).
- `SourcePackageReviewStateResponse.status` — `pending`, `approved`,
  `changes_requested`, `rejected`; `risk_level` — `low`/`medium`/`high`/
  `critical`.
- `SourcePackageUsagePolicyResponse.usage` — `approved`, `review_only`,
  `blocked`. Grounded outline/generation endpoints require `high_stakes_allowed`
  and review status `approved`.

### 6.5 Concept Graph (`schemas/source_concept_graph.py`)

The concept graph schema models a directed graph of educational concepts
extracted from a source package. Nodes are typed by their epistemic role; edges
are typed by the pedagogical relationship between concepts. This structure is
used for prerequisite chains, graph-aware evidence retrieval, and learning-path
generation.

- `ConceptNodeType` — `topic`, `skill`, `fact`, `procedure`, `principle`
- `ConceptEdgeType` — `prerequisite`, `related`, `part_of`, `generalizes`,
  `specializes`, `enables`, `conflicts`, `complements`
- Validation issue `severity` — `warning`, `critical`; concept-graph repair
  operations are a bounded set of operator-approved mutations.

### 6.6 Assessment Calibration & Fairness (`schemas/assessment.py`)

These schemas surface the psychometric analysis that Metis computes from live
learner submission data. They are returned only to course authors and admins
because they contain aggregate learner statistics.

`AssessmentCalibrationResponse` reports per-question calibration
(`QuestionCalibrationResponse`: difficulty index, IRT-style difficulty,
discrimination index, sample sizes) and a `MasteryThresholdCalibrationResponse`
(current vs. recommended threshold, balanced accuracy/precision/recall/F1).
`AssessmentFairnessReviewResponse` reports DIF reviews (`ItemDIFReviewResponse`,
recommended action `monitor`/`recalibrate`/`retire`) and adaptation-parity
reviews. These are computed from live learner submission data, not from an
external psychometrics service.

### 6.7 Credentials (`schemas/credentials.py`)

Credential schemas implement the Open Badges 3.0 and IMS Global CLR standards as
Pydantic models, ensuring that every issued credential is a conformant JSON-LD
document that any standards-compliant verifier can check.

- `CredentialType` — `open_badge`, `clr`; `CredentialStatus` — `active`,
  `expired`, `revoked`; `VerificationStatus` — `valid`, `invalid`, `expired`,
  `revoked`, `not_found`.
- `CredentialProof` uses a `DataIntegrityProof` with cryptosuite
  `HMAC-SHA256-2026`, a canonical hash, and a `proof_value`.
- Issued credentials carry Open Badges 3.0 (`open_badge`) and optional CLR
  (`clr`) JSON-LD documents with IMS Global context URLs.

### 6.8 Agent Runtime (`schemas/agent_runtime.py`)

Agent runtime schemas model the lifecycle metadata of autonomous agents and the
execution plans they follow. The kill-switch fallback behaviors (`pause`,
`skip_stage`, `text_only`, `manual_review`) are the safety levers that let
operators degrade an agent plan gracefully rather than aborting it entirely.

Agent definitions carry literal-union metadata: `AgentLifecycleState`
(`active`/`experimental`/`deprecated`/`disabled`), `AgentReleaseChannel`
(`stable`/`canary`/`shadow`), `AgentCostClass`, `AgentTrustZone`
(`public`/`tenant`/`restricted`/`operator`), `AgentRoleVisibility`, and
`AgentFamily`. Orchestration plans produce a DAG (`AgentDagNodeResponse` /
`AgentDagEdgeResponse`), a replay manifest, approval gates, and kill switches
(`AgentKillSwitchResponse` with fallback behaviors `pause`/`skip_stage`/
`text_only`/`manual_review`). Safety screenings and research-integrity
adjudications return `pass`/`review`/`blocked`-style statuses.

### 6.9 Lecture Packages (`schemas/lecture.py`)

Lecture package schemas capture both the generation request (what the instructor
wants produced) and the generated result (per-lesson script, captions, slides,
render batches, and export bundles). The `subtitle_formats`, `dubbing_locales`,
and `pronunciation_overrides` fields are what enable accessible, multi-language
lecture delivery.

`LectureGenerationRequest` controls prerecorded lecture generation:
`target_locale`, `voice_pack_id`, `presentation_template`, caption/transcript
toggles, `subtitle_formats` (`vtt`/`srt`), `dubbing_locales`,
`pronunciation_overrides`, and `include_scorm_bundle`.
`CourseLecturePackageResponse` returns per-lesson packages with script sections,
caption cues, slides, diagrams, notes, narration variants, render batches, and
export bundles (with provenance and rights metadata).

---

## 7. Cross-Domain Integration

Metis integrates with four external systems. Each integration has a clear owner
of the boundary contract: Metis owns the schema mirroring and consuming-side
validation; the upstream domain owns adjudication, rendering, and transport.

### 7.1 AI Providers

`TutoringService` (`services/tutoring_service.py`) calls LLM providers directly
via their official SDKs. `Settings.ai_provider` selects `openai` or `anthropic`;
the corresponding SDK (`openai.AsyncOpenAI` / `anthropic.AsyncAnthropic`) is
used with `openai_model` (default `gpt-4o`) or `anthropic_model` (default
`claude-sonnet-4-20250514`). The `ai` extras group in `pyproject.toml` provides
`openai`, `anthropic`, `langchain`, and `chromadb`. If no provider key is
configured, tutoring falls back to a structured pedagogical response. There is
no provider-abstraction layer and no RunPod/GPU dispatch in this service.

### 7.2 Psyche (Live Voice Runtime)

Psyche owns audio transport and voice-activity detection. Metis owns the
tutoring session and the pedagogical turn-taking state machine. The live-voice
tutoring runtime is provisioned through Psyche-namespaced settings
(`psyche_voice_runtime_provider` default `psyche-openai-realtime`,
`psyche_voice_runtime_public_url`, token TTLs, latency/fidelity budgets).
Tutoring sessions can request `live_voice` / `live_avatar` delivery and degrade
to text via the `TutoringDeliveryFallback` mechanism when Psyche is unavailable
or fails to meet latency/fidelity budgets.

### 7.3 Yemaya (Lecture Rendering)

Yemaya owns GPU-bound media rendering. Metis orchestrates what to render and
labels the batches with provider `yemaya-batch-render` and pipeline
`metis-lecture-render-pipeline`. In the current service the
`LectureGenerationService` and `lecture_render_pipeline_helpers` produce
rendered slide/diagram/notes assets and write asset files to local storage; the
course lecture-asset endpoint serves those files back. The labeling convention
is the contract point: Yemaya workers watch for batches with this
provider/pipeline tag to claim and execute them.

### 7.4 Themis (Academic Integrity)

Themis owns detection and adjudication; Metis owns the learner-facing
presentation and appeal routing. Assessment submissions can carry an
`AcademicIntegrityVerdict` produced by Themis adjudication, stored as
`submissions.academic_integrity_verdict_json` and surfaced as
`SubmissionResponse.academic_integrity_verdict`. The
`AcademicIntegrityVerdictResponse` schema (`schemas/assessment.py`) mirrors the
canonical Themis contract, including detection signals, classifier outputs,
evidence excerpts, a decision with appeal path, policy binding, and governance
metadata. A `model_validator` enforces audit-completeness invariants (e.g.
violation verdicts require detection signals and evidence excerpts; severe
verdicts require human review; clear verdicts cannot recommend sanctions).
Governance fields fix `source_of_record = "themis"` and
`consuming_domain = "metis"`.

---

## 8. Configuration

All settings load from `METIS_`-prefixed environment variables via
`pydantic-settings` (`metis.config.Settings`). The sections below list every
variable with its default value. In local development all defaults work with the
`docker-compose.dev.yml` stack; in staging/production each variable must be
explicitly set.

### 8.1 Application

| Variable                    | Default                                                        |
| --------------------------- | -------------------------------------------------------------- |
| `METIS_APP_NAME`            | `metis-backend`                                                |
| `METIS_APP_VERSION`         | `0.1.0`                                                        |
| `METIS_ENVIRONMENT`         | `development` (`development`/`staging`/`production`/`testing`) |
| `METIS_DEBUG`               | `false`                                                        |
| `METIS_LOG_LEVEL`           | `INFO`                                                         |
| `METIS_HOST` / `METIS_PORT` | `0.0.0.0` / `8000`                                             |
| `METIS_ALLOWED_ORIGINS`     | localhost:3000/4200/5173                                       |

### 8.2 Database

A `postgresql://` URL is auto-rewritten to `postgresql+asyncpg://` for the async
engine. The `sync_database_url` property strips the `+asyncpg` driver suffix so
Alembic (which uses a synchronous connection) receives a compatible URL.

| Variable                      | Default                                                         |
| ----------------------------- | --------------------------------------------------------------- |
| `METIS_DATABASE_URL`          | `postgresql+asyncpg://oshun:oshun_dev@localhost:5432/oshun_dev` |
| `METIS_DATABASE_POOL_SIZE`    | `20`                                                            |
| `METIS_DATABASE_MAX_OVERFLOW` | `10`                                                            |
| `METIS_DATABASE_POOL_TIMEOUT` | `30`                                                            |
| `METIS_DATABASE_POOL_RECYCLE` | `1800`                                                          |

### 8.3 Redis

Redis serves three separate purposes in Metis: application cache (database 0),
Celery broker (database 1), and Celery result backend (database 2). Keeping them
on separate Redis logical databases prevents cache eviction from interfering
with task queue state.

| Variable                      | Default                    |
| ----------------------------- | -------------------------- |
| `METIS_REDIS_URL`             | `redis://localhost:6379/0` |
| `METIS_REDIS_PREFIX`          | `metis:`                   |
| `METIS_REDIS_DEFAULT_TTL`     | `3600`                     |
| `METIS_REDIS_SESSION_TTL`     | `86400`                    |
| `METIS_REDIS_CACHE_TTL`       | `900`                      |
| `METIS_REDIS_RATE_LIMIT_TTL`  | `60`                       |
| `METIS_REDIS_MAX_CONNECTIONS` | `20`                       |

### 8.4 Object Storage (S3 / MinIO)

Metis uses four S3 buckets, each serving a distinct purpose: course content,
rendered assets, exported packages, and staged uploads. Presigned URLs are used
for both upload staging (source ingestion) and download delivery (lecture
assets, exports).

| Variable                             | Default                       |
| ------------------------------------ | ----------------------------- |
| `METIS_S3_ENDPOINT`                  | `http://localhost:9000`       |
| `METIS_S3_ACCESS_KEY` / `SECRET_KEY` | `minioadmin` / `minioadmin`   |
| `METIS_S3_REGION`                    | `us-east-1`                   |
| `METIS_S3_BUCKET_COURSES`            | `metis-courses`               |
| `METIS_S3_BUCKET_ASSETS`             | `metis-assets`                |
| `METIS_S3_BUCKET_EXPORTS`            | `metis-exports`               |
| `METIS_S3_BUCKET_UPLOADS`            | `metis-uploads`               |
| `METIS_S3_PRESIGNED_URL_EXPIRY`      | `3600`                        |
| `METIS_SOURCE_INGESTION_STORAGE_DIR` | `/tmp/metis-source-ingestion` |

### 8.5 Authentication

The `METIS_JWT_SECRET` default is intentionally weak and must be replaced in any
non-development environment. The access token expiry of 30 minutes is
deliberately short; the 7-day refresh token allows clients to maintain sessions
without re-authentication.

| Variable                                | Default                                 |
| --------------------------------------- | --------------------------------------- |
| `METIS_JWT_SECRET`                      | `metis-dev-secret-change-in-production` |
| `METIS_JWT_ALGORITHM`                   | `HS256`                                 |
| `METIS_JWT_ACCESS_TOKEN_EXPIRE_MINUTES` | `30`                                    |
| `METIS_JWT_REFRESH_TOKEN_EXPIRE_DAYS`   | `7`                                     |

### 8.6 Celery

Task soft and hard time limits (300s / 600s) prevent runaway tasks from tying up
workers indefinitely. `worker_prefetch_multiplier = 1` ensures each worker
fetches at most one task at a time, which is important for long-running export
and AI tasks that should not starve short notification tasks.

| Variable                                  | Default                    |
| ----------------------------------------- | -------------------------- |
| `METIS_CELERY_BROKER_URL`                 | `redis://localhost:6379/1` |
| `METIS_CELERY_RESULT_BACKEND`             | `redis://localhost:6379/2` |
| `METIS_CELERY_TASK_SOFT_TIME_LIMIT`       | `300`                      |
| `METIS_CELERY_TASK_TIME_LIMIT`            | `600`                      |
| `METIS_CELERY_WORKER_PREFETCH_MULTIPLIER` | `1`                        |

### 8.7 AI Providers

`METIS_AI_PROVIDER` selects between `openai` and `anthropic`. If neither API key
is set the service starts successfully but tutoring sessions fall back to a
structured pedagogical response rather than a live LLM response.

| Variable                       | Default                    |
| ------------------------------ | -------------------------- |
| `METIS_OPENAI_API_KEY`         | (empty)                    |
| `METIS_OPENAI_MODEL`           | `gpt-4o`                   |
| `METIS_OPENAI_EMBEDDING_MODEL` | `text-embedding-3-small`   |
| `METIS_ANTHROPIC_API_KEY`      | (empty)                    |
| `METIS_ANTHROPIC_MODEL`        | `claude-sonnet-4-20250514` |
| `METIS_AI_PROVIDER`            | `openai`                   |
| `METIS_AI_MAX_TOKENS`          | `4096`                     |
| `METIS_AI_TEMPERATURE`         | `0.7`                      |

### 8.8 Live Voice Runtime

These variables configure the Psyche live-voice bridge. The latency and fidelity
budget values are the thresholds at which the delivery-fallback mechanism
triggers: if the voice runtime exceeds `METIS_PSYCHE_VOICE_LATENCY_BUDGET_MS` or
falls below `METIS_PSYCHE_VOICE_MIN_FIDELITY_SCORE`, tutoring degrades from
voice to text automatically and a learner-visible disclosure is emitted.

| Variable                                       | Default                        |
| ---------------------------------------------- | ------------------------------ |
| `METIS_PSYCHE_VOICE_RUNTIME_PROVIDER`          | `psyche-openai-realtime`       |
| `METIS_PSYCHE_VOICE_RUNTIME_PUBLIC_URL`        | `wss://voice.metis.local/live` |
| `METIS_PSYCHE_VOICE_RUNTIME_TOKEN_TTL_SECONDS` | `900`                          |
| `METIS_PSYCHE_VOICE_LATENCY_BUDGET_MS`         | `900`                          |
| `METIS_PSYCHE_VOICE_MIN_FIDELITY_SCORE`        | `0.72`                         |
| `METIS_PSYCHE_AVATAR_LATENCY_BUDGET_MS`        | `1400`                         |
| `METIS_PSYCHE_AVATAR_MIN_FIDELITY_SCORE`       | `0.8`                          |

### 8.9 Observability & Rate Limiting

OpenTelemetry is opt-in (`METIS_OTEL_ENABLED = false` by default) to keep
development startup fast. `METIS_OTEL_TRACES_SAMPLER_RATIO = 1.0` means all
traces are recorded when OTel is enabled — reduce this in high-traffic
production environments to control span volume.

| Variable                               | Default                 |
| -------------------------------------- | ----------------------- |
| `METIS_OTEL_ENABLED`                   | `false`                 |
| `METIS_OTEL_SERVICE_NAME`              | `metis-backend`         |
| `METIS_OTEL_EXPORTER_ENDPOINT`         | `http://localhost:4317` |
| `METIS_OTEL_TRACES_SAMPLER_RATIO`      | `1.0`                   |
| `METIS_RATE_LIMIT_REQUESTS_PER_MINUTE` | `60`                    |
| `METIS_RATE_LIMIT_BURST`               | `10`                    |

### 8.10 Notifications

Push notifications are optional. If `METIS_PUSH_NOTIFICATION_ENDPOINT_URL` is
empty the push channel in `send_notification` is a no-op; email and in-app
channels still work.

| Variable                                     | Default |
| -------------------------------------------- | ------- |
| `METIS_PUSH_NOTIFICATION_ENDPOINT_URL`       | (empty) |
| `METIS_PUSH_NOTIFICATION_API_KEY`            | (empty) |
| `METIS_PUSH_NOTIFICATION_TIMEOUT_SECONDS`    | `5.0`   |
| `METIS_PUSH_NOTIFICATION_OUTBOX_TTL_SECONDS` | 30 days |

---

## 9. Observability

Metis provides three layers of operational visibility, designed so that a
problem can be spotted in Grafana (tracing), diagnosed in log aggregation
(structured logs), or checked instantly from a load balancer (health endpoints).

- **Structured logging:** `structlog` with a console renderer in
  development/testing and a JSON renderer in staging/production. Context vars
  are merged so request-scoped fields propagate across log lines.
- **Tracing:** OpenTelemetry is optional (`otel_enabled`). When enabled,
  `_setup_opentelemetry` configures an OTLP gRPC span exporter, a
  `TraceIdRatioBased` sampler, a resource with service name/version/environment,
  and auto-instruments FastAPI (excluding `health`/`ready`). SQLAlchemy
  instrumentation is available via the installed
  `opentelemetry-instrumentation-sqlalchemy` package.
- **Request timing:** every response carries an `X-Request-Time` header.
- **Health/readiness:** `/health` (liveness) and `/ready` (verifies database and
  Redis; returns 503 if either is down).
- **Error handling:** `RequestValidationError` returns a structured 422 with
  per-field detail; unhandled exceptions log a full traceback and return a
  generic 500.

---

## 10. Quality Standards

These standards define the bar that every change to Metis must meet before it is
considered shippable. The OpenAPI contract test is particularly important: it
prevents accidental schema drift between the live service and the exported
`metis.openapi.json` artifact that downstream consumers depend on.

- **Type checking:** `mypy` in `strict` mode with the `pydantic.mypy` plugin
  (`pyproject.toml`); `tests`/`build`/`dist`/`alembic` are excluded.
- **Linting:** `ruff` with rule sets `E,W,F,I,B,C4,UP,ARG,SIM,N`, line length
  100, `target-version = py311`.
- **Testing:** `pytest` + `pytest-asyncio` (`asyncio_mode = "auto"`); test files
  under `services/metis/tests/`. Markers: `slow`, `integration`, `unit`.
- **Coverage:** branch coverage on `src/metis`, `fail_under = 80`.
- **OpenAPI contract:** `scripts/export_openapi.py` and
  `metis.openapi_contracts` render a deterministic, key-sorted OpenAPI document
  with a canonical root-key order; `tests/test_openapi_contract.py` guards it
  against drift.

---

## 11. Acceptance Criteria

A change to Metis is acceptance-ready when all six conditions below are
satisfied. These criteria exist because each one guards a different class of
regression: type errors, logic bugs, schema drift, coverage gaps, auth holes,
and cross-domain contract violations.

1. `mypy --strict` and `ruff` pass for `services/metis/src`.
2. `pytest` passes, including the OpenAPI contract test
   (`tests/test_openapi_contract.py`) and the export-script test
   (`tests/test_export_openapi_script.py`).
3. New ORM columns are accompanied by an Alembic migration and matching Pydantic
   request/response schemas.
4. Branch coverage stays at or above 80%.
5. New endpoints declare explicit auth dependencies (`get_current_user`,
   `get_current_instructor_user`, or `get_current_admin_user`) and enforce
   resource-level ownership where applicable.
6. Cross-domain payloads (Themis verdicts, Psyche runtime events, Yemaya render
   results) conform to the schemas in `services/metis/src/metis/schemas`.

---

## 12. Correctness Verification Subsystem

The correctness-verification subsystem lives in the TypeScript library
`@metis/verification` (with claim/media agents in `@metis/agents`). It is
consumed by content generation and gates release of generated lessons. Unlike
the Python backend specced above, it is library code; the contract below
describes its types and decision algebra.

### 12.1 Gate decision algebra

- `Verifier.verify(content, context) → VerifierResult` returns a normalized
  `score` (0–1), a calibrated `confidence`, a `passed` flag, an optional
  `critique`, and may return `{ notConfigured: true }` when its backing model is
  absent.
- `VerificationGate` composes verifiers (marked `required` or advisory) and
  aggregates to a `GateDecision` of `pass | needs-human | block`:
  - any **required** verifier that is `notConfigured` or fails → `block`
    (fail-loud; the gate never silently passes a missing required check);
  - low aggregate confidence or high judge disagreement → `needs-human`;
  - otherwise → `pass`.
- `composeP0Gate(config)` builds the default panel; `runVerifiedGeneration`
  drives `generate → verify → (regenerate|refine) → re-gate` up to a bound;
  `runVerifierGuidedGeneration` scores N candidates and selects pessimistically.

### 12.2 Verifiers

| Verifier                | Method                                                                                              |
| ----------------------- | --------------------------------------------------------------------------------------------------- |
| Factuality              | atomic claim decomposition → evidence retrieval → entailment → aggregate; claim→source span linking |
| Faithfulness (TRACe)    | groundedness / relevance / completeness over RAG context                                            |
| Citation sufficiency    | per-statement support adequacy                                                                      |
| Math correctness        | CAS-lite expression evaluation + identity testing                                                   |
| Code correctness        | sandboxed JavaScript execution against declared cases/checks                                        |
| Process (worked steps)  | per-transition validity of an equation chain                                                        |
| Pedagogical judge panel | rubric scoring + ICC consistency / Spearman alignment / discrimination                              |
| Selective evaluation    | conformal threshold (Clopper–Pearson exact, Hoeffding, RCPS) → abstain                              |
| Contradiction           | negation / value-mismatch heuristic + optional NLI seam                                             |
| Uncertainty calibration | expected calibration error + histogram calibrator                                                   |
| Pedagogy (curriculum)   | Flesch–Kincaid reading level, prerequisite order, misconception scan                                |

### 12.3 Supporting machinery

- **Evidence ledger** — `buildEvidenceRecord` stamps run id, model, content
  sha256, per-claim verdicts, judge scores, disagreement, and the gate decision;
  `verifyEvidenceBinding` detects tampering.
- **Eval harness** — `scoreVerifier` computes precision/recall/F1, Cohen's κ,
  and calibration error against a versioned JSONL gold set; recorded baselines
  live in `EVAL_BASELINES.md` (e.g. the lexical-default factuality baseline: P
  0.516 / R 1.0 / F1 0.681 / κ 0.0625 / MAE 0.464, motivating the NLI swap).
- **HITL** — `selectForReview` routes low-confidence / high-disagreement items
  to a versioned `FeedbackStore`; labels feed judge calibration, the selective
  threshold, and per-criterion reliability.
- **Budgets & hardening** — `ComputeBudgetMeter` + `MODE_BUDGETS` per call;
  `checkBudgetCeilings` asserts p95 latency + total cost against `MODE_CEILINGS`
  and recommends a kill-switch; `monitorAgreementDrift` (Welch z) and
  `evaluateVerifierPromotion` (two-proportion z) gate champion-challenger
  promotion.

## 13. Agentic Media & Manim Render Service

The agentic teaching-media pipeline (`@metis/agents` Planner/Coder/Critic +
`@metis/multimedia` loop/compositor) produces verified animations; the rendering
itself runs in a sandboxed Python worker under
`services/metis/src/metis/media/`.

### 13.1 Author→critic loop

`runAgenticMediaLoop(lesson, seams, config)` executes: verify lesson (a `block`
aborts before any media) → plan scenes → for each scene
`{ code → render → critique → (repair) }*` until the critic score ≥
`minSceneScore` or `maxIterationsPerScene` / `maxTotalRenders` is hit → optional
learning-outcome probe → release decision. The renderer, planner, coder, critic,
lesson verifier, and outcome probe are all injected seams.

### 13.2 Manim render service contract

`ManimRenderService.render(job)` executes generated Manim code out-of-process:

- **Request** (`RenderJob` / wire `ManimRenderRequest`): `sceneName`, `code`,
  `resolution {width,height}`, `fps`, `format` (`mp4|webm|mov|gif`),
  `outputFrames`, `timeoutMs`.
- **Sandbox**: a per-job working directory; the `manim` CLI is spawned in its
  own session with a wall-clock timeout (whole process-group killed on expiry)
  and POSIX `RLIMIT_CPU` / optional `RLIMIT_AS` limits; the timeout is clamped
  to a service ceiling.
- **Response** (`RenderResult` / wire `ManimRenderResponse`): on success a video
  path, ffprobe-measured `durationSeconds`, and optional ffmpeg-extracted
  `framePaths`; on failure a structured error of kind
  `syntax | runtime | timeout | unknown` with stderr + offending line — never a
  thrown crash. With no Manim binary the service raises
  `RendererNotConfiguredError` (fail-loud, never a fabricated artifact).
- **Transports**: a standalone FastAPI app (`manim_render_app`, `POST /render`,
  `GET /healthz`) and a one-shot stdin/stdout CLI (`manim_render_cli`); the TS
  `ManimClient` reaches either through `createHttpManimTransport` or
  `createChildProcessManimTransport`. The worker requires the `render` extra
  (`manim`) plus the `ffmpeg`/`ffprobe` system binaries (and a LaTeX toolchain
  for MathTex).

### 13.3 Segment composition

`composeMediaSegment(plan, rendered, options)` assembles a finished segment: one
full-frame `VideoCompositor` background layer per rendered scene, laid
back-to-back; one real TTS narration `AudioSegment` per scene, retimed to its
window; each scene window widened to fit the longest of {planned,
rendered-video, narration} duration so video and speech stay synced; a
WebVTT/SRT caption track and a timestamped transcript; and an optional
avatar/lip-sync picture-in-picture overlay. A planned scene with no rendered
artifact raises `MissingSceneRenderError` rather than compositing a gap.