Generated: 2026-03-24 Scope: Comprehensive test coverage comparison
between the Minerva Python source (minerva/) and the Metis TypeScript port
(libs/metis/)
1. Executive Summary#
| Metric |
Minerva (Python) |
Metis (TypeScript) |
| Total test files |
1,822 |
188 |
| Total test functions/cases |
37,977 |
15,913 |
| Test directories/libraries |
78 (with tests) |
22 |
| Source modules |
93 |
22 |
| Root-level test files |
380 |
- |
| Root-level test functions |
5,790 |
- |
Metis currently covers 42% of Minerva's test count across 24% of Minerva's
module surface area. The ported libraries have strong test density (averaging
725 tests per library) but significant Minerva modules remain unported.
2. Minerva Python Test Suite Overview#
2.1 Directory Structure#
Minerva's test suite is organized into 78 subdirectories plus 380 root-level
test files. The root-level files contain 5,790 test functions covering
cross-cutting concerns, integration scenarios, and modules that span multiple
domains.
2.2 Test Counts by Module (Top 30 by Test Functions)#
| Minerva Module |
Test Files |
Test Functions |
Category |
| research |
96 |
5,989 |
Knowledge |
| verification |
65 |
3,103 |
Quality |
| integration |
65 |
2,536 |
Cross-cutting |
| validation |
54 |
2,191 |
Quality |
| api |
69 |
2,028 |
Infrastructure |
| linters |
64 |
1,963 |
Quality |
| learning |
41 |
1,502 |
Adaptive |
| video |
28 |
1,176 |
Multimedia |
| tutor |
24 |
921 |
Tutoring |
| models |
22 |
869 |
Core |
| audio |
26 |
842 |
Multimedia |
| cli |
65 |
721 |
Infrastructure |
| content |
95 |
656 |
Content |
| agents |
54 |
519 |
AI Agents |
| byom |
20 |
499 |
Custom Models |
| knowledge |
19 |
496 |
Knowledge |
| assessment |
20 |
481 |
Assessment |
| experiments |
15 |
460 |
Experimentation |
| content_analytics |
9 |
352 |
Analytics |
| providers |
15 |
341 |
LLM Providers |
| database |
30 |
313 |
Storage |
| security |
29 |
305 |
Security |
| background |
15 |
261 |
Infrastructure |
| accessibility |
56 |
238 |
Accessibility |
| storyboard |
7 |
234 |
Content |
| prompts |
25 |
204 |
LLM |
| guidance |
7 |
200 |
Pedagogical |
| common |
21 |
194 |
Shared Utilities |
| rag |
8 |
187 |
Retrieval |
| llm_integration |
9 |
185 |
LLM |
2.3 Test Counts by Module (Remaining)#
| Minerva Module |
Test Files |
Test Functions |
Category |
| course |
62 |
161 |
Course |
| syllabus |
5 |
143 |
Course |
| multimedia |
14 |
126 |
Multimedia |
| scripts |
10 |
111 |
Tooling |
| diff |
12 |
110 |
Content |
| vlm |
40 |
106 |
Vision LM |
| comfyui_nodes |
4 |
106 |
Image Gen |
| templates |
3 |
104 |
Templates |
| tools |
9 |
95 |
Tooling |
| orchestration |
12 |
93 |
Orchestration |
| document_processing |
2 |
92 |
Content |
| visualization |
8 |
91 |
Visualization |
| observability |
44 |
85 |
Infrastructure |
| concept_graph |
3 |
85 |
Knowledge |
| performance |
8 |
82 |
Infrastructure |
| embeddings |
2 |
71 |
ML |
| course_creation_asst |
15 |
69 |
Course |
| ui |
21 |
55 |
Frontend |
| content_generation |
4 |
43 |
Content |
| analytics |
15 |
40 |
Analytics |
| e2e |
2 |
39 |
Integration |
| settings |
2 |
37 |
Configuration |
| monitoring |
4 |
28 |
Infrastructure |
| dashboard |
11 |
21 |
Frontend |
| export |
2 |
18 |
Export |
| benchmarks |
3 |
18 |
Performance |
| integrations |
7 |
16 |
Third-party |
| data |
7 |
16 |
Data |
| unified |
2 |
14 |
Orchestration |
| compliance |
4 |
11 |
Compliance |
| administration |
5 |
11 |
Admin |
| metrics |
3 |
8 |
Metrics |
| progress |
2 |
7 |
Progress |
| lms |
2 |
7 |
LMS |
| tui |
3 |
6 |
CLI/TUI |
| analysis |
3 |
5 |
Analysis |
| optimization |
2 |
4 |
Optimization |
| llm |
2 |
2 |
LLM |
Root-level tests (380 files, 5,790 functions) cover cross-cutting concerns
including accessibility compliance, diagram rendering, content moderation, error
handling, mathematical verification, multimedia pipelines, provider
configuration, syllabus processing, and end-to-end workflows.
3. Metis TypeScript Test Suite Current State#
3.1 Per-Library Test Coverage#
| Metis Library |
Test Files |
Test Cases |
Source Files |
Status |
| @metis/tutoring |
7 |
1,252 |
8 |
Complete |
| @metis/assessment |
19 |
1,112 |
10 |
Complete |
| @metis/prompt-mgmt |
5 |
1,095 |
7 |
Complete |
| @metis/llm-client |
5 |
1,056 |
7 |
Complete |
| @metis/cost-tracking |
5 |
1,036 |
7 |
Complete |
| @metis/linters-types |
6 |
965 |
7 |
Complete |
| @metis/research |
7 |
964 |
8 |
Complete |
| @metis/ab-testing |
5 |
952 |
7 |
Complete |
| @metis/learning |
7 |
878 |
8 |
Complete |
| @metis/model-registry |
5 |
863 |
7 |
Complete |
| @metis/api-client |
7 |
726 |
9 |
Complete |
| @metis/database-types |
5 |
715 |
6 |
Complete |
| @metis/multimedia |
19 |
710 |
5 |
Complete |
| @metis/quality |
15 |
618 |
5 |
Complete |
| @metis/course |
7 |
577 |
8 |
Complete |
| @metis/agents |
19 |
535 |
3 |
Complete |
| @metis/adaptive |
12 |
437 |
4 |
Complete |
| @metis/knowledge-graph |
10 |
403 |
4 |
Complete |
| @metis/models |
9 |
393 |
10 |
Complete |
| @metis/core |
7 |
322 |
9 |
Complete |
| @metis/agents-types |
6 |
224 |
7 |
Complete |
| @metis/integrations |
1 |
78 |
7 |
Partial |
| Total |
188 |
15,913 |
157 |
|
3.2 Test Density Analysis#
| Metric |
Value |
| Average tests per library |
725 |
| Median tests per library |
726 |
| Highest test count |
1,252 (@metis/tutoring) |
| Lowest test count (excl. integ) |
224 (@metis/agents-types) |
| Average tests per test file |
85 |
4. Coverage Gap Analysis#
4.1 Ported Modules: Minerva → Metis Mapping#
These Minerva modules have corresponding Metis TypeScript libraries with tests.
| Minerva Module |
Minerva Tests |
Metis Library |
Metis Tests |
Coverage Ratio |
| models |
869 |
@metis/models |
393 |
45% |
| agents |
519 |
@metis/agents + agents-types |
759 |
146% |
| api |
2,028 |
@metis/api-client |
726 |
36% |
| assessment |
481 |
@metis/assessment |
1,112 |
231% |
| course |
161 |
@metis/course |
577 |
358% |
| learning |
1,502 |
@metis/learning |
878 |
58% |
| multimedia |
126 |
@metis/multimedia |
710 |
563% |
| research |
5,989 |
@metis/research |
964 |
16% |
| linters |
1,963 |
@metis/linters-types |
965 |
49% |
| knowledge |
496 |
@metis/knowledge-graph |
403 |
81% |
| tutor |
921 |
@metis/tutoring |
1,252 |
136% |
| database |
313 |
@metis/database-types |
715 |
228% |
Key observations:
- Over-covered modules (>100%): assessment, course, multimedia, database,
tutoring, and agents have MORE tests in Metis than Minerva. This reflects the
TypeScript ports being comprehensive type-level libraries with extensive
validation tests, while Minerva's tests for these modules may be more
integration-focused.
- Under-covered modules (<50%): research (16%), api (36%), models (45%), and
linters (49%) have significantly fewer Metis tests than Minerva. These are the
largest and most complex Minerva modules.
- Research is the biggest gap: 5,989 Minerva tests vs 964 Metis tests. This
is expected since Metis research is currently a types library, while Minerva
research includes full knowledge acquisition, crawling, embedding, and
verification logic.
4.2 Additional Metis Libraries (No Direct Minerva Test Directory)#
These Metis libraries were created from cross-cutting Minerva functionality or
new Oshun integration needs rather than mapping 1:1 to a Minerva test directory.
| Metis Library |
Tests |
Derived From (Minerva) |
| @metis/llm-client |
1,056 |
llm_utils.py, providers/, llm/ |
| @metis/prompt-management |
1,095 |
prompts/, agents/ prompt patterns |
| @metis/model-registry |
863 |
providers/, experiments/ |
| @metis/cost-tracking |
1,036 |
analytics/, providers/ cost tracking |
| @metis/ab-testing |
952 |
experiments/, content_analytics/ |
| @metis/adaptive |
437 |
learning/ adaptive algorithms |
| @metis/quality |
618 |
verification/, validation/, linters/ |
| @metis/integrations |
78 |
integrations/, integration/ |
4.3 Unported Minerva Modules#
The following Minerva modules have tests but no corresponding Metis library
yet. Ordered by test function count (descending priority).
| Minerva Module |
Test Files |
Test Functions |
Priority |
Notes |
| verification |
65 |
3,103 |
P0 |
Core quality — partially in @metis/quality |
| integration |
65 |
2,536 |
P1 |
Cross-module integration tests |
| validation |
54 |
2,191 |
P0 |
Core quality — partially in @metis/quality |
| video |
28 |
1,176 |
P1 |
Multimedia production |
| audio |
26 |
842 |
P1 |
TTS and audio pipeline |
| cli |
65 |
721 |
P2 |
Command-line interface |
| content |
95 |
656 |
P0 |
Content management core |
| byom |
20 |
499 |
P2 |
Bring-your-own-model |
| experiments |
15 |
460 |
P1 |
Partially in @metis/ab-testing |
| content_analytics |
9 |
352 |
P1 |
Content performance analytics |
| providers |
15 |
341 |
P1 |
Partially in @metis/llm-client |
| security |
29 |
305 |
P0 |
Authentication, authorization |
| background |
15 |
261 |
P2 |
Background job processing |
| accessibility |
56 |
238 |
P1 |
WCAG compliance |
| storyboard |
7 |
234 |
P2 |
Storyboard generation |
| prompts |
25 |
204 |
P1 |
Partially in @metis/prompt-mgmt |
| guidance |
7 |
200 |
P2 |
Pedagogical guidance |
| common |
21 |
194 |
P1 |
Shared utilities |
| rag |
8 |
187 |
P1 |
Retrieval-augmented generation |
| llm_integration |
9 |
185 |
P1 |
Partially in @metis/llm-client |
| diff |
12 |
110 |
P2 |
Content diffing |
| vlm |
40 |
106 |
P2 |
Vision language models |
| comfyui_nodes |
4 |
106 |
P2 |
Image generation |
| templates |
3 |
104 |
P2 |
Template system |
| orchestration |
12 |
93 |
P1 |
Multi-agent orchestration |
| observability |
44 |
85 |
P2 |
Monitoring and tracing |
| concept_graph |
3 |
85 |
P2 |
Partially in @metis/knowledge-graph |
| performance |
8 |
82 |
P2 |
Performance benchmarking |
| embeddings |
2 |
71 |
P1 |
Vector embeddings |
| visualization |
8 |
91 |
P2 |
Data visualization |
| syllabus |
5 |
143 |
P1 |
Partially in @metis/course |
| document_processing |
2 |
92 |
P2 |
Document parsing |
| Root-level |
380 |
5,790 |
Mixed |
Cross-cutting, integration, E2E |
Total unported test surface: ~21,684 test functions across 1,254 test files
(plus 5,790 root-level tests).
5. Migration Validation Criteria#
5.1 Per-Module Porting Success Criteria#
A Minerva module is considered successfully ported to Metis when:
| Criterion |
Requirement |
| Type coverage |
All Pydantic models have TypeScript/Zod equivalents |
| API parity |
All public Python functions have TypeScript equivalents |
| Test coverage ratio |
>= 80% of Minerva test function count |
| Domain-specific tests |
Tests validate domain logic, not just CRUD |
| Edge case coverage |
Error paths, boundary conditions, empty inputs |
| Integration tests |
Cross-module interactions verified |
| Build verification |
tsc --noEmit passes, library builds cleanly |
5.2 Overall Migration Completion Thresholds#
| Milestone |
Test Count Target |
Module Count |
Description |
| Alpha |
15,000+ |
22 (current) |
Core type libraries ported |
| Beta |
25,000+ |
35+ |
Major modules ported |
| Release Candidate |
32,000+ |
50+ |
All P0/P1 modules ported |
| GA |
38,000+ |
60+ |
Full parity with Minerva |
Current status: Alpha (15,913 tests across 22 libraries).
5.3 Test Quality Requirements#
Beyond raw test counts, each ported module must demonstrate:
- Algorithm verification — Tests that validate domain-specific algorithms
produce correct results (e.g., adaptive difficulty calculations, assessment
scoring, knowledge graph traversal)
- Serialization roundtrips — Data can be serialized and deserialized
without loss
- Validation coverage — Invalid inputs are rejected with appropriate errors
- Concurrent safety — Async operations handle race conditions where
applicable
- Performance baselines — Critical path operations have timing assertions
6. Priority Test Migration Order#
Phase 1: Core Quality (P0) — Target: +5,000 tests#
| Priority |
Module |
Minerva Tests |
Gap |
Rationale |
| 1 |
verification |
3,103 |
~2,500 |
Core quality assurance pipeline |
| 2 |
validation |
2,191 |
~1,600 |
Input/output validation |
| 3 |
content |
656 |
~656 |
Content management fundamentals |
| 4 |
security |
305 |
~305 |
Auth and authorization |
Phase 2: AI and Knowledge (P1) — Target: +6,000 tests#
| Priority |
Module |
Minerva Tests |
Gap |
Rationale |
| 5 |
research (expand) |
5,989 |
~5,025 |
Knowledge acquisition engine |
| 6 |
api (expand) |
2,028 |
~1,302 |
Full API coverage |
| 7 |
learning (expand) |
1,502 |
~624 |
Adaptive learning algorithms |
| 8 |
providers |
341 |
~341 |
LLM provider abstraction |
| 9 |
rag |
187 |
~187 |
Retrieval-augmented generation |
| 10 |
embeddings |
71 |
~71 |
Vector search |
| Priority |
Module |
Minerva Tests |
Gap |
Rationale |
| 11 |
video |
1,176 |
~1,176 |
Video production pipeline |
| 12 |
audio |
842 |
~842 |
TTS and audio processing |
| 13 |
accessibility |
238 |
~238 |
WCAG compliance |
| 14 |
storyboard |
234 |
~234 |
Storyboard generation |
Phase 4: Infrastructure and Integration (P1-P2) — Target: +4,000 tests#
| Priority |
Module |
Minerva Tests |
Gap |
Rationale |
| 15 |
integration |
2,536 |
~2,536 |
Cross-module integration |
| 16 |
experiments |
460 |
~460 |
Experiment framework |
| 17 |
content_analytics |
352 |
~352 |
Content performance metrics |
| 18 |
common |
194 |
~194 |
Shared utilities |
| 19 |
orchestration |
93 |
~93 |
Multi-agent orchestration |
| 20 |
cli |
721 |
~721 |
Command-line tooling |
Phase 5: Specialized and Remaining (P2) — Target: +3,000 tests#
Covers: byom, diff, vlm, comfyui_nodes, templates, background, guidance,
observability, concept_graph, performance, visualization, document_processing,
syllabus, dashboard, monitoring, and root-level cross-cutting tests.
7. Appendix: Complete Module Inventory#
7.1 Minerva Source Modules (93 total)#
Organized by Python source file count (descending):
| Module |
Source Files |
Test Dir Exists |
Tests |
| integration |
336 |
Yes |
2,536 |
| visualization |
248 |
Yes |
91 |
| validation |
142 |
Yes |
2,191 |
| research |
142 |
Yes |
5,989 |
| agents |
120 |
Yes |
519 |
| learning |
118 |
Yes |
1,502 |
| course |
101 |
Yes |
161 |
| assessment |
99 |
Yes |
481 |
| content |
97 |
Yes |
656 |
| verification |
91 |
Yes |
3,103 |
| tutor |
88 |
Yes |
921 |
| api |
81 |
Yes |
2,028 |
| byom |
80 |
Yes |
499 |
| diff |
65 |
Yes |
110 |
| linters |
63 |
Yes |
1,963 |
| accessibility |
58 |
Yes |
238 |
| security |
57 |
Yes |
305 |
| orchestration |
57 |
Yes |
93 |
| cli |
57 |
Yes |
721 |
| observability |
54 |
Yes |
85 |
| vlm |
52 |
Yes |
106 |
| prompts |
52 |
Yes |
204 |
| video |
43 |
Yes |
1,176 |
| common |
37 |
Yes |
194 |
| audio |
35 |
Yes |
842 |
| knowledge |
30 |
Yes |
496 |
| database |
28 |
Yes |
313 |
| optimization |
27 |
Yes |
4 |
| experiments |
26 |
Yes |
460 |
| models |
25 |
Yes |
869 |
| multimedia |
23 |
Yes |
126 |
| tui |
20 |
Yes |
6 |
| course_creation_asst |
19 |
Yes |
69 |
| dependency_health |
18 |
No |
- |
| compliance_automation |
18 |
No |
- |
| background |
15 |
Yes |
261 |
| analytics |
15 |
Yes |
40 |
| administration |
15 |
Yes |
11 |
| retry |
14 |
No |
- |
| generation |
14 |
No |
- |
| document_processing |
14 |
Yes |
92 |
| devops |
14 |
No |
- |
| moderation |
13 |
Yes |
0 |
| integrations |
13 |
Yes |
16 |
| reasoning |
12 |
No |
- |
| rag |
12 |
Yes |
187 |
| concept_graph |
12 |
Yes |
85 |
| caching |
11 |
Yes |
0 |
| analysis |
11 |
Yes |
5 |
| diversity |
11 |
No |
- |
| performance |
11 |
Yes |
82 |
| providers |
11 |
Yes |
341 |
| tools |
11 |
Yes |
95 |
| version_control |
10 |
No |
- |
| dashboard |
10 |
Yes |
21 |
| guidance |
9 |
Yes |
200 |
| templates |
8 |
Yes |
104 |
| storyboard |
8 |
Yes |
234 |
| refinement |
8 |
No |
- |
| collaboration |
8 |
No |
- |
| autonomous_tour |
8 |
No |
- |
| tracking |
7 |
No |
- |
| metrics |
7 |
Yes |
8 |
| data |
7 |
Yes |
16 |
| progress |
6 |
Yes |
7 |
| io |
6 |
No |
- |
| flow |
6 |
No |
- |
| export |
6 |
Yes |
18 |
| curriculum |
6 |
No |
- |
| core |
6 |
No |
- |
| diagram |
6 |
No |
- |
| unified |
5 |
Yes |
14 |
| schemas |
5 |
No |
- |
| attribution |
5 |
No |
- |
| compliance |
4 |
Yes |
11 |
| assessment_tools |
4 |
No |
- |
| service_discovery |
3 |
No |
- |
| llm |
3 |
Yes |
2 |
| generators |
3 |
No |
- |
| embeddings |
3 |
Yes |
71 |
| documentation |
3 |
Yes |
0 |
| content_standards |
3 |
No |
- |
| cache |
3 |
Yes |
0 |
7.2 Metis Libraries (22 total)#
All libraries reside under libs/metis/ in the Oshun monorepo.
| Library |
Source Files |
Test Files |
Test Cases |
Phase |
| @metis/core |
9 |
7 |
322 |
25.4.1 |
| @metis/models |
10 |
9 |
393 |
25.4.2 |
| @metis/agents-types |
7 |
6 |
224 |
25.4.3 |
| @metis/api-client |
9 |
7 |
726 |
25.4.4 |
| @metis/course |
8 |
7 |
577 |
25.4.5 |
| @metis/assessment |
10 |
19 |
1,112 |
25.4.6 |
| @metis/multimedia |
5 |
19 |
710 |
25.4.7 |
| @metis/tutoring |
8 |
7 |
1,252 |
25.4.8 |
| @metis/linters-types |
7 |
6 |
965 |
25.4.9 |
| @metis/research |
8 |
7 |
964 |
25.4.10 |
| @metis/learning |
8 |
7 |
878 |
25.4.11 |
| @metis/database-types |
6 |
5 |
715 |
25.4.12 |
| @metis/llm-client |
7 |
5 |
1,056 |
25.2.2 |
| @metis/prompt-mgmt |
7 |
5 |
1,095 |
25.2.2 |
| @metis/model-registry |
7 |
5 |
863 |
25.2.2 |
| @metis/cost-tracking |
7 |
5 |
1,036 |
25.2.2 |
| @metis/ab-testing |
7 |
5 |
952 |
25.2.2 |
| @metis/agents |
3 |
19 |
535 |
25.2.4 |
| @metis/adaptive |
4 |
12 |
437 |
- |
| @metis/quality |
5 |
15 |
618 |
- |
| @metis/knowledge-graph |
4 |
10 |
403 |
- |
| @metis/integrations |
7 |
1 |
78 |
- |