Eve library

phase-6.md

Tracker

Revision 1

On this page

Historical task source. The Eve task board holds current task state.

Phase 6: Sophia (Research & Knowledge) Migration#

6.1 Sophia Library: Document Schemas ✅#

  • 6.1.1 Create libs/sophia/schemas/ project
  • 6.1.2 Add project.json with scope:sophia, layer:contracts tags
  • 6.1.3 Define ResearchDocument schema
  • 6.1.4 Define Citation schema
  • 6.1.5 Define Claim schema
  • 6.1.6 Define KnowledgePack schema
  • 6.1.7 Define Entity schema (for knowledge graph)
  • 6.1.8 Define Relation schema

6.2 Sophia Library: Indexing Pipelines ✅#

  • 6.2.1 Create libs/sophia/indexing/ project
  • 6.2.2 Add project.json with scope:sophia, layer:domain tags
  • 6.2.3 Migrate lilith/services/ingestion chunking logic
  • 6.2.4 Create chunking strategies (semantic, fixed, sliding window)
  • 6.2.5 Create embedding interface
  • 6.2.6 Add embedding provider adapters (OpenAI, Cohere, local)
  • 6.2.7 Create index builder utilities

6.3 Sophia Library: Vector DB ✅#

  • 6.3.1 Create libs/sophia/vectordb/ project
  • 6.3.2 Add project.json with scope:sophia, layer:data tags
  • 6.3.3 Migrate lilith/packages/vectordb (Milvus client)
  • 6.3.4 Migrate lilith/services/vectordb
  • 6.3.5 Create vector search interface
  • 6.3.6 Add hybrid search (vector + keyword)
  • 6.3.7 Create index management utilities

6.4 Sophia Library: Cultural Research ✅#

  • 6.4.1 Create libs/sophia/cultural-research/ project
  • 6.4.2 Add project.json with scope:sophia, layer:domain tags
  • 6.4.3 Migrate yemaya/packages/cultural-research
  • 6.4.4 Create AI cultural consultant interface
  • 6.4.5 Create translation pipeline
  • 6.4.6 Add inclusive design utilities

6.5 Sophia Library: Theory (Research Frameworks) ✅#

  • 6.5.1 Create libs/sophia/theory/ project
  • 6.5.2 Add project.json with scope:sophia, layer:domain tags
  • 6.5.3 Migrate yemaya/packages/theory research components
  • 6.5.4 Create research methodology frameworks
  • 6.5.5 Add citation integrity validation

6.6 Sophia Library: Evaluation#

  • 6.6.1 Create libs/sophia/evaluation/ project
  • 6.6.2 Add project.json with scope:sophia, layer:domain tags
  • 6.6.3 Create retrieval quality evaluation harness
  • 6.6.4 Create citation integrity checker
  • 6.6.5 Create groundedness scorer
  • 6.6.6 Add benchmark datasets

6.7 Sophia App: Ingestion Service#

  • 6.7.1 Create apps/sophia/ingestion/ project
  • 6.7.2 Add project.json with scope:sophia, type:app tags
  • 6.7.3 Migrate lilith/services/ingestion
  • 6.7.4 Create connector framework:
    • PDF connector
    • Web scraper connector
    • API connectors (arXiv, PubMed, etc.)
  • 6.7.5 Add OCR pipeline
  • 6.7.6 Create chunking worker
  • 6.7.7 Create enrichment pipeline (entity extraction, summarization)

6.8 Sophia App: Search/RAG API#

  • 6.8.1 Create apps/sophia/search-api/ project
  • 6.8.2 Add project.json with scope:sophia, type:app tags
  • 6.8.3 Migrate lilith/services/rag
  • 6.8.4 Create retrieval endpoints
  • 6.8.5 Create citation endpoints
  • 6.8.6 Create knowledge pack endpoints
  • 6.8.7 Add reranking pipeline
  • 6.8.8 Implement semantic search
  • 6.8.9 Add hybrid search (BM25 + vector)

6.9 Sophia App: Knowledge Graph API#

  • 6.9.1 Create apps/sophia/knowledge-graph/ project
  • 6.9.2 Add project.json with scope:sophia, type:app tags
  • 6.9.3 Migrate lilith/services/knowledge
  • 6.9.4 Migrate lilith/services/curator-admin
  • 6.9.5 Create entity CRUD endpoints
  • 6.9.6 Create relation endpoints
  • 6.9.7 Create graph query endpoints (subgraph, paths)
  • 6.9.8 Add entity resolution

6.10 Sophia App: Research Workbench#

  • 6.10.1 Create apps/sophia/workbench/ project
  • 6.10.2 Add project.json with scope:sophia, type:app tags
  • 6.10.3 Create curation UI
  • 6.10.4 Create review workflow UI
  • 6.10.5 Add annotation tools

6.11 Sophia Client SDK#

  • 6.11.1 Create libs/sophia/client/ project
  • 6.11.2 Add project.json with scope:sophia, layer:clients tags
  • 6.11.3 Generate TypeScript client from OpenAPI
  • 6.11.4 Create @oshun/sophia-client package
  • 6.11.5 Add search helpers
  • 6.11.6 Add citation retrieval helpers
  • 6.11.7 Add document ingestion helpers

6.12 Sophia Database Schema#

  • 6.12.1 Create libs/sophia/database/ project
  • 6.12.2 Add project.json with scope:sophia, layer:data tags
  • 6.12.3 Define Prisma schema for Sophia:
    • Document model
    • DocumentChunk model
    • Citation model
    • Entity model
    • Relation model
    • KnowledgePack model
    • IngestionJob model
  • 6.12.4 Create migrations
  • 6.12.5 Add seed data

6.13 Knowledge Stack Convergence (from Lilith)#

  • 6.13.1 Audit lilith/services/spiritual-embeddings for migration
    • OpenAI/HuggingFace embedding providers
    • Hash-based fallback for development
    • Batch embedding generation
  • 6.13.2 Audit lilith/services/knowledge-access for migration
    • ContentBundle, AccessPolicy, AccessPass models
    • Token-gated access control
    • IPFS storage integration
  • 6.13.3 Plan convergence of lilith knowledge services to Sophia
    • Created CONVERGENCE_PLAN.md in libs/sophia/
    • Data model mappings defined
    • Entity type and relation type mappings
  • 6.13.4 Create migration scripts for existing knowledge data
    • migrate-lilith-knowledge.ts in database/prisma/
    • Migrates traditions, concepts, figures, practices, texts, places
    • Creates cross-tradition relationships
  • 6.13.5 Update Lilith services to consume Sophia APIs
    • Created @lilith/sophia-adapter library
    • KnowledgeAccessAdapter for ContentBundle/AccessPolicy/AccessPass mapping
    • KnowledgeGraphAdapter for Entity/Relation mapping
    • EmbeddingAdapter for embedding API mapping