# @sophia - Knowledge & Research Platform

Sophia is Oshun's knowledge management and RAG (Retrieval-Augmented Generation)
infrastructure, providing document ingestion, semantic search, and citation
management.

## Overview

| Aspect              | Description                |
| ------------------- | -------------------------- |
| **Domain**          | Knowledge Management & RAG |
| **Scope Tag**       | `scope:sophia`             |
| **API Port**        | 4003                       |
| **Database Schema** | `sophia`                   |

## Capabilities

- **Document Ingestion**: PDF, DOC, HTML, Markdown, plain text
- **Semantic Search**: Vector-based similarity search
- **Knowledge Graphs**: Entity extraction and relationship mapping
- **Citation Management**: Source tracking and attribution
- **Multi-Collection**: Organized document collections
- **RAG Pipeline**: Context retrieval for LLM prompts

## Libraries

```
libs/sophia/
├── client/              # TypeScript client for other domains
├── database/            # Prisma schema & utilities
├── schemas/             # Document & collection schemas
├── indexing/            # Vector indexing & embeddings
├── vectordb/            # Vector store integration (Qdrant/Pinecone)
├── evaluation/          # Quality metrics & retrieval accuracy
├── theory/              # Framework definitions & models
└── cultural-research/   # Cultural context & reference research
```

## Quick Start

### Using the Client

```typescript
import { createSophiaClient } from '@sophia/client';

const sophia = createSophiaClient({
  baseUrl: process.env.SOPHIA_API_URL || 'http://localhost:4003',
});

// Ingest a document
const document = await sophia.documents.ingest({
  source: 'https://example.com/research.pdf',
  collection: 'research',
  metadata: { author: 'Dr. Smith', year: 2024 },
});

// Search with RAG
const results = await sophia.search({
  query: 'What are the key findings about meditation?',
  collections: ['research'],
  limit: 10,
  includeContext: true,
});

// Use results for RAG
for (const hit of results.hits) {
  console.log(`${hit.content} [${hit.citation}]`);
}
```

### Running Locally

```bash
# Start Sophia API
nx serve sophia-api

# Run tests
nx test @sophia/client

# Build all Sophia libraries
nx run-many --target=build --projects=tag:scope:sophia
```

## API Endpoints

| Method | Endpoint            | Description           |
| ------ | ------------------- | --------------------- |
| POST   | `/v1/documents`     | Ingest document       |
| GET    | `/v1/documents/:id` | Get document metadata |
| DELETE | `/v1/documents/:id` | Delete document       |
| POST   | `/v1/search`        | Semantic search       |
| GET    | `/v1/collections`   | List collections      |
| POST   | `/v1/collections`   | Create collection     |
| POST   | `/v1/embeddings`    | Generate embeddings   |

## Events

| Event                      | Description              |
| -------------------------- | ------------------------ |
| `sophia.document.ingested` | Document fully processed |
| `sophia.document.failed`   | Ingestion failed         |
| `sophia.index.updated`     | Collection index rebuilt |
| `sophia.search.completed`  | Search query executed    |

## Configuration

```bash
# Required
SOPHIA_DATABASE_URL=postgresql://...?schema=sophia
REDIS_URL=redis://localhost:6379

# Vector Store
QDRANT_URL=http://localhost:6333
# OR
PINECONE_API_KEY=...
PINECONE_ENVIRONMENT=us-east-1

# Embeddings
OPENAI_API_KEY=sk-...  # For text-embedding-3
# OR
COHERE_API_KEY=...     # For embed-v3

# Storage
DOCUMENT_BUCKET=oshun-documents
```

## Chunking Strategies

| Strategy    | Use Case                               |
| ----------- | -------------------------------------- |
| `fixed`     | Fixed-size chunks (default 512 tokens) |
| `semantic`  | Sentence-based with overlap            |
| `recursive` | Hierarchical splitting                 |
| `document`  | Preserve document structure            |

## Local Development

Start the full development environment:

```bash
# Start infrastructure (PostgreSQL, Redis, MinIO)
docker compose -f docker/docker-compose.dev.yml up -d

# Run database migrations
pnpm nx run @sophia/database:prisma:migrate:dev

# Start the Search API
pnpm nx serve sophia-search-api
```

See [CLAUDE.md](/CLAUDE.md) for detailed local development setup.

## Related Documentation

- [API Reference](/docs/api/sophia/)
- [RAG Best Practices](/docs/guides/rag-best-practices.md)
- [Cross-Domain Integration](/docs/guides/cross-domain-integration.md)
