# ADR-0002: RAG Pipeline Architecture

**Date**: 2025-09-20  
**Status**: Accepted  
**Deciders**: AI Platform Team

## Context

Lilith AI teachers need to provide accurate, cited responses grounded in
authoritative sources. Pure LLM responses risk hallucination and lack
verifiability. We need a system that:

- Grounds responses in source material
- Provides citations for claims
- Handles multi-tradition knowledge
- Maintains low latency

## Decision

Implement a **Retrieval-Augmented Generation (RAG)** pipeline with:

1. **Hybrid Search**: BM25 (lexical) + Dense embeddings (semantic)
2. **Reciprocal Rank Fusion**: Combine retrieval results
3. **Cross-Encoder Re-ranking**: Precision optimization
4. **Citation Extraction**: Map claims to sources

### Architecture

```
Query → Expansion → Hybrid Search → RRF → Re-rank → Top-K → LLM → Citations
```

## Rationale

### Why Hybrid Search

- Lexical: Exact term matching (important for spiritual terms)
- Semantic: Meaning-based matching
- Combination outperforms either alone

### Why Reciprocal Rank Fusion

- Score-agnostic combination
- Robust to different ranking scales
- Simple and effective

### Why Cross-Encoder Re-ranking

- More accurate than bi-encoders
- Applied only to top candidates (manageable latency)

## Alternatives Considered

1. **Dense-only retrieval**: Missed exact term matches
2. **Keyword search only**: Missed semantic similarity
3. **LLM-only (no RAG)**: Hallucination risk too high

## Consequences

### Positive

- Grounded, accurate responses
- Verifiable citations
- Reduced hallucination

### Negative

- Additional latency (~150ms)
- Infrastructure complexity
- Index maintenance overhead

### Performance

- Recall@10: 0.92
- MRR: 0.85
- Latency (p50): 150ms

## Related

- [Algorithms - RAG Pipeline](../algorithms.md#1-rag-pipeline)
- Sophia Search API: `apps/sophia/search-api/`
