Eve library

ADR-0002: RAG Pipeline Architecture

Guide

Revision 1

On this page

Date: 2025-09-20
Status: Accepted
Deciders: AI Platform Team

Context#

Lilith AI teachers need to provide accurate, cited responses grounded in authoritative sources. Pure LLM responses risk hallucination and lack verifiability. We need a system that:

  • Grounds responses in source material
  • Provides citations for claims
  • Handles multi-tradition knowledge
  • Maintains low latency

Decision#

Implement a Retrieval-Augmented Generation (RAG) pipeline with:

  1. Hybrid Search: BM25 (lexical) + Dense embeddings (semantic)
  2. Reciprocal Rank Fusion: Combine retrieval results
  3. Cross-Encoder Re-ranking: Precision optimization
  4. Citation Extraction: Map claims to sources

Architecture#

text
Query → Expansion → Hybrid Search → RRF → Re-rank → Top-K → LLM → Citations

Rationale#

  • Lexical: Exact term matching (important for spiritual terms)
  • Semantic: Meaning-based matching
  • Combination outperforms either alone

Why Reciprocal Rank Fusion#

  • Score-agnostic combination
  • Robust to different ranking scales
  • Simple and effective

Why Cross-Encoder Re-ranking#

  • More accurate than bi-encoders
  • Applied only to top candidates (manageable latency)

Alternatives Considered#

  1. Dense-only retrieval: Missed exact term matches
  2. Keyword search only: Missed semantic similarity
  3. LLM-only (no RAG): Hallucination risk too high

Consequences#

Positive#

  • Grounded, accurate responses
  • Verifiable citations
  • Reduced hallucination

Negative#

  • Additional latency (~150ms)
  • Infrastructure complexity
  • Index maintenance overhead

Performance#

  • Recall@10: 0.92
  • MRR: 0.85
  • Latency (p50): 150ms