Status: Final v1.0 Date: 2026-04-26 Owners: eng-iris, eng-isis
Backlog item: V1-P2-0030 Audience: every consumer that calls
IrisEmbeddingClient (psyche memory retrieval, sophia research, iris semantic
memory, veritas citation grounding, …).
1. Decision#
OSHUN's embedding stack is a three-tier cascade:
- Tier 1 — high-quality cloud: OpenAI
text-embedding-3-large(3 072 dim, 8 191 tok input). Default for English-language short-to-medium documents and for any pipeline whose downstream evaluation is calibrated against OpenAI vectors. - Tier 2 — multilingual / long-document cloud: Voyage AI
voyage-3-large(1 024 dim, 32 000 tok input, native multilingual). Default for non-English text, long-form documents, code, and any retrieval surface whose evaluation MUST exceed the nDCG@10 ≥ 0.55 multilingual gate (V1-P2-0035). - Tier 3 — on-prem / local: BAAI
BGE-M3(1 024 dim, 8 192 tok input, native multilingual + multi-granularity). Default when the call must remain inside the tenant's network, when compliance forbids egress, when the cost-per-token quota is exhausted, and for offline batch jobs.
The cascade is not automatic. Consumers explicitly pick a tier when they
call IrisEmbeddingClient.embed() via the tier request field; there is no
silent failover between tiers because dimensions differ (3 072 vs 1 024) and
embeddings from different tiers MUST NOT be co-stored in the same vector index.
2. Rationale#
2.1 Why three tiers, not one#
Public benchmarks (MTEB v1.27, MIRACL, BEIR) consistently show that no single provider dominates across every retrieval surface OSHUN serves:
| Surface | Best provider on public benchmarks (Apr 2026) |
|---|---|
| English short-form retrieval | OpenAI text-embedding-3-large |
| Multilingual / cross-lingual | Voyage voyage-3-large and BGE-M3 trade ~1 nDCG@10 |
| Code retrieval | Voyage voyage-code-3 (out of scope here) |
| Long-form (8 k–32 k tokens) | Voyage voyage-3-large (32 k window) |
| On-prem / no-egress workloads | BGE-M3 (open weights, deploys on a single A10/L40) |
Picking only the cloud provider locks out tenants with strict data residency requirements; picking only the local provider sacrifices quality on the English surfaces that drive most consumer-facing workloads. The three-tier stack lets each consumer optimise the right trade-off without forcing a global compromise.
2.2 Why these specific models within each tier#
OpenAI text-embedding-3-large wins Tier 1 over Cohere embed-v4.0 and
Voyage v3 on English MTEB by ~1.5 nDCG@10. It also supports Matryoshka dimension
truncation (3 072 → 1 024 → 512 → 256), giving callers a graceful cost/quality
knob without a model switch. The 8 191-token input is sufficient for >95% of
OSHUN's chunked passages.
Voyage voyage-3-large wins Tier 2 over Cohere embed-multilingual-v3.0 on
MIRACL (en, es, fr, ar, zh, sw — the languages V1-P2-0035 measures) and accepts
a 32 k-token window, the largest of any frontier embedding model in 2026. Voyage
publishes native multilingual training data including Arabic and Swahili, both
weak points for OpenAI's models. The 1 024-dim output keeps storage cost roughly
1/3 of Tier 1 vectors.
BGE-M3 is the only credible local option whose published nDCG@10 on MIRACL within ±2 points of cloud Tier 2. Its multi-granularity output (dense + sparse + ColBERT) exceeds OSHUN's needs, and operationally it deploys as a single 1.5 GB ONNX model on a single A10/L40 GPU or even a CPU-only fallback for low-volume tenants. The open weights satisfy data-residency / no-egress mandates that the cloud tiers cannot.
2.3 Why NOT cohere or together for Tier 1/2#
Cohere embed-v4.0 is competitive on quality but its API economics are roughly
30% above OpenAI on English and 15% above Voyage on multilingual at OSHUN's
typical batch sizes. Together's hosted embeddings are cost-competitive but lag
on quality at the surface sizes OSHUN ships. Both remain registered as
IEmbeddingProvider implementations in @iris/knowledge-embeddings/providers/
so a tenant can opt in via the tier:'custom' route.
2.4 Why dimensions are NEVER mixed#
A vector index is keyed by dimensionality — a 3 072-dim text-embedding-3-large
vector and a 1 024-dim voyage-3-large vector cannot live in the same index.
Cross-comparison via cosine similarity between vectors of different dimensions
is undefined behaviour. Consumers who change tier MUST re-embed the entire
corpus into a fresh index (the migration path is V1-P2-0033).
3. Default tier per consumer#
| Consumer | Default tier | Reason |
|---|---|---|
@psyche/memory-retrieval (semantic memory) |
Tier 1 | English-dominant; user-facing latency budget is 200 ms. |
@psyche/tavus-knowledge (avatar knowledge) |
Tier 2 | Multilingual content from end-users worldwide; long-form transcripts. |
@iris/integrations/sophia/research-assistant |
Tier 1 | English research corpora; calibrated against OpenAI vectors in evaluation suite. |
@iris/concordia-assistant |
Tier 1 | English; latency-sensitive. |
@oshun/evidence-sophia (citation grounding) |
Tier 2 | Source documents are multilingual; long-document retrieval is the load-bearing path. |
| Local-only / no-egress tenants | Tier 3 | Compliance-driven; performance acceptable for the tenant's volume. |
These defaults are routable via consumer config — a tenant can override any consumer to a different tier. The decision does NOT silently change tiers.
4. API surface (forward-looking)#
IrisEmbeddingClient (built in V1-P2-0031) exposes the following shape:
interface IrisEmbeddingClient {
embed(req: EmbeddingRequest): Promise<EmbeddingResponse>;
embedBatch(req: BatchEmbeddingRequest): Promise<BatchEmbeddingResponse>;
warmCache(
items: readonly { tier: EmbeddingTier; text: string }[]
): Promise<void>;
getStats(): EmbeddingClientStats;
}
type EmbeddingTier = 'tier1_openai' | 'tier2_voyage' | 'tier3_local';
interface EmbeddingRequest {
readonly tier: EmbeddingTier;
readonly text: string;
readonly model?: string; // tier default applies if omitted
readonly dimensions?: number; // matryoshka truncation (Tier 1 only)
readonly tenantId: string;
}
interface EmbeddingResponse {
readonly embedding: number[];
readonly model: string;
readonly dimensions: number;
readonly tokens: number;
readonly costMicrocents: number;
readonly cacheHit: boolean;
readonly latencyMs: number;
}
Existing @iris/knowledge-embeddings continues to ship as the underlying
provider toolkit (per-provider HTTP clients, cache, similarity helpers).
@iris/embeddings (V1-P2-0031) is the canonical consumer-facing facade;
consumers depend on it only.
5. Cost model#
Approximate prices (Apr 2026) per million input tokens:
| Tier | Model | $/M tokens | Encoding /1k tok | Memory @1M docs |
|---|---|---|---|---|
| Tier 1 | text-embedding-3-large 3072 | $0.13 | ~3 072 floats | ~12 GB |
| Tier 1 | text-embedding-3-large 1024 | $0.13 | ~1 024 floats | ~4 GB |
| Tier 2 | voyage-3-large | $0.18 | ~1 024 floats | ~4 GB |
| Tier 3 | BGE-M3 self-hosted | $0 (CPU) | ~1 024 floats | ~4 GB |
The IrisEmbeddingClient records cost in microcents (V1-P2-0034) matching the
canonical pricing unit defined in @oshun/contracts/llm/primitives.
6. Migration & versioning#
When OSHUN upgrades a tier's default model (e.g., from text-embedding-3-large
to a future text-embedding-4):
- The new model is added to the tier's supported-models list.
- A flag (
embeddings.defaultModel.<tier>) flips the default for new ingest only. V1-P2-0033migration utility re-embeds existing corpora in batches; affected vector indices are dual-written until backfill completes; reads switch at completion.
Tier moves (e.g., Tier 1 → Tier 2 for a consumer) are explicit config changes accompanied by a new index — never an in-place re-embed because dimensions change.
7. Verification#
V1-P2-0030 is satisfied by this document. Subsequent items (V1-P2-0031 …
V1-P2-0035) produce the implementation, cache, migration, telemetry, and
multilingual-quality verification. This document is the cross-reference point
for each.
The decision is reviewed at the start of every quarter; any provider swap
requires updating this document and the dependent
embeddings-provider-decision.md change-log section before implementation
begins.