Eve SOTA embedding and reranker pricing — eve.embedding-pricing.v1

• Task: 3.2
• Evaluated: 2026-09-05
• Decision basis: pricing-and-posture
• Chosen: perplexity/pplx-embed-v1-0.6b
• Registry leg: embedding — bound
• Probe spend: $0.28906573 over 903 cost-reported requests
• Record digest: 86122d349529f9682e6f0f8b38c1939489c29d1e4203b24a97d7347e8fcdd219

What was measured, and why a catalogue was not enough#

OpenRouter's default model listing omits embedding and rerank models entirely, so the candidate set was ASKED for (/models?output_modalities=embeddings, /models?output_modalities=rerank) rather than guessed: 37 embedding candidates and 7 rerankers. Every price below is usage.cost returned by a completed call, never pricing.prompt from the catalogue — one reranker in this table lists its prompt price as 0 and bills a real amount.

The corpus is exact: 56,970 chunks, 36,147,926 characters, longest chunk 1205 characters (319 tokens at the widest tokenizer measured). Token volume is projected by a ratio estimator over disjoint calibration batches, with its interval.

Admitted candidates, cheapest first#

20 of 37 candidates were admitted; 17 were refused.

Refused, with the measurement that refused them#

The identity floor#

Not a relevance measurement — task 3.5 owns those. This is the bar the cost rule implies: the rule binds "the cheapest model that CAN DO THE JOB", so a price ranking with no capability bar underneath it would rank models that cannot retrieve at all. Each candidate must put a page named by its own identifier above 200 chunks drawn from pages the label does not point at, using only the corpus-identifier labels task 3.1 settled without any ranker.

Several of these models are asymmetric by design and publish a query-side prefix. A model called without the one it documents is a model called wrong, so each is measured under every protocol its family documents AND under plain text as the control.

The floor uses 2 corpus-identifier queries against 200 distractor chunks. Its protocols: plain (no prefix; the control every candidate is measured under); e5-query-passage (intfloat E5 family model cards: "query: " and "passage: " prefixes); bge-en-instruction (BAAI bge-*-en-v1.5 model cards: a query-side retrieval instruction); qwen3-instruct (Qwen3-Embedding model cards: an instruction-templated query side).

Provider data posture#

Posture is measured by asking the router for it. A request that routes had an endpoint meeting the filter; one that 404s had none, which the router could only know by checking every endpoint.

Reranking, priced against what it would be added to#

deferred to task 3.4 — priced here and not adopted; the measured per-query cost is the reason it needs a quality argument before it is added, not a footnote.

One row bills nothing, and a zero there is a price, not a posture: every :free route in the embedding field was refused because no endpoint of it accepts data_collection: deny, and the refusal named "Free model training". A free reranker earns the same check before it is adopted.

The binding#

perplexity/pplx-embed-v1-0.6b is bound because it is the cheapest candidate that cleared every measured gate at 10,000 queries/month — $0.0040 per million tokens, 1024 dimensions, served by Perplexity. The runner-up is baai/bge-base-en-v1.5, $0.3352 per month dearer at that volume.

The binding is the slug, the endpoint set AND the query protocol plain — an index built under one convention and queried under another is two systems, not one.

An endpoint pin is still applied: this slug has a single endpoint, so no agreement between endpoints was measured and none is claimed; the pin names the endpoint that was.

Rollback#

• Slug override: OSHUN_ASSISTANT_EMBEDDING_MODEL (registry supplies the default underneath it)
• Endpoint override: OPENROUTER_PROVIDER_ONLY
• Degraded mode: lexical-only — apps/oshun/bff/src/assistant/docs-search.ts
• Rolling back invalidates stored vectors: true

What task 3.5 must measure#

Task 3.2 binds on price alone, and a price-only pin handed to 3.5 would leave it measuring one arm chosen by cost. The shortlist below is the admitted price/width frontier — for every admitted candidate NOT on it, some admitted candidate is at least as cheap and at least as wide. Dimensions are a capacity fact here, not a claim about retrieval.

• perplexity/pplx-embed-v1-0.6b
• qwen/qwen3-embedding-8b

What re-opens this binding#

• task 3.5 measures relevance and a different candidate wins on a paired test
• task 3.7 declines to promote dense retrieval at all, which retires this leg
• the bound endpoint set changes, because vectors are only comparable within one endpoint
• a re-priced route moves the chosen candidate out of first place at the recorded volume
• the corpus chunker changes, because the calibrated tokens-per-character no longer applies
• the query-side protocol changes, because an index built under one convention cannot be queried under another
• the bound slug gains or loses an endpoint whose quantization differs from the measured one
• the vendor re-points a versioned slug, because a vendor-versioned name is a pin only while the vendor keeps it one

Honest limits#

• This binds on PRICE and POSTURE. No relevance measurement exists yet: task 3.5 owns Recall@k and nDCG over the full set with a pre-registered paired test, and task 3.7 owns promotion. A candidate that clears the identity floor has been shown able to retrieve, not shown to retrieve well.
• The identity floor uses only the relevance set's corpus-identifier labels, because those are the ones that settle without any ranker. Four of the nine query classes still carry no labels at all (task 3.1), so no floor exists for conceptual, multi-hop, recency or contradictory queries.
• Corpus token counts are a ratio estimator over disjoint sampled batches, not an exact count. The corpus CHARACTER count is exact; buying an exact token count would mean embedding all 36 million characters for every candidate.
• Monthly figures are scenarios. This repository records no search_docs call rate, so the per-query and per-index prices are the measured facts and the monthly totals are arithmetic over a volume a reader chooses.
• Latency is wall-clock from this host on this date, over a residential-grade path to one region. It is a floor for planning, not a service-level measurement.
• Posture is measured as ROUTING behaviour — whether an endpoint meeting a filter exists — not as an audit of any provider's actual practice.
• Nothing is indexed. Task 3.3 builds the index, and until it does the bound slug has no production consumer; search_docs remains lexical BM25F.
• Prices are quoted for text as the corpus stores it. A binding whose query protocol adds a prefix costs a few tokens more per query and, where the protocol also prefixes documents, roughly two per cent more per index build; neither shifts the ranking between candidates at these magnitudes, and the protocol is recorded so the difference is recomputable.
• Each price is the single usage.cost figure the route returned for one call, retained verbatim in the probe record beside its token count. This record therefore distinguishes no cached from uncached rate, and nothing here should be read as a claim that one exists or does not.
• The identity floor is measured under the protocols this tool knows about. A model documenting a calling convention nobody here recognised is measured plain and would be under-measured exactly as the instruction-tuned models were before their protocols were added.