Eve library

Isis Deterministic-Mode Contract

Guide

Revision 1

On this page

Status: v1.0 Date: 2026-04-26 Owner: eng-isis Backlog item: V1-P2-0021

This document defines what callers can rely on when requesting deterministic generation through IsisLLMClient. Determinism in LLM inference is provider-bounded — the gateway can guarantee its own contribution is deterministic, but the upstream model's reproducibility depends on the provider's API surface and infrastructure.


1. Caller-side contract (gateway guarantees)#

Given two LLMRequest objects with byte-identical:

  • failover.primary.{provider, model}
  • messages (including content order and parts)
  • system
  • tools (if any)
  • structuredOutput (if any)
  • sampling.temperature
  • sampling.topP, sampling.topK
  • sampling.seed
  • sampling.stopSequences
  • sampling.maxOutputTokens
  • sampling.presencePenalty, sampling.frequencyPenalty
  • sampling.reasoningEffort

IsisLLMClient.complete() MUST emit a byte-identical request body to the upstream provider on every call. The verification fixture in libs/isis/llm-providers/src/client/llm-client.test.ts (V1-P2-0021 suite) issues five identical requests and asserts the captured fetch bodies are equal.

Fields that intentionally vary across calls — and therefore are NOT part of the determinism boundary — include:

  • requestId (UUID minted per call by the gateway)
  • traceId if not supplied by the caller
  • idempotencyKey (caller-supplied; absent by default)
  • metadata (caller-controlled; sent verbatim if set)
  • HTTP-level timing fields like Date, User-Agent (set by the fetch implementation)

2. Provider-side determinism#

For each supported provider, deterministic mode requires:

Provider Deterministic settings Notes
Anthropic temperature=0, fixed top_p, fixed top_k (optional) Anthropic does not currently expose a request seed. Identical requests may still produce slightly varying outputs across versions of the underlying model server. Pin to a versioned model id (e.g., claude-sonnet-4-20250514) to maximise reproducibility.
OpenAI temperature=0, seed=N OpenAI honors seed per their API docs; identical seed + temperature 0 produces the same output for the SAME system_fingerprint (returned in the response). When system_fingerprint changes, identity is not guaranteed.
Google (Gemini) temperature=0, topP, topK Gemini does not expose a request seed in the public REST API. Reproducibility is best-effort under temperature 0.

The gateway documents these guarantees but cannot enforce upstream reproducibility. Callers depending on byte-identical OUTPUT must pin to OpenAI with a fixed seed and assert system_fingerprint stability across calls.


3. Cost / quota bookkeeping#

Deterministic mode does not affect cost or quota arithmetic. Identical requests still consume one quota unit per call and accumulate in the ledger normally.


4. Verification fixture#

V1-P2-0021 is verified by the test suite IsisLLMClient — V1-P2-0021 deterministic-mode contract in libs/isis/llm-providers/src/client/llm-client.test.ts. The fixture:

  1. Builds a buildRequest(...) envelope with temperature=0, fixed seed=42, topP=0.95, topK=40, fixed maxOutputTokens, and fixed messages.
  2. Issues five complete() calls back-to-back against a fetch mock.
  3. Asserts the five captured request bodies (the bytes sent to the provider) are pairwise identical.

The mock returns the same recorded response on every call so no upstream non-determinism is involved; the test isolates the gateway's own determinism contract.