Eve library

Isis LLM Gateway Audit

Guide

Revision 1

On this page

Status: Final v1.0 (audit only — no code changes) Date: 2026-04-26 Owner: eng-isis Scope: libs/isis/llm-providers/ Backlog item: V1-P2-0001 (in docs/releases/v1/specs/todos-p2.md §4)

This document inventories the current @isis/llm-providers library, identifies the canonical entrypoint, catalogues request/response schemas, supported models, cost-per-token tables, retry semantics, and streaming/tool-calling contracts. Where contract surface is absent, that fact is recorded so dependent P2 items (V1-P2-0002 … V1-P2-0027) can fill the gap.

All file references below resolve against libs/isis/llm-providers/src/index.ts and the per-module sources it re-exports.


1. Executive Summary#

@isis/llm-providers is not a gateway client. It is a five-module toolkit that supports a future gateway — registry, rate limiting, token counting, cost accounting, and health/circuit-breaker bookkeeping — but it contains no HTTP/SDK transport, no provider adapters, no streaming reader, no tool-calling pipeline, no auth handling, no failover wiring, and no exported IsisLLMClient symbol.

Concern State Evidence
Canonical entrypoint (IsisLLMClient) Missing index.ts exports only registry/limiter/counter/calculator/monitor symbols; no client class.
Request/response schemas Partial Types only describe registration/usage/cost/health records. No LLMRequest / LLMResponse / LLMStreamChunk / ToolCall / ToolResult.
Provider transport adapters Missing PROVIDER_BASE_URLS exists but there is no fetch/axios/SDK glue.
Streaming contract Missing ModelCapability.STREAMING is declared but no SSE reader, async iterator, or chunk type exists.
Retry semantics Partial Pure functions (calculateExponentialBackoff, parseRetryAfterHeader) exist; no retry executor exists.
Cost tracking Implemented (with unit mismatch) CostCalculator is real and tested, but DEFAULT_MODEL_PRICING (cents/1k) and ModelRegistration.{input,output}PricePer1kTokens (dollars/1k) use different units.
Quota enforcement Sliding-window only Per-tenant quotas, daily/monthly caps, and budget alarms (V1-P2-0014) not modelled.
Health / circuit breaker Implemented ProviderHealthMonitor class and pure helpers.
Token counting Heuristic only Char/word averages with per-provider factors; no tokenizer integration (tiktoken, @anthropic-ai/tokenizer, etc.).
Tool calling Missing ModelCapability.TOOL_USE is declared; pipeline absent.
Vision input Tokenization only Pixel-based token estimate exists; no input-payload schema for image content.
Structured output Missing STRUCTURED_OUTPUT capability declared; no JSON-schema mode plumbing.
OpenTelemetry instrumentation Missing No spans emitted from any module.
Error taxonomy (LLMRateLimitError, LLMQuotaError, …) Missing Errors are thrown as plain Error.

Bottom line: for V1-P2-0002 through V1-P2-0027 the team must (a) define the canonical IsisLLMClient and request/response schemas in @oshun/contracts/llm, (b) build a real transport layer (recorded provider adapters for Anthropic / OpenAI / Google / etc.), (c) wire the existing toolkit modules into that transport, and (d) reconcile the pricing-unit mismatch documented in §6.5.


2. Module Inventory#

The library exports five subsystems via libs/isis/llm-providers/src/index.ts. Each subsystem owns its own folder under src/. Lines below are 1-indexed and cross-reference the actual export blocks.

2.1 Public surface (src/index.ts)#

Block Lines Re-exports from
Type re-exports index.ts:4-32 ./types.js
ID factories index.ts:35-42 ./types.js
Const-object enums index.ts:45-55 ./types.js
Provider registry index.ts:60-85 ./provider-registry/index.js
Rate limiter index.ts:90-112 ./rate-limiter/index.js
Token counter index.ts:117-142 ./token-counter/index.js
Cost calculator index.ts:147-170 ./cost-calculator/index.js
Provider health index.ts:175-197 ./provider-health/index.js

There is no client/ or gateway/ subfolder. There is no factory function named createIsisLLMClient, IsisLLMClient, or similar.

2.2 Type vocabulary (src/types.ts)#

The 11-member provider enum, capability enum, accuracy enum, pricing-tier enum, rate-limit window/action enums, health-status enum, circuit-state enum, and token-type enum are all declared as as const objects (types.ts:42-135).

Branded ID types (types.ts:4-9): ProviderId, ModelId, RateLimiterId, TokenCountId, CostEntryId, HealthCheckId. Counters reset to 0 on module load (types.ts:11-16) — IDs are not stable across processes; this is acceptable for in-process bookkeeping but unfit for persisted records, which V1-P2-0003/V1-P2-0016 must address.

Domain interfaces declared (types.ts:142-374): ModelRegistration, ProviderRegistration, ProviderRegistryConfig, RateLimitRule, RateLimitCheckResult, RateLimitUsage, RateLimiterConfig, TokenCountResult, TokenEstimationFactors, TokenCounterConfig, CostResult, VolumeDiscount, CostSummary, CostCalculatorConfig, HealthCheckResult, CircuitBreakerState, AvailabilitySummary, ProviderHealthConfig, LLMProviderEvents, LLMProviderEventType, LLMProviderEventHandler.

Missing from the vocabulary (must be added in @oshun/contracts/llm per V1-P2-0002):

  • LLMRequest / LLMRequestMessage / LLMRequestOptions
  • LLMResponse / LLMResponseChoice / LLMResponseUsage
  • LLMStreamChunk / LLMStreamEvent (SSE reader contract)
  • LLMToolDefinition / LLMToolCall / LLMToolResult
  • LLMImagePart / LLMTextPart (multimodal content union)
  • LLMStructuredOutputSchema (JSON-schema mode)
  • LLMQuotaSpec / LLMQuotaSnapshot (per-tenant)
  • LLMTrace / LLMTraceSpan (OTel attribute envelope)
  • Error taxonomy: LLMRateLimitError, LLMQuotaError, LLMProviderUnavailableError, LLMTimeoutError, LLMContextLengthExceededError, LLMInvalidRequestError, LLMAuthenticationError, LLMSafetyFilterError.

2.3 Event bus#

LLMProviderEvents (types.ts:380-407) covers eleven event types: model_registered, model_resolved, rate_limit_hit, rate_limit_reset, tokens_counted, cost_recorded, budget_alert, health_check, circuit_opened, circuit_closed, provider_registered.

There is no event for: request-start, request-complete, request-error, streaming-chunk, tool-call-issued, tool-call-completed, failover-triggered, quota-warning. These must be added by V1-P2-0018 (OTel) and V1-P2-0006 (streaming).


3. Provider Registry (src/provider-registry/provider-registry.ts)#

Exported via src/provider-registry/index.ts; consumed in src/index.ts:60-85.

3.1 Public symbols#

  • DEFAULT_PROVIDER_REGISTRY_CONFIG (lines 81-87) — allowDuplicateAliases: false, defaultProvider: ANTHROPIC, maxModelsPerProvider: 50.
  • PROVIDER_DISPLAY_NAMES (lines 90-102) — covers all 11 providers.
  • DEFAULT_PROVIDER_PRIORITY (lines 105-117) — Anthropic 1, OpenAI 2, Google 3, DeepSeek 4, Mistral 5, xAI 6, Cohere 7, Together 8, Groq 9, Ollama 10, Local 11.
  • PROVIDER_BASE_URLS (lines 120-132) — see §6.1.
  • CAPABILITY_REQUIREMENTS (lines 135-144) — eight named capability sets (CHAT_BASIC, CHAT_WITH_TOOLS, VISION_CHAT, CODE_ASSISTANT, REASONING, EMBEDDING, FULL_FEATURED, LONG_CONTEXT_CHAT).
  • Pure helpers: createModelRegistration (153-182), createProviderRegistration (187-209), resolveModelAlias (215-245), filterModelsByCapability (250-259), filterModelsByProvider (264-269), findCheapestModel (275-290), findLargestContextModel (295-308), compareModelPricing (314-318), isModelDeprecated (323-333), findReplacementChain (340-363), buildModelIndex (369-391), validateProviderRegistration (396-531).
  • Class: ProviderRegistry (706-1101). Methods: registerProvider (734-788), registerModel (794-833), resolveModel (839-869), findModels (874-906), getProvider (911-914), getProviders (919-922), getModel (927-930), getModels (935-938), getAliases (943-954), removeProvider (959-966), removeModel (971-986), config get/set, event on/onAll, clear, isDestroyed, destroy.
  • Factories: createProviderRegistry, createPreloadedRegistry (registers the three preloaded providers built in buildPreloadedProviders, lines 537-696), createMinimalRegistry.

3.2 Preloaded model catalogue (lines 537-696)#

Provider Model Aliases Context / Output Input $/1k Output $/1k
Anthropic claude-opus-4-20250514 claude-opus-4, claude-4-opus, opus-4 200 000 / 32 000 0.015 0.075
Anthropic claude-sonnet-4-20250514 claude-sonnet-4, claude-4-sonnet, sonnet-4 200 000 / 16 000 0.003 0.015
Anthropic claude-3-5-haiku-20241022 claude-3-5-haiku, claude-3.5-haiku, haiku-3.5 200 000 / 8 192 0.0008 0.004
Anthropic claude-3-5-sonnet-20241022 (deprecated 2025-10-01 → claude-sonnet-4-20250514) claude-3-5-sonnet, claude-3.5-sonnet, sonnet-3.5 200 000 / 8 192 0.003 0.015
OpenAI gpt-4o-2024-08-06 gpt-4o, gpt4o 128 000 / 16 384 0.0025 0.01
OpenAI gpt-4o-mini-2024-07-18 gpt-4o-mini, gpt4o-mini 128 000 / 16 384 0.00015 0.0006
OpenAI o3-2025-04-16 o3 200 000 / 100 000 0.01 0.04
OpenAI o3-mini-2025-01-31 o3-mini 200 000 / 100 000 0.0011 0.0044
Google gemini-2.5-pro-preview-05-06 gemini-2.5-pro, gemini-pro 1 048 576 / 65 536 0.00125 0.01
Google gemini-2.5-flash-preview-05-20 gemini-2.5-flash, gemini-flash 1 048 576 / 65 536 0.00015 0.0006
Google gemini-2.0-flash gemini-2-flash 1 048 576 / 8 192 0.0001 0.0004

The currency unit on these prices is USD per 1 000 tokens — see §6.5 for the discrepancy with cost-calculator.

The preloaded set covers Anthropic / OpenAI / Google. The other eight providers declared in LLMProviderName (XAI, OLLAMA, MISTRAL, COHERE, TOGETHER, GROQ, DEEPSEEK, LOCAL) have constants in PROVIDER_DISPLAY_NAMES, DEFAULT_PROVIDER_PRIORITY, and PROVIDER_BASE_URLS but no preloaded model entries — V1-P2-0002/V1-P2-0026 should decide whether to expand the preload set or drop unused enum members.

3.3 Resolution algorithm#

resolveModelAlias (lines 215-245) does case-insensitive matching in three passes: exact name, alias, then prefix. Prefix matching is best-effort and returns the first prefix hit, which is non-deterministic if multiple models share a prefix — V1-P2-0002 should specify whether prefix matching is part of the canonical contract or removed for safety.

Class-level resolveModel (lines 839-869) emits model_resolved events but does not rewrite the alias or model id on the call path — consumers using the resolved model still pass the alias through, which means observability breadcrumbs may carry inconsistent modelId values.


4. Rate Limiter (src/rate-limiter/rate-limiter.ts)#

Exported via src/rate-limiter/index.ts; consumed in src/index.ts:90-112.

4.1 Public symbols#

  • DEFAULT_RATE_LIMITER_CONFIG (lines 64-73) — 60 rpm / 100 000 tpm / burst 10 / queue disabled / slidingWindowGranularityMs: 1000.
  • WINDOW_DURATION_MS (lines 76-81) — PER_SECOND: 1 000, PER_MINUTE: 60 000, PER_HOUR: 3 600 000, PER_DAY: 86 400 000.
  • DEFAULT_PROVIDER_LIMITS (lines 84-96) — see §6.2.
  • BACKOFF_MULTIPLIERS (line 99) — [1, 2, 4, 8, 16, 32, 64, 128, 256, 512].
  • MIN_RETRY_AFTER_MS (line 102) — 1 000 ms.
  • Pure helpers: createRateLimitRule (111-128), getWindowDurationMs (133), calculateSlidingWindowUsage (141-154), calculateTokenUsage (160-173), checkRateLimit (179-223), determineAction (229-241), calculateRetryAfter (247-255), parseRetryAfterHeader (262-280), calculateExponentialBackoff (286-298), pruneOldEntries (304-311), calculatePercentUsed (317-320), buildRateLimitUsage (325-347).
  • Class: RateLimiter (379-892). Methods: addRule (404-425), checkLimit (436-496), recordRequest (502-518), getUsage (528-588), getAllUsage (593-604), resetProvider (609-641), getRemainingCapacity (647-715), config get/set, event on/onAll, clear, isDestroyed, destroy.
  • Factories: createRateLimiter, createStrictRateLimiter, createHighThroughputRateLimiter.

4.2 Algorithmic correctness#

  • checkRateLimit (179-223) uses a single-window check; if multiple rules apply (e.g., per-second + per-minute) the class-level checkLimit resolves the most restrictive rule by walking getApplicableRules and returning the worst result (lines 462-480).
  • Burst allowance is treated as additive headroom; when consumption falls in [maxRequests, maxRequests+burstAllowance) the action is THROTTLE, not ALLOW (lines 199-209). Token-burst headroom is computed proportionally (lines 185-189).
  • calculateRetryAfter (lines 247-255) waits until the oldest in-window request expires; this matches sliding-window expectations.
  • parseRetryAfterHeader (262-280) accepts both integer-seconds and HTTP-date formats; integer parsing requires a strict String(asInt) === trimmed check to reject "30s" / "30 ms".
  • calculateExponentialBackoff (286-298) applies jitter via 0.5 + Math.random() * 0.5 — this is the only Math.random call in implementation source and is a deliberate jitter primitive (audited; not a stub).

4.3 Gaps (for V1-P2-0013, V1-P2-0014)#

  • No Retry-After integration on the call path — the limiter exposes parseRetryAfterHeader as a pure helper but does not currently re-record the wait time when a 429 is observed, because there is no provider transport to wire it to.
  • No tenant axis. RateLimitRule keys on (provider, model) only (lines 184-192). Per-tenant request budgets (V1-P2-0014) require a third dimension.
  • Pruning is performed only on recordRequest (line 517); a long-idle period followed by a getUsage call returns stale records inside the window recomputation but no cleanup. Acceptable for memory but worth documenting.

5. Token Counter (src/token-counter/token-counter.ts)#

Exported via src/token-counter/index.ts; consumed in src/index.ts:117-142.

5.1 Public symbols#

  • DEFAULT_TOKEN_COUNTER_CONFIG (lines 83-90) — 4 chars/token, caching enabled, cache size 1 000, TTL 5 minutes, default provider Anthropic.
  • PROVIDER_TOKEN_FACTORS (lines 93-182) — per-provider char/word/image/tool factors for all 11 providers.
  • DEFAULT_TOKEN_ESTIMATION_FACTORS (lines 185-192) — fallback constants.
  • IMAGE_TOKEN_ESTIMATES (lines 195-200) — sized buckets small/medium/large/xlarge.
  • TOOL_DEFINITION_TOKENS (line 203) — fallback per-tool overhead 150.
  • Pure helpers: estimateTokensByChars (213-216), estimateTokensByWords (222-227), estimateTokens (233-239), estimateImageTokens (245-253), estimateToolTokens (259-262), estimateSystemPromptTokens (268-275), countTokensForMessage (281-297), countTokensForMessages (303-318), getProviderFactors (324-326), calculateEstimatedCost (331-342), classifyAccuracy (347-354), buildTokenCountResult (359-380).
  • Class: TokenCounter (419-end). Methods: countTokens, countMessages, countWithImages, plus tool/system overhead helpers, cache management, events, lifecycle.
  • Factories: createTokenCounter, createFastTokenCounter, createPreciseTokenCounter.

5.2 Heuristic note#

estimateTokens blends 0.6 * byChars + 0.4 * byWords (line 238). This is heuristic, not a tokenizer — accuracy on code, JSON, or non-Latin text is poor (often ±25%). V1-P2-0011 and V1-P2-0044 (vision tokenization) should either:

  1. Adopt provider-specific tokenizers (tiktoken, @anthropic-ai/tokenizer, vertex-ai SDK helpers) and downgrade this module to a fallback for unknown providers, or
  2. Document the heuristic accuracy in the canonical contract.

5.3 Cost-unit drift#

calculateEstimatedCost (lines 331-342) treats inputPrice/outputPrice as dollars per 1k tokens and converts to cents via * 100. This is the opposite convention to cost-calculator/cost-calculator.ts:212-227, which treats pricePer1k as cents directly. See §6.5.


6. Cost Calculator (src/cost-calculator/cost-calculator.ts)#

Exported via src/cost-calculator/index.ts; consumed in src/index.ts:147-170.

6.1 Public symbols#

  • DEFAULT_COST_CALCULATOR_CONFIG (lines 30-37) — monthlyBudgetCents: 100_000 (= $1 000), budgetAlertThreshold: 0.75, trackingWindowDays: 30.
  • DEFAULT_MODEL_PRICING (lines 46-94) — see §6.4.
  • VOLUME_DISCOUNT_TIERS (lines 102-185) — five-tier table.
  • BUDGET_ALERT_LEVELS (line 194) — [0.5, 0.75, 0.9, 1.0].
  • COST_PRECISION_DECIMALS (line 199) — 4.
  • Pure helpers: calculateInputCost (212-215), calculateOutputCost (224-227), calculateTotalCost (238-247), applyVolumeDiscount (261-272), findApplicableDiscount (285-298), roundCost (307-310), getModelPricing (321-333), calculateCostPerRequest (342-345), calculateCostPer1kTokens (354-357), projectMonthlyCost (369-376), buildCostSummary (388-448), checkBudgetAlert (461-479), buildCostResult (496-524).
  • Class: CostCalculator (553-end). Methods: calculateCost (589-626), recordCost (636-648), getSummary (658-665), getEntriesByProvider (673-676), getEntriesByModel (684-687), getBudgetStatus (694-714), getProjectedMonthlyCost (721-746), setCustomPricing (755-758), resetTracking (763-767), config get/set, events, lifecycle.
  • Factories: createCostCalculator, createEnterpriseCostCalculator, createBudgetCostCalculator.

6.2 Volume discounts#

Five tiers (PAY_AS_YOU_GO, STANDARD, PROFESSIONAL, ENTERPRISE, ACADEMIC) each with two-to-three monthly-spend thresholds. The most generous tier whose threshold is met wins (findApplicableDiscount, lines 285-298). Discounts are applied at applyVolumeDiscount (lines 261-272) and then again by buildCostResult (lines 496-524) when discount > 0 — read the call sites carefully because applyVolumeDiscount returns a costCents that includes the discount, but buildCostResult recomputes it from originalCost. Both paths converge on the same value but the duplication is fragile and V1-P2-0036 should refactor.

6.3 Budget alerting#

checkBudgetAlert (lines 461-479) iterates BUDGET_ALERT_LEVELS ascending and returns the highest crossed threshold. The class-level checkAndEmitBudgetAlerts (lines 893-913) emits at most once per upward threshold crossing (tracked via lastAlertThreshold).

6.4 Pricing table (cents per 1 000 tokens)#

The DEFAULT_MODEL_PRICING table (lines 46-94) lists 24 models. Excerpt:

Provider Model Input ¢/1k Output ¢/1k
Anthropic claude-opus-4 1.5 7.5
Anthropic claude-sonnet-4 0.3 1.5
Anthropic claude-haiku-3.5 0.08 0.4
OpenAI gpt-4o 0.25 1.0
OpenAI gpt-4o-mini 0.015 0.06
OpenAI o1 1.5 6.0
OpenAI o3-mini 0.11 0.44
Google gemini-2.0-flash 0.01 0.04
Google gemini-1.5-pro 0.125 0.5
DeepSeek deepseek-r1 0.055 0.219
Cohere command-r-plus 0.25 1.0

6.5 Pricing-unit discrepancy (must-fix in V1-P2-0002 / V1-P2-0036)#

Source Field Unit
provider-registry/provider-registry.ts:540-548 (preloaded models) inputPricePer1kTokens / outputPricePer1kTokens USD per 1k tokens (e.g., 0.015 = $0.015)
token-counter/token-counter.ts:331-342 (calculateEstimatedCost) inputPrice / outputPrice USD per 1k tokens — multiplied by 100 to convert to cents
cost-calculator/cost-calculator.ts:46-94 (DEFAULT_MODEL_PRICING) input / output ¢ per 1k tokens (e.g., 1.5 = $0.015)
cost-calculator/cost-calculator.ts:212-227 (calculateInputCost/calculateOutputCost) pricePer1k Treated as cents with no conversion

Result: passing a model registered via createPreloadedRegistry directly into CostCalculator.calculateCost(...) would report a cost one order of magnitude too low unless the model is also resolvable by name in DEFAULT_MODEL_PRICING. The runtime is currently shielded because getModelPricing looks up the model name string, and the preloaded names (claude-opus-4-20250514) don't match DEFAULT_MODEL_PRICING keys (claude-opus-4) — so the call returns null and the caller has to know to supply pricing by name. V1-P2-0002 must standardise the unit (recommend fractional cents per 1 000 tokens as a number) and the canonical contract must enforce it via Zod refinement.


7. Provider Health (src/provider-health/provider-health.ts)#

Exported via src/provider-health/index.ts; consumed in src/index.ts:175-197.

7.1 Public symbols#

  • DEFAULT_PROVIDER_HEALTH_CONFIG (lines 31-41) — failure 5 / success 3 / open 30 s / latency 5 s / error rate 0.5 / history 200 / circuit enabled.
  • DEFAULT_LATENCY_THRESHOLDS (lines 49-61) — Anthropic/OpenAI/Mistral 5 s, Google/Together 4 s, xAI/Cohere/DeepSeek 6 s, Ollama 10 s, Groq 2 s, Local 15 s.
  • HEALTH_STATUS_WEIGHTS (lines 69-75) — HEALTHY: 100, DEGRADED: 60, UNHEALTHY: 20, UNKNOWN: 10, MAINTENANCE: 0.
  • CIRCUIT_BREAKER_DEFAULTS (lines 83-87).
  • PERCENTILE_VALUES (line 92) — [50, 75, 90, 95, 99].
  • Pure helpers: calculateErrorRate (143-146), calculateSuccessRate (155-158), calculateUptimePercent (167-170), calculatePercentile (182-195), classifyHealthStatus (210-225), buildHealthCheckResult (239-279), shouldOpenCircuit (291-294), shouldCloseCircuit (306-309), shouldAttemptHalfOpen (322-330), transitionCircuitState (348-412), calculateAverageLatency (420-427), buildAvailabilitySummary (438-494).
  • Class: ProviderHealthMonitor (520-end). Methods: recordSuccess (548-575), recordFailure (588-618), getHealth (626-635), getCircuitState (646-662), isAvailable (674-700), getAvailabilitySummary (711-716), getAllHealth (723-738), getHealthiestProvider (751-789), resetProvider (799-802), config get/set, events, lifecycle.
  • Factories: createProviderHealthMonitor, createSensitiveHealthMonitor, createResilientHealthMonitor.

7.2 Circuit breaker semantics#

State machine (transitionCircuitState, 348-412):

  • CLOSED + success → reset failure count, increment success count.
  • CLOSED + failure → increment failure count; if ≥ failureThreshold transition to OPEN, set openedAt.
  • OPEN: on any call, if now - openedAt ≥ openDurationMs, transition to HALF_OPEN.
  • HALF_OPEN + success → increment success count; if ≥ successThreshold transition to CLOSED.
  • HALF_OPEN + failure → re-OPEN, reset openedAt.

getHealthiestProvider (751-789) ranks by HEALTH_STATUS_WEIGHTS then by average latency (lower wins). Providers with OPEN circuits past their openDurationMs are not filtered (they would be eligible for half-open trial). The _requiredCapabilities parameter is reserved (line 752) — no integration with ProviderRegistry.findModels exists yet; V1-P2-0008 must wire this.


8. Streaming, Tool Calling, Vision, Structured Output#

All four are declared as capabilities in types.ts:58-69 (STREAMING, TOOL_USE, VISION, STRUCTURED_OUTPUT) and referenced in CAPABILITY_REQUIREMENTS (provider-registry/provider-registry.ts:135-144). No runtime support exists for any of them in this library.

Capability What's missing Resolved by
Streaming SSE reader, async iterator, chunk types, abort propagation V1-P2-0006
Cancellation AbortController plumbing on the call path V1-P2-0007
Failover Multi-provider attempt sequencing, idempotency tracking V1-P2-0008
Quota Tenant axis on RateLimitRule, daily/monthly budgets V1-P2-0009, V1-P2-0014
Structured output JSON-schema → provider-specific encoding V1-P2-0010
Vision Image content-part schema, ingestion pipeline V1-P2-0011
Tool calling Tool def schema, tool-call request/response, tool-result feedback loop V1-P2-0012

9. Retry, Logging, and Telemetry#

9.1 Retry semantics#

The library exposes pure helpers (calculateExponentialBackoff, parseRetryAfterHeader, BACKOFF_MULTIPLIERS) but no class encodes a "retry executor" — there is no method that wraps an HTTP call with configurable retry-on-429, retry-on-5xx, retry-on-network-error, or jittered backoff with per-attempt timeout. V1-P2-0003 must add this on the new IsisLLMClient.

9.2 Structured logging#

Search for log emission yields zero hits — there is no logger import and no console.* usage in implementation files. V1-P2-0016 must:

  • Choose a logger (likely @oshun/observability/logger or pino).
  • Define PII redaction policy and a default redaction set (api_key, authorization, email, phone, request bodies?).
  • Emit at request start / end / error / retry / failover boundaries.

9.3 OpenTelemetry#

No OTel import, no Tracer, no spans. V1-P2-0018 must instrument IsisLLMClient with attributes: llm.provider, llm.model, llm.tenant_id, llm.input_tokens, llm.output_tokens, llm.cost_cents, llm.cache_hit, llm.streamed, llm.tool_calls, error.type.


10. Adversarial Verification Pass#

Per OSHUN_V1_TODOS_P2.md §2.6, the mandatory grep was executed against the audited paths:

bash
grep -rnE "in production[,:]?\\s+(this|we|it|you|the)|\
in a (real|full) (implementation|integration)|\
(would|will) (use|call|invoke|fetch|implement|need|require)|\
simulated runtime|using simulated|\
not yet implemented|TODO:|FIXME:|HACK:|\
placeholder|hardcoded|hard-coded|fake|\
\\bMath\\.random\\b" \
  --include="*.ts" --include="*.tsx" \
  --exclude="*.test.ts" --exclude="*.spec.ts" \
  libs/isis/llm-providers/src/

Result: one hit — libs/isis/llm-providers/src/rate-limiter/rate-limiter.ts:295 — which is the intentional jitter primitive in calculateExponentialBackoff. This is not a stub: jitter is the documented purpose, and the function is unit-tested with a non-zero range assertion. All other source files are clean of stub indicators.

The library passes the §2.6 adversarial gate as a toolkit. It does not satisfy the §2.4 test verification rule for a gateway client because it does not yet contain a gateway client.


11. Cross-references to Downstream P2 Items#

This audit feeds the following P2 work; each item must cite this document in its verification note when re-checking the corresponding V1-* item.

P2 ID Section above Action implied
V1-P2-0002 §2.2, §6.5 Define canonical IsisLLMClient contract in libs/contracts/src/llm/; standardise pricing units.
V1-P2-0003 §3, §4, §6, §7, §9.1 Build real IsisLLMClient wiring registry + limiter + counter + calculator + monitor; remove simulate* fallbacks in downstream consumers.
V1-P2-0004 §2.2 Add LLMProviderTestDouble recording calls and returning scripted responses.
V1-P2-0005 §6, §4 Document cost-tracking + quota enforcement contract (cents per 1k tokens, tenant axis).
V1-P2-0006 §8 SSE reader and LLMStreamChunk async iterator.
V1-P2-0007 §8 AbortController plumbing through the request path.
V1-P2-0008 §7, §8 Failover ordering using ProviderHealthMonitor.getHealthiestProvider.
V1-P2-0009 §4.3 Tenant axis on RateLimitRule; typed quota errors.
V1-P2-0010 §8 Structured output / JSON schema mode.
V1-P2-0011 §8 Vision input schema + token estimation hook.
V1-P2-0012 §8 Tool/function calling pipeline.
V1-P2-0013 §4.3 Honor Retry-After on the call path.
V1-P2-0014 §4.3, §6.3 Per-tenant request budget alarms.
V1-P2-0015 §2.2 Prompt-injection-defence layer wrapping user-supplied content.
V1-P2-0016 §9.2 Structured logging with PII redaction.
V1-P2-0017 §2.2 Per-call feature-flag override / model pinning.
V1-P2-0018 §9.3 OpenTelemetry spans per call.
V1-P2-0019 §2.2 Error-type taxonomy.
V1-P2-0020 §6.5 Contract test that IsisLLMClient honours pricing-unit and budget alarm.

12. Verification Summary#

  • Sources read: libs/isis/llm-providers/src/index.ts (full), types.ts (full), provider-registry/provider-registry.ts (full), rate-limiter/rate-limiter.ts (full), token-counter/token-counter.ts (full through line ~600 — interfaces and pure utilities; class methods inspected via spot-check), cost-calculator/cost-calculator.ts (full), provider-health/provider-health.ts (full).
  • Adversarial grep: clean except for one audited jitter primitive (§10).
  • Cross-references in this document use path:line form so future re-verification can grep the citations directly.
  • Pricing-unit discrepancy (§6.5) and missing canonical IsisLLMClient (§1, §2.1) are the two largest findings; both block multiple downstream P2 items and are routed to V1-P2-0002 / V1-P2-0003 / V1-P2-0036.

This audit completes V1-P2-0001. The corresponding box in docs/releases/v1/specs/todos-p2.md:144-148 may now be marked [x] with this document as the verification note.