Status: Final v1.0 (audit only — no code changes) Date: 2026-04-26
Owner: eng-isis Scope: libs/isis/llm-providers/ Backlog item:
V1-P2-0001 (in docs/releases/v1/specs/todos-p2.md §4)
This document inventories the current @isis/llm-providers library, identifies
the canonical entrypoint, catalogues request/response schemas, supported models,
cost-per-token tables, retry semantics, and streaming/tool-calling contracts.
Where contract surface is absent, that fact is recorded so dependent P2
items (V1-P2-0002 … V1-P2-0027) can fill the gap.
All file references below resolve against libs/isis/llm-providers/src/index.ts
and the per-module sources it re-exports.
1. Executive Summary#
@isis/llm-providers is not a gateway client. It is a five-module
toolkit that supports a future gateway — registry, rate limiting, token
counting, cost accounting, and health/circuit-breaker bookkeeping — but it
contains no HTTP/SDK transport, no provider adapters, no streaming reader, no
tool-calling pipeline, no auth handling, no failover wiring, and no exported
IsisLLMClient symbol.
| Concern | State | Evidence |
|---|---|---|
Canonical entrypoint (IsisLLMClient) |
Missing | index.ts exports only registry/limiter/counter/calculator/monitor symbols; no client class. |
| Request/response schemas | Partial | Types only describe registration/usage/cost/health records. No LLMRequest / LLMResponse / LLMStreamChunk / ToolCall / ToolResult. |
| Provider transport adapters | Missing | PROVIDER_BASE_URLS exists but there is no fetch/axios/SDK glue. |
| Streaming contract | Missing | ModelCapability.STREAMING is declared but no SSE reader, async iterator, or chunk type exists. |
| Retry semantics | Partial | Pure functions (calculateExponentialBackoff, parseRetryAfterHeader) exist; no retry executor exists. |
| Cost tracking | Implemented (with unit mismatch) | CostCalculator is real and tested, but DEFAULT_MODEL_PRICING (cents/1k) and ModelRegistration.{input,output}PricePer1kTokens (dollars/1k) use different units. |
| Quota enforcement | Sliding-window only | Per-tenant quotas, daily/monthly caps, and budget alarms (V1-P2-0014) not modelled. |
| Health / circuit breaker | Implemented | ProviderHealthMonitor class and pure helpers. |
| Token counting | Heuristic only | Char/word averages with per-provider factors; no tokenizer integration (tiktoken, @anthropic-ai/tokenizer, etc.). |
| Tool calling | Missing | ModelCapability.TOOL_USE is declared; pipeline absent. |
| Vision input | Tokenization only | Pixel-based token estimate exists; no input-payload schema for image content. |
| Structured output | Missing | STRUCTURED_OUTPUT capability declared; no JSON-schema mode plumbing. |
| OpenTelemetry instrumentation | Missing | No spans emitted from any module. |
Error taxonomy (LLMRateLimitError, LLMQuotaError, …) |
Missing | Errors are thrown as plain Error. |
Bottom line: for V1-P2-0002 through V1-P2-0027 the team must (a) define
the canonical IsisLLMClient and request/response schemas in
@oshun/contracts/llm, (b) build a real transport layer (recorded provider
adapters for Anthropic / OpenAI / Google / etc.), (c) wire the existing toolkit
modules into that transport, and (d) reconcile the pricing-unit mismatch
documented in §6.5.
2. Module Inventory#
The library exports five subsystems via libs/isis/llm-providers/src/index.ts.
Each subsystem owns its own folder under src/. Lines below are 1-indexed and
cross-reference the actual export blocks.
2.1 Public surface (src/index.ts)#
| Block | Lines | Re-exports from |
|---|---|---|
| Type re-exports | index.ts:4-32 |
./types.js |
| ID factories | index.ts:35-42 |
./types.js |
| Const-object enums | index.ts:45-55 |
./types.js |
| Provider registry | index.ts:60-85 |
./provider-registry/index.js |
| Rate limiter | index.ts:90-112 |
./rate-limiter/index.js |
| Token counter | index.ts:117-142 |
./token-counter/index.js |
| Cost calculator | index.ts:147-170 |
./cost-calculator/index.js |
| Provider health | index.ts:175-197 |
./provider-health/index.js |
There is no client/ or gateway/ subfolder. There is no factory function
named createIsisLLMClient, IsisLLMClient, or similar.
2.2 Type vocabulary (src/types.ts)#
The 11-member provider enum, capability enum, accuracy enum, pricing-tier enum,
rate-limit window/action enums, health-status enum, circuit-state enum, and
token-type enum are all declared as as const objects (types.ts:42-135).
Branded ID types (types.ts:4-9): ProviderId, ModelId, RateLimiterId,
TokenCountId, CostEntryId, HealthCheckId. Counters reset to 0 on module
load (types.ts:11-16) — IDs are not stable across processes; this is
acceptable for in-process bookkeeping but unfit for persisted records, which
V1-P2-0003/V1-P2-0016 must address.
Domain interfaces declared (types.ts:142-374): ModelRegistration,
ProviderRegistration, ProviderRegistryConfig, RateLimitRule,
RateLimitCheckResult, RateLimitUsage, RateLimiterConfig,
TokenCountResult, TokenEstimationFactors, TokenCounterConfig,
CostResult, VolumeDiscount, CostSummary, CostCalculatorConfig,
HealthCheckResult, CircuitBreakerState, AvailabilitySummary,
ProviderHealthConfig, LLMProviderEvents, LLMProviderEventType,
LLMProviderEventHandler.
Missing from the vocabulary (must be added in @oshun/contracts/llm per
V1-P2-0002):
LLMRequest/LLMRequestMessage/LLMRequestOptionsLLMResponse/LLMResponseChoice/LLMResponseUsageLLMStreamChunk/LLMStreamEvent(SSE reader contract)LLMToolDefinition/LLMToolCall/LLMToolResultLLMImagePart/LLMTextPart(multimodal content union)LLMStructuredOutputSchema(JSON-schema mode)LLMQuotaSpec/LLMQuotaSnapshot(per-tenant)LLMTrace/LLMTraceSpan(OTel attribute envelope)- Error taxonomy:
LLMRateLimitError,LLMQuotaError,LLMProviderUnavailableError,LLMTimeoutError,LLMContextLengthExceededError,LLMInvalidRequestError,LLMAuthenticationError,LLMSafetyFilterError.
2.3 Event bus#
LLMProviderEvents (types.ts:380-407) covers eleven event types:
model_registered, model_resolved, rate_limit_hit, rate_limit_reset,
tokens_counted, cost_recorded, budget_alert, health_check,
circuit_opened, circuit_closed, provider_registered.
There is no event for: request-start, request-complete, request-error,
streaming-chunk, tool-call-issued, tool-call-completed, failover-triggered,
quota-warning. These must be added by V1-P2-0018 (OTel) and V1-P2-0006
(streaming).
3. Provider Registry (src/provider-registry/provider-registry.ts)#
Exported via src/provider-registry/index.ts; consumed in src/index.ts:60-85.
3.1 Public symbols#
DEFAULT_PROVIDER_REGISTRY_CONFIG(lines 81-87) —allowDuplicateAliases: false,defaultProvider: ANTHROPIC,maxModelsPerProvider: 50.PROVIDER_DISPLAY_NAMES(lines 90-102) — covers all 11 providers.DEFAULT_PROVIDER_PRIORITY(lines 105-117) — Anthropic 1, OpenAI 2, Google 3, DeepSeek 4, Mistral 5, xAI 6, Cohere 7, Together 8, Groq 9, Ollama 10, Local 11.PROVIDER_BASE_URLS(lines 120-132) — see §6.1.CAPABILITY_REQUIREMENTS(lines 135-144) — eight named capability sets (CHAT_BASIC,CHAT_WITH_TOOLS,VISION_CHAT,CODE_ASSISTANT,REASONING,EMBEDDING,FULL_FEATURED,LONG_CONTEXT_CHAT).- Pure helpers:
createModelRegistration(153-182),createProviderRegistration(187-209),resolveModelAlias(215-245),filterModelsByCapability(250-259),filterModelsByProvider(264-269),findCheapestModel(275-290),findLargestContextModel(295-308),compareModelPricing(314-318),isModelDeprecated(323-333),findReplacementChain(340-363),buildModelIndex(369-391),validateProviderRegistration(396-531). - Class:
ProviderRegistry(706-1101). Methods:registerProvider(734-788),registerModel(794-833),resolveModel(839-869),findModels(874-906),getProvider(911-914),getProviders(919-922),getModel(927-930),getModels(935-938),getAliases(943-954),removeProvider(959-966),removeModel(971-986), config get/set, eventon/onAll,clear,isDestroyed,destroy. - Factories:
createProviderRegistry,createPreloadedRegistry(registers the three preloaded providers built inbuildPreloadedProviders, lines 537-696),createMinimalRegistry.
3.2 Preloaded model catalogue (lines 537-696)#
| Provider | Model | Aliases | Context / Output | Input $/1k | Output $/1k |
|---|---|---|---|---|---|
| Anthropic | claude-opus-4-20250514 |
claude-opus-4, claude-4-opus, opus-4 |
200 000 / 32 000 | 0.015 | 0.075 |
| Anthropic | claude-sonnet-4-20250514 |
claude-sonnet-4, claude-4-sonnet, sonnet-4 |
200 000 / 16 000 | 0.003 | 0.015 |
| Anthropic | claude-3-5-haiku-20241022 |
claude-3-5-haiku, claude-3.5-haiku, haiku-3.5 |
200 000 / 8 192 | 0.0008 | 0.004 |
| Anthropic | claude-3-5-sonnet-20241022 (deprecated 2025-10-01 → claude-sonnet-4-20250514) |
claude-3-5-sonnet, claude-3.5-sonnet, sonnet-3.5 |
200 000 / 8 192 | 0.003 | 0.015 |
| OpenAI | gpt-4o-2024-08-06 |
gpt-4o, gpt4o |
128 000 / 16 384 | 0.0025 | 0.01 |
| OpenAI | gpt-4o-mini-2024-07-18 |
gpt-4o-mini, gpt4o-mini |
128 000 / 16 384 | 0.00015 | 0.0006 |
| OpenAI | o3-2025-04-16 |
o3 |
200 000 / 100 000 | 0.01 | 0.04 |
| OpenAI | o3-mini-2025-01-31 |
o3-mini |
200 000 / 100 000 | 0.0011 | 0.0044 |
gemini-2.5-pro-preview-05-06 |
gemini-2.5-pro, gemini-pro |
1 048 576 / 65 536 | 0.00125 | 0.01 | |
gemini-2.5-flash-preview-05-20 |
gemini-2.5-flash, gemini-flash |
1 048 576 / 65 536 | 0.00015 | 0.0006 | |
gemini-2.0-flash |
gemini-2-flash |
1 048 576 / 8 192 | 0.0001 | 0.0004 |
The currency unit on these prices is USD per 1 000 tokens — see §6.5 for the
discrepancy with cost-calculator.
The preloaded set covers Anthropic / OpenAI / Google. The other eight providers
declared in LLMProviderName (XAI, OLLAMA, MISTRAL, COHERE, TOGETHER,
GROQ, DEEPSEEK, LOCAL) have constants in PROVIDER_DISPLAY_NAMES,
DEFAULT_PROVIDER_PRIORITY, and PROVIDER_BASE_URLS but no preloaded model
entries — V1-P2-0002/V1-P2-0026 should decide whether to expand the preload
set or drop unused enum members.
3.3 Resolution algorithm#
resolveModelAlias (lines 215-245) does case-insensitive matching in three
passes: exact name, alias, then prefix. Prefix matching is best-effort and
returns the first prefix hit, which is non-deterministic if multiple models
share a prefix — V1-P2-0002 should specify whether prefix matching is part of
the canonical contract or removed for safety.
Class-level resolveModel (lines 839-869) emits model_resolved events but
does not rewrite the alias or model id on the call path — consumers using
the resolved model still pass the alias through, which means observability
breadcrumbs may carry inconsistent modelId values.
4. Rate Limiter (src/rate-limiter/rate-limiter.ts)#
Exported via src/rate-limiter/index.ts; consumed in src/index.ts:90-112.
4.1 Public symbols#
DEFAULT_RATE_LIMITER_CONFIG(lines 64-73) — 60 rpm / 100 000 tpm / burst 10 / queue disabled /slidingWindowGranularityMs: 1000.WINDOW_DURATION_MS(lines 76-81) —PER_SECOND: 1 000,PER_MINUTE: 60 000,PER_HOUR: 3 600 000,PER_DAY: 86 400 000.DEFAULT_PROVIDER_LIMITS(lines 84-96) — see §6.2.BACKOFF_MULTIPLIERS(line 99) —[1, 2, 4, 8, 16, 32, 64, 128, 256, 512].MIN_RETRY_AFTER_MS(line 102) — 1 000 ms.- Pure helpers:
createRateLimitRule(111-128),getWindowDurationMs(133),calculateSlidingWindowUsage(141-154),calculateTokenUsage(160-173),checkRateLimit(179-223),determineAction(229-241),calculateRetryAfter(247-255),parseRetryAfterHeader(262-280),calculateExponentialBackoff(286-298),pruneOldEntries(304-311),calculatePercentUsed(317-320),buildRateLimitUsage(325-347). - Class:
RateLimiter(379-892). Methods:addRule(404-425),checkLimit(436-496),recordRequest(502-518),getUsage(528-588),getAllUsage(593-604),resetProvider(609-641),getRemainingCapacity(647-715), config get/set, eventon/onAll,clear,isDestroyed,destroy. - Factories:
createRateLimiter,createStrictRateLimiter,createHighThroughputRateLimiter.
4.2 Algorithmic correctness#
checkRateLimit(179-223) uses a single-window check; if multiple rules apply (e.g., per-second + per-minute) the class-levelcheckLimitresolves the most restrictive rule by walkinggetApplicableRulesand returning the worst result (lines 462-480).- Burst allowance is treated as additive headroom; when consumption falls in
[maxRequests, maxRequests+burstAllowance)the action isTHROTTLE, notALLOW(lines 199-209). Token-burst headroom is computed proportionally (lines 185-189). calculateRetryAfter(lines 247-255) waits until the oldest in-window request expires; this matches sliding-window expectations.parseRetryAfterHeader(262-280) accepts both integer-seconds and HTTP-date formats; integer parsing requires a strictString(asInt) === trimmedcheck to reject"30s"/"30 ms".calculateExponentialBackoff(286-298) applies jitter via0.5 + Math.random() * 0.5— this is the onlyMath.randomcall in implementation source and is a deliberate jitter primitive (audited; not a stub).
4.3 Gaps (for V1-P2-0013, V1-P2-0014)#
- No
Retry-Afterintegration on the call path — the limiter exposesparseRetryAfterHeaderas a pure helper but does not currently re-record the wait time when a 429 is observed, because there is no provider transport to wire it to. - No tenant axis.
RateLimitRulekeys on(provider, model)only (lines 184-192). Per-tenant request budgets (V1-P2-0014) require a third dimension. - Pruning is performed only on
recordRequest(line 517); a long-idle period followed by agetUsagecall returns stale records inside the window recomputation but no cleanup. Acceptable for memory but worth documenting.
5. Token Counter (src/token-counter/token-counter.ts)#
Exported via src/token-counter/index.ts; consumed in src/index.ts:117-142.
5.1 Public symbols#
DEFAULT_TOKEN_COUNTER_CONFIG(lines 83-90) — 4 chars/token, caching enabled, cache size 1 000, TTL 5 minutes, default provider Anthropic.PROVIDER_TOKEN_FACTORS(lines 93-182) — per-provider char/word/image/tool factors for all 11 providers.DEFAULT_TOKEN_ESTIMATION_FACTORS(lines 185-192) — fallback constants.IMAGE_TOKEN_ESTIMATES(lines 195-200) — sized bucketssmall/medium/large/xlarge.TOOL_DEFINITION_TOKENS(line 203) — fallback per-tool overhead150.- Pure helpers:
estimateTokensByChars(213-216),estimateTokensByWords(222-227),estimateTokens(233-239),estimateImageTokens(245-253),estimateToolTokens(259-262),estimateSystemPromptTokens(268-275),countTokensForMessage(281-297),countTokensForMessages(303-318),getProviderFactors(324-326),calculateEstimatedCost(331-342),classifyAccuracy(347-354),buildTokenCountResult(359-380). - Class:
TokenCounter(419-end). Methods:countTokens,countMessages,countWithImages, plus tool/system overhead helpers, cache management, events, lifecycle. - Factories:
createTokenCounter,createFastTokenCounter,createPreciseTokenCounter.
5.2 Heuristic note#
estimateTokens blends 0.6 * byChars + 0.4 * byWords (line 238). This is
heuristic, not a tokenizer — accuracy on code, JSON, or non-Latin text is
poor (often ±25%). V1-P2-0011 and V1-P2-0044 (vision tokenization) should
either:
- Adopt provider-specific tokenizers (
tiktoken,@anthropic-ai/tokenizer,vertex-aiSDK helpers) and downgrade this module to a fallback for unknown providers, or - Document the heuristic accuracy in the canonical contract.
5.3 Cost-unit drift#
calculateEstimatedCost (lines 331-342) treats inputPrice/outputPrice as
dollars per 1k tokens and converts to cents via * 100. This is the
opposite convention to cost-calculator/cost-calculator.ts:212-227, which
treats pricePer1k as cents directly. See §6.5.
6. Cost Calculator (src/cost-calculator/cost-calculator.ts)#
Exported via src/cost-calculator/index.ts; consumed in src/index.ts:147-170.
6.1 Public symbols#
DEFAULT_COST_CALCULATOR_CONFIG(lines 30-37) —monthlyBudgetCents: 100_000(= $1 000),budgetAlertThreshold: 0.75,trackingWindowDays: 30.DEFAULT_MODEL_PRICING(lines 46-94) — see §6.4.VOLUME_DISCOUNT_TIERS(lines 102-185) — five-tier table.BUDGET_ALERT_LEVELS(line 194) —[0.5, 0.75, 0.9, 1.0].COST_PRECISION_DECIMALS(line 199) —4.- Pure helpers:
calculateInputCost(212-215),calculateOutputCost(224-227),calculateTotalCost(238-247),applyVolumeDiscount(261-272),findApplicableDiscount(285-298),roundCost(307-310),getModelPricing(321-333),calculateCostPerRequest(342-345),calculateCostPer1kTokens(354-357),projectMonthlyCost(369-376),buildCostSummary(388-448),checkBudgetAlert(461-479),buildCostResult(496-524). - Class:
CostCalculator(553-end). Methods:calculateCost(589-626),recordCost(636-648),getSummary(658-665),getEntriesByProvider(673-676),getEntriesByModel(684-687),getBudgetStatus(694-714),getProjectedMonthlyCost(721-746),setCustomPricing(755-758),resetTracking(763-767), config get/set, events, lifecycle. - Factories:
createCostCalculator,createEnterpriseCostCalculator,createBudgetCostCalculator.
6.2 Volume discounts#
Five tiers (PAY_AS_YOU_GO, STANDARD, PROFESSIONAL, ENTERPRISE,
ACADEMIC) each with two-to-three monthly-spend thresholds. The most generous
tier whose threshold is met wins (findApplicableDiscount, lines 285-298).
Discounts are applied at applyVolumeDiscount (lines 261-272) and then
again by buildCostResult (lines 496-524) when discount > 0 — read the call
sites carefully because applyVolumeDiscount returns a costCents that
includes the discount, but buildCostResult recomputes it from originalCost.
Both paths converge on the same value but the duplication is fragile and
V1-P2-0036 should refactor.
6.3 Budget alerting#
checkBudgetAlert (lines 461-479) iterates BUDGET_ALERT_LEVELS ascending and
returns the highest crossed threshold. The class-level
checkAndEmitBudgetAlerts (lines 893-913) emits at most once per upward
threshold crossing (tracked via lastAlertThreshold).
6.4 Pricing table (cents per 1 000 tokens)#
The DEFAULT_MODEL_PRICING table (lines 46-94) lists 24 models. Excerpt:
| Provider | Model | Input ¢/1k | Output ¢/1k |
|---|---|---|---|
| Anthropic | claude-opus-4 |
1.5 | 7.5 |
| Anthropic | claude-sonnet-4 |
0.3 | 1.5 |
| Anthropic | claude-haiku-3.5 |
0.08 | 0.4 |
| OpenAI | gpt-4o |
0.25 | 1.0 |
| OpenAI | gpt-4o-mini |
0.015 | 0.06 |
| OpenAI | o1 |
1.5 | 6.0 |
| OpenAI | o3-mini |
0.11 | 0.44 |
gemini-2.0-flash |
0.01 | 0.04 | |
gemini-1.5-pro |
0.125 | 0.5 | |
| DeepSeek | deepseek-r1 |
0.055 | 0.219 |
| Cohere | command-r-plus |
0.25 | 1.0 |
6.5 Pricing-unit discrepancy (must-fix in V1-P2-0002 / V1-P2-0036)#
| Source | Field | Unit |
|---|---|---|
provider-registry/provider-registry.ts:540-548 (preloaded models) |
inputPricePer1kTokens / outputPricePer1kTokens |
USD per 1k tokens (e.g., 0.015 = $0.015) |
token-counter/token-counter.ts:331-342 (calculateEstimatedCost) |
inputPrice / outputPrice |
USD per 1k tokens — multiplied by 100 to convert to cents |
cost-calculator/cost-calculator.ts:46-94 (DEFAULT_MODEL_PRICING) |
input / output |
¢ per 1k tokens (e.g., 1.5 = $0.015) |
cost-calculator/cost-calculator.ts:212-227 (calculateInputCost/calculateOutputCost) |
pricePer1k |
Treated as cents with no conversion |
Result: passing a model registered via createPreloadedRegistry directly into
CostCalculator.calculateCost(...) would report a cost one order of magnitude
too low unless the model is also resolvable by name in
DEFAULT_MODEL_PRICING. The runtime is currently shielded because
getModelPricing looks up the model name string, and the preloaded names
(claude-opus-4-20250514) don't match DEFAULT_MODEL_PRICING keys
(claude-opus-4) — so the call returns null and the caller has to know to
supply pricing by name. V1-P2-0002 must standardise the unit (recommend
fractional cents per 1 000 tokens as a number) and the canonical contract
must enforce it via Zod refinement.
7. Provider Health (src/provider-health/provider-health.ts)#
Exported via src/provider-health/index.ts; consumed in src/index.ts:175-197.
7.1 Public symbols#
DEFAULT_PROVIDER_HEALTH_CONFIG(lines 31-41) — failure 5 / success 3 / open 30 s / latency 5 s / error rate 0.5 / history 200 / circuit enabled.DEFAULT_LATENCY_THRESHOLDS(lines 49-61) — Anthropic/OpenAI/Mistral 5 s, Google/Together 4 s, xAI/Cohere/DeepSeek 6 s, Ollama 10 s, Groq 2 s, Local 15 s.HEALTH_STATUS_WEIGHTS(lines 69-75) —HEALTHY: 100,DEGRADED: 60,UNHEALTHY: 20,UNKNOWN: 10,MAINTENANCE: 0.CIRCUIT_BREAKER_DEFAULTS(lines 83-87).PERCENTILE_VALUES(line 92) —[50, 75, 90, 95, 99].- Pure helpers:
calculateErrorRate(143-146),calculateSuccessRate(155-158),calculateUptimePercent(167-170),calculatePercentile(182-195),classifyHealthStatus(210-225),buildHealthCheckResult(239-279),shouldOpenCircuit(291-294),shouldCloseCircuit(306-309),shouldAttemptHalfOpen(322-330),transitionCircuitState(348-412),calculateAverageLatency(420-427),buildAvailabilitySummary(438-494). - Class:
ProviderHealthMonitor(520-end). Methods:recordSuccess(548-575),recordFailure(588-618),getHealth(626-635),getCircuitState(646-662),isAvailable(674-700),getAvailabilitySummary(711-716),getAllHealth(723-738),getHealthiestProvider(751-789),resetProvider(799-802), config get/set, events, lifecycle. - Factories:
createProviderHealthMonitor,createSensitiveHealthMonitor,createResilientHealthMonitor.
7.2 Circuit breaker semantics#
State machine (transitionCircuitState, 348-412):
CLOSED + success→ reset failure count, increment success count.CLOSED + failure→ increment failure count; if≥ failureThresholdtransition toOPEN, setopenedAt.OPEN: on any call, ifnow - openedAt ≥ openDurationMs, transition toHALF_OPEN.HALF_OPEN + success→ increment success count; if≥ successThresholdtransition toCLOSED.HALF_OPEN + failure→ re-OPEN, resetopenedAt.
getHealthiestProvider (751-789) ranks by HEALTH_STATUS_WEIGHTS then by
average latency (lower wins). Providers with OPEN circuits past their
openDurationMs are not filtered (they would be eligible for half-open
trial). The _requiredCapabilities parameter is reserved (line 752) — no
integration with ProviderRegistry.findModels exists yet; V1-P2-0008 must
wire this.
8. Streaming, Tool Calling, Vision, Structured Output#
All four are declared as capabilities in types.ts:58-69 (STREAMING,
TOOL_USE, VISION, STRUCTURED_OUTPUT) and referenced in
CAPABILITY_REQUIREMENTS (provider-registry/provider-registry.ts:135-144).
No runtime support exists for any of them in this library.
| Capability | What's missing | Resolved by |
|---|---|---|
| Streaming | SSE reader, async iterator, chunk types, abort propagation | V1-P2-0006 |
| Cancellation | AbortController plumbing on the call path |
V1-P2-0007 |
| Failover | Multi-provider attempt sequencing, idempotency tracking | V1-P2-0008 |
| Quota | Tenant axis on RateLimitRule, daily/monthly budgets |
V1-P2-0009, V1-P2-0014 |
| Structured output | JSON-schema → provider-specific encoding | V1-P2-0010 |
| Vision | Image content-part schema, ingestion pipeline | V1-P2-0011 |
| Tool calling | Tool def schema, tool-call request/response, tool-result feedback loop | V1-P2-0012 |
9. Retry, Logging, and Telemetry#
9.1 Retry semantics#
The library exposes pure helpers (calculateExponentialBackoff,
parseRetryAfterHeader, BACKOFF_MULTIPLIERS) but no class encodes a "retry
executor" — there is no method that wraps an HTTP call with configurable
retry-on-429, retry-on-5xx, retry-on-network-error, or jittered backoff with
per-attempt timeout. V1-P2-0003 must add this on the new IsisLLMClient.
9.2 Structured logging#
Search for log emission yields zero hits — there is no logger import and no
console.* usage in implementation files. V1-P2-0016 must:
- Choose a logger (likely
@oshun/observability/loggerorpino). - Define PII redaction policy and a default redaction set (
api_key,authorization,email,phone, request bodies?). - Emit at request start / end / error / retry / failover boundaries.
9.3 OpenTelemetry#
No OTel import, no Tracer, no spans. V1-P2-0018 must instrument
IsisLLMClient with attributes: llm.provider, llm.model, llm.tenant_id,
llm.input_tokens, llm.output_tokens, llm.cost_cents, llm.cache_hit,
llm.streamed, llm.tool_calls, error.type.
10. Adversarial Verification Pass#
Per OSHUN_V1_TODOS_P2.md §2.6, the mandatory grep was executed against the
audited paths:
grep -rnE "in production[,:]?\\s+(this|we|it|you|the)|\
in a (real|full) (implementation|integration)|\
(would|will) (use|call|invoke|fetch|implement|need|require)|\
simulated runtime|using simulated|\
not yet implemented|TODO:|FIXME:|HACK:|\
placeholder|hardcoded|hard-coded|fake|\
\\bMath\\.random\\b" \
--include="*.ts" --include="*.tsx" \
--exclude="*.test.ts" --exclude="*.spec.ts" \
libs/isis/llm-providers/src/
Result: one hit —
libs/isis/llm-providers/src/rate-limiter/rate-limiter.ts:295 — which is the
intentional jitter primitive in calculateExponentialBackoff. This is not a
stub: jitter is the documented purpose, and the function is unit-tested with a
non-zero range assertion. All other source files are clean of stub
indicators.
The library passes the §2.6 adversarial gate as a toolkit. It does not satisfy the §2.4 test verification rule for a gateway client because it does not yet contain a gateway client.
11. Cross-references to Downstream P2 Items#
This audit feeds the following P2 work; each item must cite this document in its
verification note when re-checking the corresponding V1-* item.
| P2 ID | Section above | Action implied |
|---|---|---|
V1-P2-0002 |
§2.2, §6.5 | Define canonical IsisLLMClient contract in libs/contracts/src/llm/; standardise pricing units. |
V1-P2-0003 |
§3, §4, §6, §7, §9.1 | Build real IsisLLMClient wiring registry + limiter + counter + calculator + monitor; remove simulate* fallbacks in downstream consumers. |
V1-P2-0004 |
§2.2 | Add LLMProviderTestDouble recording calls and returning scripted responses. |
V1-P2-0005 |
§6, §4 | Document cost-tracking + quota enforcement contract (cents per 1k tokens, tenant axis). |
V1-P2-0006 |
§8 | SSE reader and LLMStreamChunk async iterator. |
V1-P2-0007 |
§8 | AbortController plumbing through the request path. |
V1-P2-0008 |
§7, §8 | Failover ordering using ProviderHealthMonitor.getHealthiestProvider. |
V1-P2-0009 |
§4.3 | Tenant axis on RateLimitRule; typed quota errors. |
V1-P2-0010 |
§8 | Structured output / JSON schema mode. |
V1-P2-0011 |
§8 | Vision input schema + token estimation hook. |
V1-P2-0012 |
§8 | Tool/function calling pipeline. |
V1-P2-0013 |
§4.3 | Honor Retry-After on the call path. |
V1-P2-0014 |
§4.3, §6.3 | Per-tenant request budget alarms. |
V1-P2-0015 |
§2.2 | Prompt-injection-defence layer wrapping user-supplied content. |
V1-P2-0016 |
§9.2 | Structured logging with PII redaction. |
V1-P2-0017 |
§2.2 | Per-call feature-flag override / model pinning. |
V1-P2-0018 |
§9.3 | OpenTelemetry spans per call. |
V1-P2-0019 |
§2.2 | Error-type taxonomy. |
V1-P2-0020 |
§6.5 | Contract test that IsisLLMClient honours pricing-unit and budget alarm. |
12. Verification Summary#
- Sources read:
libs/isis/llm-providers/src/index.ts(full),types.ts(full),provider-registry/provider-registry.ts(full),rate-limiter/rate-limiter.ts(full),token-counter/token-counter.ts(full through line ~600 — interfaces and pure utilities; class methods inspected via spot-check),cost-calculator/cost-calculator.ts(full),provider-health/provider-health.ts(full). - Adversarial grep: clean except for one audited jitter primitive (§10).
- Cross-references in this document use
path:lineform so future re-verification can grep the citations directly. - Pricing-unit discrepancy (§6.5) and missing canonical
IsisLLMClient(§1, §2.1) are the two largest findings; both block multiple downstream P2 items and are routed toV1-P2-0002/V1-P2-0003/V1-P2-0036.
This audit completes V1-P2-0001. The corresponding box in
docs/releases/v1/specs/todos-p2.md:144-148 may now be marked [x] with this
document as the verification note.