Eve library

Kuan Yin — Performance Tuning

Guide

Revision 1

On this page

Technical doc 32.16.1.11. Engine in @kuanyin/dharma-analytics (performance-sla.ts, realtime-processing.ts); benchmarks in @kuanyin/evaluation/performance-benchmarks.spec.ts.

Service-level objectives#

Path Target
Intent analysis < 100 ms (p95)
Toxicity fast path < 50 ms
Live-stream protection < 300 ms

slaVerdict(observedMs, targetMs) and computeLatencyPercentiles(samples) compute these; the benchmark suite asserts them against measured latencies.

Hot-path techniques#

  • Progressive analysis — routeAnalysis runs a fast first pass and only escalates uncertain content to deep analysis, keeping the median cheap.
  • Streaming early decision — streamEarlyDecision decides character-by- character so obvious cases resolve before the message is finished.
  • Priority queuing — live-stream content is admitted ahead of non-urgent content; scheduleBatch (EDF) batches non-urgent work.
  • Edge + region routing — routeToRegion picks the lowest-RTT node; model quantization (selectQuantization, fp32→int8/int4) shrinks edge inference.
  • Circuit breaking + degradation — evaluateCircuitBreaker opens on error spikes; degradationPlan(load) sheds deep analysis, then non-critical traffic, to protect the SLA on critical paths.

Caching#

Reputation and health reads are cached (CacheRepository.setWithTTL) with short TTLs and tag/prefix invalidation on the relevant events:

  • invalidate trust:<user> on merit.awarded and trust recompute;
  • invalidate health:<community> on community.alert.

Aim for a high hot-key hit rate; the benchmark suite asserts 100% hits at

10k reads/sec for a hot key set. cdnCacheDecision chooses CDN TTLs for static intervention assets.

Throughput#

The deterministic engines sustain thousands of ops/sec per core. For high-volume message processing, batch by community and reuse the seeded stores (they are module-level and read-only on the hot path). The token-bucket gate (tokenBucketAdmit / sdkCreateRateLimiting) admits requests in O(1).

Watch for#

  • Memory — repositories are in-memory by default; back them with the database adapter in production and cap cache size. The benchmark suite checks that 20k records release fully on clear() (no unbounded growth).
  • GC pauses — keep per-request allocations small; the scoring functions avoid building large intermediate arrays on the hot path.