Eve library

Psyche — Systems Deep Dive

Guide

Revision 1

On this page

The libs/psyche/ area: ~133 Nx projects that make up the Psyche hyper-realistic AI virtual-assistant platform — a Python service foundation plus a large fleet of TypeScript libraries for avatar rendering, voice, perception, memory, agentic computer-use, conferencing, translation, the Tavus CVI integration, and infrastructure/observability.

What this area is#

Psyche is the "AI virtual assistant" domain: a realtime, multi-modal avatar that can see, listen, speak, remember, drive a computer, and join video calls. Unlike most Oshun domains this is not one library — it is ~133 separately-tracked Nx projects rooted at libs/psyche/, every one with its own project.json. They split cleanly into two technology tiers, which is the first thing to understand about the area.

The two tiers#

A Python service foundation (lang:python tags) provides the cross-cutting runtime plumbing every Psyche backend service shares: psyche-auth, psyche-cache, psyche-database, psyche-logging, psyche-messaging, psyche-state-sync, psyche-storage, psyche-tracing, and the catch-all psyche-common. Each is a real Poetry-packaged Python library under src/psyche_* and is explicitly "aligned with" its TypeScript counterpart in the shared @oshun/* libraries (e.g. psyche-logging mirrors @oshun/logging), giving cross-language parity. psyche-platform itself is the root Poetry/pytest aggregator (libs/psyche/pyproject.toml, conftest.py, packages = []) — it ships no code of its own but owns shared test config and declares the Nx implicitDependencies that bind the foundation and many feature libs into one affected graph.

A large TypeScript feature fleet (the remaining ~120 type:lib projects) implements the actual assistant capabilities as standalone, mostly dependency-light libraries: each is a src/index.ts barrel over a set of domain-specific modules, with Zod schemas at the edges and deterministic algorithms inside (FACS action units, saccade main-sequence kinematics, BM25 + RRF hybrid retrieval, BLEU/METEOR/TER, Dawid-Skene aggregation, Holt-Winters forecasting, MemGPT-style paging, and so on). These are genuine implementations, not CRUD shells; the per-library @example blocks in each barrel show the public API.

How the feature libraries relate#

The TS fleet is organised by capability sub-system: avatar rendering (avatar-*, 3D Gaussian Splatting + FLAME), voice/speech (voice-*, speech-*, viseme-generator, noise-handling), facial/non-verbal perception and behavior (face-*, emotion-*, gaze-*, head-*, gesture-system, posture-system, *-expressions, behavior-*), conversation/engagement/proactivity (dialogue-manager, engagement-*, proactive-*, *-triggers), hierarchical memory (memory-*), agentic computer-use (computer-use-core, browser-automation, screen-analysis, sandbox, action-safety, tool-*), knowledge/RAG (knowledge-*, sophia-*-integration), conferencing/meetings (conferencing-core + platform adapters + participant-*/meeting-*), translation/localization (translation-*, cultural-adaptation, dialect-handling, language-detection), the Tavus CVI integration (@psyche/tavus-*), and an infrastructure/observability cluster (k8s-*, *-alerting, *-anomaly, log-metrics, latency-analysis, etc.).

These libraries compose: perception libs feed signals into emotion-engine and engagement-detector; viseme-generator/avatar-lipsync drive avatar-core; memory-* and knowledge-* share embedding/retrieval patterns; and the Tavus fleet and Psyche-native avatar/perception stacks are bridged by @psyche/tavus-hybrid.

How it fits the wider system#

Consumers are Psyche's own services and the BFF. The Python foundation is imported by Psyche backend services for auth/db/cache/messaging/observability; the TS feature libs are composed by the realtime avatar/conferencing runtimes and by agent loops. Several libs are explicitly integration seams to other Oshun domains: psyche-avatar-isis-integration bridges @isis/3d-generation, psyche-sophia-integration/psyche-sophia-search-integration and psyche-memory-embeddings bridge Sophia's ingestion/search/embedding stack, and the conferencing adapters wrap third-party SDKs (Zoom, Teams, Meet, Webex, Recall.ai, Tavus). Many libraries are designed around injectable provider boundaries — e.g. conferencing-core's createMockConferenceSession, memory-archival's createMockEmbeddingProvider, the speech/TTS provider abstractions — so the deterministic logic is unit-testable offline while real SDK/credentialed providers are supplied in production. Walk the "used by" edges on any node below to see exactly who depends on it.

Entity reference#

psyche-platform#

Root Poetry/pytest aggregator for the Python tier (libs/psyche/pyproject.toml, conftest.py, sourceRoot: libs/psyche); ships no package of its own (packages = []) but owns shared test/lint config and the Nx implicitDependencies that wire the foundation and feature libs into one graph.

psyche-auth#

Python authentication library (src/psyche_auth) aligned with @oshun/auth-primitives: JWT signing/verification, in-memory + Redis session stores, API-key management, Argon2/bcrypt/PBKDF2 password utilities, and FastAPI auth middleware (per its README.md).

psyche-cache#

Python caching library (src/psyche_cache) aligned with @oshun/cache: LRU+TTL in-memory cache, Redis distributed cache, namespaced key building, pub/sub state sync, Redlock-compatible distributed locking, and standard TTL presets.

psyche-common#

The largest Python foundation lib (src/common, ~16.7K LOC): shared utilities across all Psyche services — GPU optimization (quantization, batch inference, TensorRT, memory management), structured logging, success-metrics tracking (human-likeness, conversation quality, cost), performance/streaming pipelines, and horizontal-scaling helpers (per README.md).

psyche-database#

Dual-language data layer: a Python async client (src/psyche_database, PostgreSQL via asyncpg, Redis, transactions, migrations, query builder, SQLAlchemy models, Qdrant vector storage) plus a Prisma-generated TypeScript client under src/generated/client/ (models such as AvatarPack, VoiceProfile). Aligned with @oshun/database.

psyche-logging#

Python structured-logging library (src/psyche_logging) aligned with @oshun/logging: JSON log schema, contextvars-based context propagation, log sampling strategies, PII redaction, optional OpenTelemetry trace enrichment, and child loggers.

psyche-messaging#

Python event bus (src/psyche_messaging) aligned with @oshun/event-bus: pattern/wildcard pub-sub, event persistence with replay TTL, retry with exponential/linear/fixed backoff, dead-letter queue, correlation tracking, and cross-domain event routing.

psyche-state-sync#

Python realtime state-synchronization library (src/psyche_state_sync) for assistant sessions: avatar state (position/rotation, 52 FACS blendshapes, gaze, animations, visemes), voice state, session lifecycle/emotional state, and multi-participant conferencing state with turn-taking and screen sharing.

psyche-storage#

Python S3/MinIO-compatible storage client (src/psyche_storage) aligned with @oshun/storage: async operations, multipart uploads with progress, pre-signed URLs, and specialized asset managers for avatar models, voice recordings, and knowledge documents.

psyche-tracing#

Python distributed-tracing library (src/psyche_tracing) aligned with @oshun/tracing: OpenTelemetry SDK + OTLP export, W3C traceparent/tracestate propagation, instrumentation decorators, AWS X-Ray header support, and branded trace/span ID types.

psyche-avatar-core#

3D Gaussian Splatting avatar rendering core (src/index.ts): FLAME parametric model utilities (neutral state, expression mapping, coefficient blending, pose), viseme/lip-sync application, and the shared avatar type surface for the rest of the avatar-* fleet.

psyche-avatar-cache#

Tiered avatar caching for fast switching/preloading: memory + persistent cache entries, expression caching, preloading, and statistics/event types, for low-latency avatar operations (src/index.ts).

psyche-avatar-expressions#

Expression blendshapes and FACS mapping for avatars: ARKit blendshapes, FACS action units, emotion presets, micro-expressions, transitions, asymmetry, and a controller-state surface with Zod schemas (src/index.ts).

psyche-avatar-isis-integration#

Integration layer bridging @isis/3d-generation and psyche-avatar-core: text-to-3D and photo-to-3D avatar generation requests/results, blend-shape mapping/conversion between the two systems, and an avatar generator (src/index.ts). A cross-domain seam.

psyche-avatar-lipsync#

Lip-sync/viseme animation: a 19-class viseme system with 31 blendshape parameters, ARPAbet/IPA phoneme-to-viseme mapping, coarticulation blending, and a real-time StreamingAligner with lookahead buffering targeting <33ms latency (src/index.ts).

psyche-avatar-quality#

Adaptive quality system for avatar rendering: GPU detection, quality levels, dynamic resolution scaling, and Gaussian-count adaptation with performance/Gaussian types and Zod schemas (src/index.ts).

psyche-avatar-taa#

Temporal anti-aliasing for Gaussian-splatting avatars: jitter generation, motion vectors, history accumulation, and ghost reduction for stable temporal output, with vector/matrix/color helper types (src/index.ts).

psyche-avatar-training#

Avatar training pipeline (monocular video → 3D Gaussian Splatting): video preprocessing orchestration, FLAME parameter fitting, Gaussian-splat training, blendshape optimization, and expression calibration (src/index.ts).

psyche-voice-synthesis#

TTS library with ElevenLabs and Cartesia providers (src/index.ts): streaming, voice cloning, SSML/prosody control, and a provider abstraction with voice settings, audio-output, and error types.

psyche-voice-streaming#

Streaming TTS playback (src/index.ts): audio chunking, buffer management, network-quality adaptation, and streaming-session/playback/event types for low-latency voice output.

psyche-voice-consistency#

Voice consistency for natural speech (src/index.ts): voice-identity preservation via profile management, emotion-appropriate prosody with transitions, context-aware speaking-rate adaptation, and automatic consistency corrections.

psyche-voice-dubbing#

Target-language voice dubbing (src/index.ts): speaker detection, voice mapping, prosody transfer, timing alignment, multi-language synthesis, quality control, and project management with metrics.

psyche-viseme-generator#

Text-to-viseme prediction with audio-aligned timing (src/index.ts): grapheme-to-phoneme conversion, viseme timing, optional coarticulation, and a textToVisemes convenience plus a configurable generator.

psyche-speech-recognition#

Speech-to-text library (src/index.ts): multi-provider support (Deepgram, Whisper, WebSpeech), voice-activity detection, streaming and batch transcription, word-level timestamps, and multi-language support.

psyche-speech-translation#

Full speech-to-speech translation pipeline (src/index.ts): audio → STT → text translation → TTS → audio, with multi-speaker voice mapping, prosody transfer, realtime streaming, translation memory, glossary, and quality monitoring.

psyche-noise-handling#

Audio pre-processing for voice (src/index.ts): noise suppression, echo cancellation, and audio normalization, composable individually or via a unified createAudioProcessor operating on sample buffers.

psyche-face-detection#

Face detection and landmark tracking (src/index.ts): realtime detection, the 478-point MediaPipe face mesh, IoU-based tracking with smoothing, head-pose estimation, and a pluggable detection backend.

psyche-face-analysis#

Facial analysis on landmarks (src/index.ts): FACS action-unit detection, Ekman + extended expression classification, the valence-arousal-dominance circumplex, age/gender estimation, and temporal smoothing.

psyche-emotion-recognition#

Realtime multi-face emotion pipeline (src/index.ts): per-face emotion tracking with persistent identity, temporal pattern + micro-expression analysis, dominant emotion calculation, suppression/genuineness detection, and quality scoring.

psyche-emotion-engine#

Emotion modeling for avatar animation (src/index.ts): primary/extended emotions, the Russell circumplex (valence-arousal), compatibility-based blending, emotion-specific transitions, mood persistence/contamination, social display rules, and FACS/blendshape output.

psyche-facs-expressions#

Facial Action Coding System library (src/index.ts): action-unit definitions, intensity control, AU-to-emotion mapping, blendshape combinations, and expression state management for facial animation.

psyche-micro-expressions#

Micro-expression generation (src/index.ts): brief involuntary expressions (40–200ms), natural blink patterns with variability, eye-moisture/tearing simulation, subtle tics/twitches/saccades, and suppression/leakage and deception simulation.

psyche-gaze-estimation#

Eye-gaze estimation from landmarks (src/index.ts): eye-region extraction, iris/pupil tracking, pitch/yaw gaze direction with head-pose correction, calibrated screen mapping, exponential+Kalman filtering, and fixation/saccade/blink and AOI analysis.

psyche-gaze-control#

Biologically-grounded gaze generation (src/index.ts, cites saccade main-sequence and conversational-gaze research): saccades with main-sequence kinematics, context-aware fixations, microsaccade/drift, smooth pursuit, vergence, conversation-aware coordination, cultural profiles, and ARKit output.

psyche-gaze-awareness-behaviors#

Gaze-awareness behaviors (src/index.ts): maps gaze points/targets onto content and derives awareness-driven behaviors, with Zod input schemas for gaze points, targets, and configuration.

psyche-head-pose-estimation#

Head-pose estimation from landmarks (src/index.ts): PnP-based pose with a geometric fallback, multi-format landmark support (MediaPipe/dlib/OpenPose), movement detection (nods/shakes/tilts/turns), stability and range-of-motion tracking, and fatigue indicators.

psyche-head-movement#

Generative head movement (src/index.ts): conversational nods, disagreement shakes, interest/confusion/empathy/thinking tilts, speech-synced emphasis, idle micro-sway/drift, stimulus responses, and cultural/personality adaptation.

psyche-gesture-system#

Gesture generation/animation (src/index.ts): 54+ predefined gesture types (beats, emblematic, pointing, iconic, metaphoric), keyframe animation with easing, priority-based multi-gesture blending, speech-synced beats, cultural adaptation by locale, and personality variation.

psyche-posture-system#

Posture animation (src/index.ts): breathing, weight shifting, and social mirroring, exposing the core posture types and a posture controller.

psyche-behavior-coordinator#

Priority-based behavior coordination for avatar animation (src/index.ts): behavior definitions with categories/priorities/durations, a priority queue with conflict resolution, fade-in/out timing, multi-channel output blending, and state machines via a BehaviorOrchestrator.

psyche-behavior-prediction#

User-behavior prediction (src/index.ts): Markov-chain transition modeling, sequence pattern mining, temporal pattern detection, ensemble prediction, accuracy tracking, and analytics, with Zod event/config schemas.

@psyche/behavior-anomaly-detector#

Behavioral anomaly detection for meeting participants (src/index.ts): inactivity detection, erratic-behavior patterns, conflict-escalation, cross-participant analysis, and meeting-atmosphere scoring with a default config.

@psyche/uncanny-valley#

Uncanny-valley detection/scoring for avatar quality (src/index.ts): per-category collectors (blink, gaze, expression, head/body movement, response timing, lip-sync, visual quality) scored against DEFAULT_BASELINES human baselines with weighted aggregation into an UncannyValleyScore.

psyche-human-likeness-estimation#

Automated human-likeness estimation via multi-modal signal fusion (src/index.ts): behavioral, conversational, emotional, linguistic, and consistency signals with feature-weighted scoring, artifact detection, ground-truth calibration, trend analysis, A/B testing, and benchmarking.

psyche-dialogue-manager#

Dialogue management for natural conversation flow (src/index.ts): turn-taking (floor control, overlap, push-to-talk), interruption detection/handling, conversation-state tracking, intent detection, and multi-participant support in an event-driven design.

psyche-conversation-sentiment-tracking#

Conversation-level sentiment tracking via signal fusion (src/index.ts): aggregates facial/vocal/text/engagement/atmosphere signals into smoothed temporal sentiment arcs with turning-point detection, intervention recommendations, phase analysis, speaker congruence, and cross-conversation comparison.

@psyche/attention-tracker#

Attention tracking/analysis (src/index.ts): attention-state tracking, shift and lapse detection with recovery, collective multi-participant attention, attention heatmaps, engagement scoring, topic-attention correlation, pattern detection, and fatigue recommendations.

psyche-engagement-detector#

Multi-dimensional engagement detection (src/index.ts): gaze, behavioral, conversational, temporal, and content engagement combined with configurable weights into smoothed scores, trend detection, and signal detection (attention loss, interest spikes, fatigue) with session history.

psyche-engagement-recovery#

Disengagement detection and recovery orchestration (src/index.ts): a disengagement detector feeding recovery-strategy selection, with Zod schemas for engagement snapshots and recovery config.

@psyche/context-monitoring#

Realtime context monitoring (src/index.ts): signal collection/aggregation, pattern and anomaly detection, state tracking, threshold-based alerting, and analytics, with Zod schemas for signals and configuration.

psyche-anticipatory-responses#

Anticipatory response modeling (src/index.ts): an anticipation modeler over incoming signals (with AnticipationSignalSchema) to pre-compute likely responses, plus utility/constant helpers.

psyche-follow-up-initiation#

Follow-up initiation (src/index.ts): types and logic for deciding when and how the assistant should proactively initiate a follow-up. A compact lib (one barrel over a types module and engine).

psyche-proactive-suggestions#

Proactive suggestion engine (src/index.ts): signal analysis, trigger-based suggestion generation, suppression/deduplication, user-feedback tracking, pattern learning, and analytics, with Zod schemas.

psyche-proactive-summarization#

Proactive content summarization (src/index.ts): a content collector feeding trigger detection, format selection, user-preference learning, and delivery orchestration, with ContentItemInputSchema/config schemas.

psyche-proactive-assistance-triggers#

Proactive-assistance trigger detection (src/index.ts): a need detector over indicator inputs (NeedIndicatorInputSchema) plus constants, utilities, and factory functions for firing assistance offers.

psyche-help-offer-triggers#

Help-offer triggering (src/index.ts): a help-signal aggregator over HelpSignalInputSchema inputs with HelpOfferConfigSchema to decide when to surface help. A small, focused trigger lib.

psyche-memory-core#

MemGPT-style hierarchical memory (src/index.ts): multi-tier storage with automatic paging and capacity management via a VirtualContextManager (add, search, context-window assembly, background monitoring). The anchor of the memory-* cluster.

psyche-memory-in-context#

8K-token in-context memory (src/index.ts): per-message extraction of key facts, topics, and entities, with buildContext assembling an optimized, token-estimated context for the LLM.

psyche-memory-working#

32K-token working memory (src/index.ts): LRU eviction with auto promotion/ demotion between tiers, importance-weighted entries, and statistics via a WorkingMemoryManager.

psyche-memory-archival#

Unlimited archival memory (src/index.ts): vector-based semantic retrieval, importance/tagging, and consolidation, built around an injectable embedding provider (createMockEmbeddingProvider for offline tests; real providers in production).

psyche-memory-consolidation#

Memory consolidation (src/index.ts): time-based decay (exponential, power-law, stepped), multi-factor importance scoring, similar-memory merging, low-value pruning, and automated consolidation scheduling.

psyche-memory-persistence#

Cross-session memory persistence (src/index.ts): multi-backend storage (memory, file, PostgreSQL, Redis, S3), gzip/brotli compression, AES-256-GCM encryption at rest, serialization, session management, and integrity verification.

psyche-memory-embeddings#

Bridge between Psyche memory and @sophia/indexing embeddings (src/index.ts): memory-specific embedding strategies, tier-aware TTL caching, a batch pipeline with progress, and similarity search over memory blocks. A cross-domain seam.

psyche-memory-retrieval#

Advanced memory retrieval (src/index.ts): hybrid semantic (vector) + keyword (BM25/TF-IDF) search, re-ranking (RRF, MMR, importance, recency, hybrid-score), and deduplication, via a MemoryRetrievalService.

psyche-memory-tools#

LLM-callable MemGPT-style memory tools (src/index.ts): tool schema/definitions, an execution engine, and backend integration that let an LLM manage its own memory blocks.

psyche-computer-use-core#

Provider-neutral native desktop control (src/index.ts): separately injected planning/vision bindings, governed screenshots, native action/result types, and an observe-reason-act agent. Every run requires exact app/window/action/network/ task-file authority, process-confinement attestation, risk-based confirmation, fresh-frame and focus interlocks, step/action/rate/time/token/interrupt budgets, and non-model end-state verification. The library is not a browser driver and is not yet admitted to Eve/Drawer; ADR-0077 owns browser separation and ADR-0080 owns the per-run native control boundary.

psyche-browser-automation#

Playwright-backed browser automation (src/index.ts): navigation/interaction, form filling, the Page Object pattern, network interception/mocking, screenshots, cookie/storage management, and retry/wait utilities via createBrowserAutomation.

psyche-screen-analysis#

Screen analysis (src/index.ts, ~94K LOC, the largest lib): screenshot capture across providers, Tesseract.js OCR, heuristic UI-element detection, document classification, and screenshot comparison/change detection.

psyche-sandbox#

Sandboxed execution for agents (src/index.ts): process/Docker/VM isolation backends, CPU/memory/time resource limits, network restrictions, filesystem access control, and security policies via createSandbox.

psyche-action-safety#

Action validation and dangerous-action blocking (src/index.ts): a SafetyChecker, confirmation workflows, rate limiting, and a policy engine that classify and gate agent actions (e.g. blocking rm -rf /).

psyche-tool-registry#

MCP tool registry (src/index.ts): tool registration, discovery, and metadata management, with parameter/return types, external-format conversion, search, and versioning over registry events.

psyche-tool-execution#

Tool-call execution pipeline (src/index.ts): tool-call parsing, an execution pipeline with handlers, and result handling for AI agent systems, with event/config and parser types.

psyche-knowledge-ingestion#

RAG ingestion pipeline (src/index.ts): multiple chunking strategies (fixed-size, semantic, recursive, token-based), an embedding-service abstraction with caching/batching, vector-index management, document versioning, and progress reporting via createIngestionPipeline.

psyche-knowledge-context#

RAG context assembly (src/index.ts): a ContextAssembler with strategies, a TokenBudgetManager, SourceAttribution, and a ContextFormatter for citation-aware context construction within a token budget.

psyche-knowledge-retrieval#

RAG retrieval (src/index.ts): semantic vector search (cosine/euclidean/dot/ manhattan), BM25/TF-IDF keyword search, hybrid retrieval (RRF, weighted, cascade), and re-ranking (MMR, weighted, recency decay) with query/result caching.

psyche-sophia-integration#

Integration bridging Sophia's document-ingestion pipeline into Psyche knowledge management (src/index.ts): a SophiaIngestionAdapter, full pipeline orchestration, and bidirectional document/chunking/embedding adapters. A cross-domain seam.

psyche-sophia-search-integration#

Integration bridging Sophia search into Psyche (src/index.ts): a PsycheSearchAdapter, a fluent PsycheQueryBuilder, a citation-producing PsycheRAGIntegration, and a result mapper translating Sophia results to Psyche form.

psyche-conferencing-core#

Platform-agnostic video-conferencing abstraction (src/index.ts): the IConferenceSession interface, video injection modes (replace/overlay/PIP/ background), audio injection/mixing (replace/blend/ducking/sidechain) with EQ, compression, and noise gating, plus createMockConferenceSession for testing. The base every conferencing adapter implements.

psyche-zoom-integration#

Zoom Meeting SDK adapter (src/index.ts): a meeting-lifecycle wrapper, JWT/OAuth auth, a REST client, raw audio/video access, and an IConferenceSession implementation over conferencing-core. Requires real Zoom SDK credentials.

psyche-teams-integration#

Microsoft Teams adapter (src/index.ts): Bot Framework + Real-time Media Platform integration for joining meetings with raw audio/video and participant management, exposing createTeamsConferenceSession. Requires Teams tenant/app credentials.

psyche-meet-integration#

Google Meet adapter (src/index.ts): browser-automation (Puppeteer) + Calendar API integration for joining meetings with raw audio/video, via createMeetConferenceSession. Requires Google OAuth credentials.

psyche-webex-integration#

Cisco Webex adapter (src/index.ts): Webex SDK + REST API integration for joining meetings with raw audio/video and participant management, via createWebexConferenceSession. Requires a Webex access token.

psyche-recall-integration#

Recall.ai meeting-bot integration (src/index.ts): a single multi-platform bot (Zoom/Meet/Teams/Webex) with auth manager, API client, and bot SDK (createRecallBotSDK) for realtime audio/video streaming. Requires a Recall.ai API key.

psyche-participant-manager#

Multi-participant stream management (src/index.ts, ~85K LOC, one of the largest libs): participant tracking, gallery pagination with active-speaker follow, processing queues, and feature degradation for large meetings via createParticipantTracker/ createGalleryManager/createProcessingQueueManager/createDegradationManager.

@psyche/participation-tracker#

Meeting participation tracking (src/index.ts): speaker analysis, balance metrics, a MeetingDynamicsAnalyzer, a ParticipationTrendAnalyzer, and the main ParticipationTracker, correlating participation with engagement.

psyche-participant-language-preference#

Per-participant language preference management (src/index.ts): source-priority preference resolution, dialect identifiers, auto-detection integration, fallback chains, conflict resolution, meeting language-policy enforcement, change history, and translation routing.

psyche-meeting-summarizer#

AI meeting summarization (src/index.ts): processes transcripts, participant data, events, and engagement metrics into structured summaries (topics, action items, decisions, questions, engagement analysis) with input/output models.

psyche-caption-streaming#

Realtime translated-caption stream management (src/index.ts): caption-segment ingestion, multi-language output channels, speaker attribution, word-level timing, translation integration, buffering, latency tracking, rendering, synchronization, and transcript aggregation.

psyche-language-detection#

Language detection (src/index.ts): text statistical analysis, script detection, n-gram profiling, audio spectral/phonemic analysis, multi-language content detection, language-change tracking, and realtime streaming detection, with Zod schemas.

psyche-cultural-adaptation#

Translation-aware cultural adaptation (src/index.ts): idiom detection/ adaptation, cultural-reference handling, formality-register mapping, honorific systems, measurement conversion, sensitivity filtering, and dialect selection over regional databases.

psyche-dialect-handling#

Dialect/accent handling (src/index.ts): regional variant detection, rule-based text adaptation between dialects, accent-profile management, formality registers, multi-dialect conversation coordination, and metrics, with Zod schemas.

psyche-translation-memory#

Translation memory + glossary management (src/index.ts): fuzzy matching via Levenshtein distance, quality-weighted retrieval, LRU/LFU/quality/oldest eviction, glossary term enforcement, TMX export, and project-based orchestration via createTranslationMemoryToolkit.

psyche-translation-quality#

Translation-quality monitoring (src/index.ts): automatic metric computation (BLEU, METEOR, TER, chrF), threshold alerting, per-language-pair tracking, trend/ degradation analysis, feedback integration, reference corpora, quality gates, and benchmarking, with Zod schemas.

psyche-translation-streaming#

Realtime translation streaming (src/index.ts): chunked processing pipelines, multi-protocol transport, circular buffering with backpressure, adaptive quality, latency optimization, exponential-backoff reconnection, and metrics, with Zod schemas.

@psyche/tavus-client#

TypeScript API client for the Tavus CVI (Conversational Video Interface) platform (src/index.ts): replica/persona/conversation operations with response caching, request batching, and usage monitoring via createTavusClient. The base the other Tavus libs build on.

@psyche/tavus-conversation#

Tavus conversation management (src/index.ts): configuration presets (e.g. CUSTOMER_SERVICE_PRESET), recording management, transcript handling, analytics, and pluggable storage (createInMemoryStorage) via createConversationManager.

@psyche/tavus-persona-manager#

Tavus persona management (src/index.ts): persona templating with variable substitution (e.g. CUSTOMER_SERVICE_TEMPLATE), versioning, cloning, and configuration validation via createPersonaManager.

@psyche/tavus-replica-manager#

Tavus replica lifecycle management (src/index.ts): versioning, quality validation, workflow automation, and metadata storage via createReplicaManager/ createReplicaId.

@psyche/tavus-pipeline#

Full Tavus CVI pipeline (src/index.ts): WebRTC + Daily.co room management, data channel communication, and conversation-lifecycle management via createCVIPipeline, integrating turn-taking, perception, and tool calling.

@psyche/tavus-pipecat#

Pipecat-inspired frame-based pipeline architecture for Tavus (src/index.ts): frame types, processors, pipelines, and services for realtime conversational AI.

@psyche/tavus-phoenix#

Phoenix-4 rendering-pipeline integration for Tavus (src/index.ts): WebRTC video streaming, frame synchronization, and quality management via createRenderPipeline/ createQualityManager with QUALITY_PRESETS.

@psyche/tavus-perception#

Raven-1 visual-perception integration for Tavus (src/index.ts): emotion detection, video forwarding, a perception handler, LLM context, a facial-expression pipeline, caching/filtering/analytics, screen-content analysis, gesture recognition, and a learning optimizer, with hybrid Tavus+Psyche signals.

@psyche/tavus-turntaking#

Sparrow-1 turn-taking integration for Tavus (src/index.ts): floor manager, turn detector, speaker adaptation, prosodic signals, syllable detection, hesitation handling, cultural timing, and multi-speaker support in hybrid Tavus+Psyche mode.

@psyche/tavus-expression#

Expression/emotion control bridge for Tavus (src/index.ts): a TavusExpressionController (extends EventEmitter) providing realtime expression updates, lip sync, and quality monitoring.

@psyche/tavus-tts#

TTS integration for Tavus (src/index.ts): Cartesia and ElevenLabs voice selection, configuration, presets, and persona-TTS management via createTavusTTSManager/getTTSConfigForUseCase/buildPersonaTTSUpdate.

@psyche/tavus-llm#

Custom-LLM integration for Tavus (src/index.ts): an OpenAI-compatible API proxy for Claude — OpenAI↔Claude message translation, streaming SSE translation, a /chat/completions proxy server, system-prompt sync, and tool-calling — via createProxyServer/buildTavusLLMConfig.

@psyche/tavus-knowledge#

Knowledge-base/RAG integration for Tavus (src/index.ts): document management, retrieval configuration, and cross-system synchronization of knowledge between Tavus and Psyche.

@psyche/tavus-memories#

Memory management/synchronization for Tavus (src/index.ts): cross-conversation persistence, tag management, and Psyche memory integration.

@psyche/tavus-objectives#

Objectives and guardrails management for Tavus (src/index.ts): workflow configuration, compliance templates, and trigger handling for conversation objectives.

@psyche/tavus-tools#

Tool-calling integration for Tavus (src/index.ts): tool-event handling, execution, security/audit, and built-in tools, with branded tool/execution types.

@psyche/tavus-hybrid#

Hybrid-mode orchestration between Psyche-native and Tavus avatar systems (src/index.ts): a unified avatar interface, mode switching, health monitoring, failover, and graceful degradation — the bridge that lets the two stacks run interchangeably.

@psyche/tavus-bridge#

Video-conferencing bridge for Tavus (libs/psyche/tavus-bridge, ~48K LOC across the package; src/index.ts is the 641-line barrel): connects Tavus avatar streams to virtual camera/audio devices (WebRTC, virtual-camera/-audio types) for use with Zoom, Meet, Teams, etc.

@psyche/aiops-anomaly#

AIOps anomaly detection (src/index.ts): statistical detectors (Z-score, modified Z-score, IQR, Grubbs, MAD, EWMA), time-series trend/seasonality/forecasting, CUSUM changepoint detection, metric correlation (Pearson/Spearman/cross-corr), root-cause inference, and alert management.

@psyche/log-metrics#

Log-based metrics (src/index.ts): multi-format parsing (JSON, logfmt, CLF, Combined, Syslog, custom regex), level classification, timestamp normalization, pattern matching, rule-based metric/label extraction, time-windowed aggregation, and analytics.

@psyche/ml-log-analysis#

ML log analysis (src/index.ts): log clustering (Drain, cosine, Jaccard, Levenshtein, token-frequency), template extraction, TF-IDF vectorization, frequency anomaly detection, classification, sequential-pattern mining, and root-cause inference.

@psyche/predictive-alerting#

Predictive alerting (src/index.ts): forecasting (linear regression, exponential smoothing, Holt's, Holt-Winters, moving average, ARIMA-like), trend detection, capacity/exhaustion forecasting, lead-time alert rules, notification routing with rate limiting, suppression/maintenance windows, and accuracy tracking.

@psyche/slo-alerting#

SLO-based alerting (src/index.ts, cites Google SRE): SLI definition/measurement, SLO management, error-budget tracking, multi-window burn-rate alerting, and Prometheus-compatible alert-rule generation.

@psyche/latency-analysis#

Latency analysis (src/index.ts): component-level latency tracking/breakdown, histograms with streaming percentile estimation, statistical regression detection, latency-threshold/anomaly alert rules, and trace critical-path analysis.

psyche-predictive-performance-modeling#

Predictive performance modeling (src/index.ts): multi-metric performance tracking, model training (linear regression, exponential smoothing, Holt-Winters, moving average, ensemble), degradation/capacity-exhaustion/anomaly forecasting, risk assessment, and orchestrated monitoring sessions.

psyche-quality-trend-analysis#

Domain-agnostic quality-trend analysis (src/index.ts): multi-metric time-series tracking, configurable smoothing, linear-regression trending, degradation/ improvement and changepoint detection, forecasting, ensemble scoring, alerting, and orchestrated monitoring sessions.

@psyche/disaster-recovery#

Kubernetes disaster recovery (src/index.ts): backup scheduling/execution/ verification/retention, recovery with validation+rollback, cross-site replication with lag monitoring, RPO/RTO/SLA tracking, failover-plan orchestration, DR test planning, and posture scoring.

@psyche/k8s-federation#

Kubernetes multi-cluster federation (src/index.ts): cluster registration/ lifecycle/health, federated resource distribution with pluggable placement, cross-cluster service discovery, multi-strategy traffic routing (round-robin, weighted, latency, geo, failover), and automatic failover/recovery.

@psyche/k8s-security#

Kubernetes security (src/index.ts): IRSA (IAM Roles for Service Accounts) for AWS EKS, Pod Security Standards validation, Network Policy management, and RBAC configuration/analysis.

@psyche/database-sharding#

Database sharding (src/index.ts): strategies (hash, range, consistent-hash, directory, geographic, composite), a consistent-hash ring with virtual nodes (FNV-1a/DJB2/Murmur-like), shard-key extraction, query routing (single/ scatter-gather/broadcast/targeted), migration, rebalancing, split/merge, and hotspot detection.

@psyche/timeseries-optimization#

Time-series optimization (src/index.ts): downsampling, compression, retention policies, time-bucketing, query optimization, storage tiering, materialized-view management, and LRU caching, with Zod schemas.

@psyche/human-evaluation#

Continuous human-evaluation pipeline (src/index.ts): evaluation sessions with counterbalancing, multiple rating types (Likert/binary/ranking/continuous), attention checks, inter-rater reliability (ICC, Fleiss' Kappa, Krippendorff's Alpha), significance testing, trend analysis, and quality alerts.

psyche-crowdsourced-evaluation#

Crowdsourced evaluation at scale (src/index.ts): worker management with qualification tiers, HIT lifecycle, quality control (gold standards, spam detection), response aggregation (majority vote, weighted average, Dawid-Skene EM, median, trimmed mean), cost tracking, and campaign orchestration.

@psyche/nps#

Net Promoter Score collection/analysis (src/index.ts): survey management, response recording, NPS calculation, and an analytics engine producing reports via createNPSSurveyManager/createNPSAnalyticsEngine/calculateNPS.

@psyche/satisfaction#

Customer satisfaction surveys (CSAT, CES) for AI evaluation (src/index.ts): survey creation/publishing, response recording, and CSAT/CES calculation via createSatisfactionSurveyManager/calculateCSAT/calculateCES.

psyche-accessibility#

Accessibility library for Psyche agents (src/index.ts, ~30K LOC): WCAG 2.2 compliance — contrast checking (4.5:1), pointer alternatives for gestures, minimum target sizes (24×24), redundant-entry prevention — plus captioning, screen-reader, motor, and hearing accessibility.

psyche-mobile-sdk#

Mobile SDK surface (src/index.ts): iOS and Android SDK modules, cross-platform wrappers, white-label capabilities, edge/offline deployment, and multi-agent collaboration (Phase 24.18), re-exported from per-platform modules.

psyche-e2e-tests#

End-to-end test suite (*.e2e.test.ts, ~4K LOC across four specs): vitest tests exercising avatar rendering, voice pipeline (STT→LLM→TTS with <500ms latency), conferencing, and performance, using mock providers to simulate without live API calls. A test project, not a runtime library.

psyche-integration-tests#

Python integration test suite (tests/test_*.py, ~3.3K LOC across three modules): pytest tests covering cross-service event flows, pipeline integration, and service communication for the Python foundation. A test project, not a runtime library.

@psyche/training-data#

Phase 85–86 flywheel producer for the avatar/assistant domain (libs/psyche/training-data/src), split into two pipelines rather than the single-pipeline shape its siblings use: PsycheAvatarSignalPipeline captures avatar-animation signals (typed FlameFrame 3DMM coefficients and VisemeFrame lip-sync frames) and PsychePerceptionSignalPipeline captures perception signals, both under an explicit PsycheTrainingConsent and a shared PsycheTrainingSink seam.