The
libs/psyche/area: ~133 Nx projects that make up the Psyche hyper-realistic AI virtual-assistant platform — a Python service foundation plus a large fleet of TypeScript libraries for avatar rendering, voice, perception, memory, agentic computer-use, conferencing, translation, the Tavus CVI integration, and infrastructure/observability.
What this area is#
Psyche is the "AI virtual assistant" domain: a realtime, multi-modal avatar that
can see, listen, speak, remember, drive a computer, and join video calls. Unlike
most Oshun domains this is not one library — it is ~133 separately-tracked
Nx projects rooted at libs/psyche/, every one with its own project.json.
They split cleanly into two technology tiers, which is the first thing to
understand about the area.
The two tiers#
A Python service foundation (lang:python tags) provides the cross-cutting
runtime plumbing every Psyche backend service shares: psyche-auth,
psyche-cache, psyche-database, psyche-logging, psyche-messaging,
psyche-state-sync, psyche-storage, psyche-tracing, and the catch-all
psyche-common. Each is a real Poetry-packaged Python library under
src/psyche_* and is explicitly "aligned with" its TypeScript counterpart in
the shared @oshun/* libraries (e.g. psyche-logging mirrors
@oshun/logging), giving cross-language parity. psyche-platform itself is the
root Poetry/pytest aggregator (libs/psyche/pyproject.toml, conftest.py,
packages = []) — it ships no code of its own but owns shared test config and
declares the Nx implicitDependencies that bind the foundation and many feature
libs into one affected graph.
A large TypeScript feature fleet (the remaining ~120 type:lib projects)
implements the actual assistant capabilities as standalone, mostly
dependency-light libraries: each is a src/index.ts barrel over a set of
domain-specific modules, with Zod schemas at the edges and deterministic
algorithms inside (FACS action units, saccade main-sequence kinematics, BM25 +
RRF hybrid retrieval, BLEU/METEOR/TER, Dawid-Skene aggregation, Holt-Winters
forecasting, MemGPT-style paging, and so on). These are genuine implementations,
not CRUD shells; the per-library @example blocks in each barrel show the
public API.
How the feature libraries relate#
The TS fleet is organised by capability sub-system: avatar rendering
(avatar-*, 3D Gaussian Splatting + FLAME), voice/speech (voice-*,
speech-*, viseme-generator, noise-handling), facial/non-verbal perception
and behavior (face-*, emotion-*, gaze-*, head-*, gesture-system,
posture-system, *-expressions, behavior-*),
conversation/engagement/proactivity (dialogue-manager, engagement-*,
proactive-*, *-triggers), hierarchical memory (memory-*), agentic
computer-use (computer-use-core, browser-automation, screen-analysis,
sandbox, action-safety, tool-*), knowledge/RAG (knowledge-*,
sophia-*-integration), conferencing/meetings (conferencing-core + platform
adapters + participant-*/meeting-*), translation/localization
(translation-*, cultural-adaptation, dialect-handling,
language-detection), the Tavus CVI integration (@psyche/tavus-*), and an
infrastructure/observability cluster (k8s-*, *-alerting, *-anomaly,
log-metrics, latency-analysis, etc.).
These libraries compose: perception libs feed signals into emotion-engine and
engagement-detector; viseme-generator/avatar-lipsync drive avatar-core;
memory-* and knowledge-* share embedding/retrieval patterns; and the Tavus
fleet and Psyche-native avatar/perception stacks are bridged by
@psyche/tavus-hybrid.
How it fits the wider system#
Consumers are Psyche's own services and the BFF. The Python foundation is
imported by Psyche backend services for auth/db/cache/messaging/observability;
the TS feature libs are composed by the realtime avatar/conferencing runtimes
and by agent loops. Several libs are explicitly integration seams to other
Oshun domains: psyche-avatar-isis-integration bridges @isis/3d-generation,
psyche-sophia-integration/psyche-sophia-search-integration and
psyche-memory-embeddings bridge Sophia's ingestion/search/embedding stack, and
the conferencing adapters wrap third-party SDKs (Zoom, Teams, Meet, Webex,
Recall.ai, Tavus). Many libraries are designed around injectable provider
boundaries — e.g. conferencing-core's createMockConferenceSession,
memory-archival's createMockEmbeddingProvider, the speech/TTS provider
abstractions — so the deterministic logic is unit-testable offline while real
SDK/credentialed providers are supplied in production. Walk the "used by" edges
on any node below to see exactly who depends on it.
Entity reference#
psyche-platform#
Root Poetry/pytest aggregator for the Python tier (libs/psyche/pyproject.toml,
conftest.py, sourceRoot: libs/psyche); ships no package of its own
(packages = []) but owns shared test/lint config and the Nx
implicitDependencies that wire the foundation and feature libs into one graph.
psyche-auth#
Python authentication library (src/psyche_auth) aligned with
@oshun/auth-primitives: JWT signing/verification, in-memory + Redis session
stores, API-key management, Argon2/bcrypt/PBKDF2 password utilities, and FastAPI
auth middleware (per its README.md).
psyche-cache#
Python caching library (src/psyche_cache) aligned with @oshun/cache: LRU+TTL
in-memory cache, Redis distributed cache, namespaced key building, pub/sub state
sync, Redlock-compatible distributed locking, and standard TTL presets.
psyche-common#
The largest Python foundation lib (src/common, ~16.7K LOC): shared utilities
across all Psyche services — GPU optimization (quantization, batch inference,
TensorRT, memory management), structured logging, success-metrics tracking
(human-likeness, conversation quality, cost), performance/streaming pipelines,
and horizontal-scaling helpers (per README.md).
psyche-database#
Dual-language data layer: a Python async client (src/psyche_database,
PostgreSQL via asyncpg, Redis, transactions, migrations, query builder,
SQLAlchemy models, Qdrant vector storage) plus a Prisma-generated TypeScript
client under src/generated/client/ (models such as AvatarPack,
VoiceProfile). Aligned with @oshun/database.
psyche-logging#
Python structured-logging library (src/psyche_logging) aligned with
@oshun/logging: JSON log schema, contextvars-based context propagation, log
sampling strategies, PII redaction, optional OpenTelemetry trace enrichment, and
child loggers.
psyche-messaging#
Python event bus (src/psyche_messaging) aligned with @oshun/event-bus:
pattern/wildcard pub-sub, event persistence with replay TTL, retry with
exponential/linear/fixed backoff, dead-letter queue, correlation tracking, and
cross-domain event routing.
psyche-state-sync#
Python realtime state-synchronization library (src/psyche_state_sync) for
assistant sessions: avatar state (position/rotation, 52 FACS blendshapes, gaze,
animations, visemes), voice state, session lifecycle/emotional state, and
multi-participant conferencing state with turn-taking and screen sharing.
psyche-storage#
Python S3/MinIO-compatible storage client (src/psyche_storage) aligned with
@oshun/storage: async operations, multipart uploads with progress, pre-signed
URLs, and specialized asset managers for avatar models, voice recordings, and
knowledge documents.
psyche-tracing#
Python distributed-tracing library (src/psyche_tracing) aligned with
@oshun/tracing: OpenTelemetry SDK + OTLP export, W3C traceparent/tracestate
propagation, instrumentation decorators, AWS X-Ray header support, and branded
trace/span ID types.
psyche-avatar-core#
3D Gaussian Splatting avatar rendering core (src/index.ts): FLAME parametric
model utilities (neutral state, expression mapping, coefficient blending, pose),
viseme/lip-sync application, and the shared avatar type surface for the rest of
the avatar-* fleet.
psyche-avatar-cache#
Tiered avatar caching for fast switching/preloading: memory + persistent cache
entries, expression caching, preloading, and statistics/event types, for
low-latency avatar operations (src/index.ts).
psyche-avatar-expressions#
Expression blendshapes and FACS mapping for avatars: ARKit blendshapes, FACS
action units, emotion presets, micro-expressions, transitions, asymmetry, and a
controller-state surface with Zod schemas (src/index.ts).
psyche-avatar-isis-integration#
Integration layer bridging @isis/3d-generation and psyche-avatar-core:
text-to-3D and photo-to-3D avatar generation requests/results, blend-shape
mapping/conversion between the two systems, and an avatar generator
(src/index.ts). A cross-domain seam.
psyche-avatar-lipsync#
Lip-sync/viseme animation: a 19-class viseme system with 31 blendshape
parameters, ARPAbet/IPA phoneme-to-viseme mapping, coarticulation blending, and
a real-time StreamingAligner with lookahead buffering targeting <33ms latency
(src/index.ts).
psyche-avatar-quality#
Adaptive quality system for avatar rendering: GPU detection, quality levels,
dynamic resolution scaling, and Gaussian-count adaptation with
performance/Gaussian types and Zod schemas (src/index.ts).
psyche-avatar-taa#
Temporal anti-aliasing for Gaussian-splatting avatars: jitter generation, motion
vectors, history accumulation, and ghost reduction for stable temporal output,
with vector/matrix/color helper types (src/index.ts).
psyche-avatar-training#
Avatar training pipeline (monocular video → 3D Gaussian Splatting): video
preprocessing orchestration, FLAME parameter fitting, Gaussian-splat training,
blendshape optimization, and expression calibration (src/index.ts).
psyche-voice-synthesis#
TTS library with ElevenLabs and Cartesia providers (src/index.ts): streaming,
voice cloning, SSML/prosody control, and a provider abstraction with voice
settings, audio-output, and error types.
psyche-voice-streaming#
Streaming TTS playback (src/index.ts): audio chunking, buffer management,
network-quality adaptation, and streaming-session/playback/event types for
low-latency voice output.
psyche-voice-consistency#
Voice consistency for natural speech (src/index.ts): voice-identity
preservation via profile management, emotion-appropriate prosody with
transitions, context-aware speaking-rate adaptation, and automatic consistency
corrections.
psyche-voice-dubbing#
Target-language voice dubbing (src/index.ts): speaker detection, voice
mapping, prosody transfer, timing alignment, multi-language synthesis, quality
control, and project management with metrics.
psyche-viseme-generator#
Text-to-viseme prediction with audio-aligned timing (src/index.ts):
grapheme-to-phoneme conversion, viseme timing, optional coarticulation, and a
textToVisemes convenience plus a configurable generator.
psyche-speech-recognition#
Speech-to-text library (src/index.ts): multi-provider support (Deepgram,
Whisper, WebSpeech), voice-activity detection, streaming and batch
transcription, word-level timestamps, and multi-language support.
psyche-speech-translation#
Full speech-to-speech translation pipeline (src/index.ts): audio → STT → text
translation → TTS → audio, with multi-speaker voice mapping, prosody transfer,
realtime streaming, translation memory, glossary, and quality monitoring.
psyche-noise-handling#
Audio pre-processing for voice (src/index.ts): noise suppression, echo
cancellation, and audio normalization, composable individually or via a unified
createAudioProcessor operating on sample buffers.
psyche-face-detection#
Face detection and landmark tracking (src/index.ts): realtime detection, the
478-point MediaPipe face mesh, IoU-based tracking with smoothing, head-pose
estimation, and a pluggable detection backend.
psyche-face-analysis#
Facial analysis on landmarks (src/index.ts): FACS action-unit detection,
Ekman + extended expression classification, the valence-arousal-dominance
circumplex, age/gender estimation, and temporal smoothing.
psyche-emotion-recognition#
Realtime multi-face emotion pipeline (src/index.ts): per-face emotion tracking
with persistent identity, temporal pattern + micro-expression analysis, dominant
emotion calculation, suppression/genuineness detection, and quality scoring.
psyche-emotion-engine#
Emotion modeling for avatar animation (src/index.ts): primary/extended
emotions, the Russell circumplex (valence-arousal), compatibility-based
blending, emotion-specific transitions, mood persistence/contamination, social
display rules, and FACS/blendshape output.
psyche-facs-expressions#
Facial Action Coding System library (src/index.ts): action-unit definitions,
intensity control, AU-to-emotion mapping, blendshape combinations, and
expression state management for facial animation.
psyche-micro-expressions#
Micro-expression generation (src/index.ts): brief involuntary expressions
(40–200ms), natural blink patterns with variability, eye-moisture/tearing
simulation, subtle tics/twitches/saccades, and suppression/leakage and deception
simulation.
psyche-gaze-estimation#
Eye-gaze estimation from landmarks (src/index.ts): eye-region extraction,
iris/pupil tracking, pitch/yaw gaze direction with head-pose correction,
calibrated screen mapping, exponential+Kalman filtering, and
fixation/saccade/blink and AOI analysis.
psyche-gaze-control#
Biologically-grounded gaze generation (src/index.ts, cites saccade
main-sequence and conversational-gaze research): saccades with main-sequence
kinematics, context-aware fixations, microsaccade/drift, smooth pursuit,
vergence, conversation-aware coordination, cultural profiles, and ARKit output.
psyche-gaze-awareness-behaviors#
Gaze-awareness behaviors (src/index.ts): maps gaze points/targets onto content
and derives awareness-driven behaviors, with Zod input schemas for gaze points,
targets, and configuration.
psyche-head-pose-estimation#
Head-pose estimation from landmarks (src/index.ts): PnP-based pose with a
geometric fallback, multi-format landmark support (MediaPipe/dlib/OpenPose),
movement detection (nods/shakes/tilts/turns), stability and range-of-motion
tracking, and fatigue indicators.
psyche-head-movement#
Generative head movement (src/index.ts): conversational nods, disagreement
shakes, interest/confusion/empathy/thinking tilts, speech-synced emphasis, idle
micro-sway/drift, stimulus responses, and cultural/personality adaptation.
psyche-gesture-system#
Gesture generation/animation (src/index.ts): 54+ predefined gesture types
(beats, emblematic, pointing, iconic, metaphoric), keyframe animation with
easing, priority-based multi-gesture blending, speech-synced beats, cultural
adaptation by locale, and personality variation.
psyche-posture-system#
Posture animation (src/index.ts): breathing, weight shifting, and social
mirroring, exposing the core posture types and a posture controller.
psyche-behavior-coordinator#
Priority-based behavior coordination for avatar animation (src/index.ts):
behavior definitions with categories/priorities/durations, a priority queue with
conflict resolution, fade-in/out timing, multi-channel output blending, and
state machines via a BehaviorOrchestrator.
psyche-behavior-prediction#
User-behavior prediction (src/index.ts): Markov-chain transition modeling,
sequence pattern mining, temporal pattern detection, ensemble prediction,
accuracy tracking, and analytics, with Zod event/config schemas.
@psyche/behavior-anomaly-detector#
Behavioral anomaly detection for meeting participants (src/index.ts):
inactivity detection, erratic-behavior patterns, conflict-escalation,
cross-participant analysis, and meeting-atmosphere scoring with a default
config.
@psyche/uncanny-valley#
Uncanny-valley detection/scoring for avatar quality (src/index.ts):
per-category collectors (blink, gaze, expression, head/body movement, response
timing, lip-sync, visual quality) scored against DEFAULT_BASELINES human
baselines with weighted aggregation into an UncannyValleyScore.
psyche-human-likeness-estimation#
Automated human-likeness estimation via multi-modal signal fusion
(src/index.ts): behavioral, conversational, emotional, linguistic, and
consistency signals with feature-weighted scoring, artifact detection,
ground-truth calibration, trend analysis, A/B testing, and benchmarking.
psyche-dialogue-manager#
Dialogue management for natural conversation flow (src/index.ts): turn-taking
(floor control, overlap, push-to-talk), interruption detection/handling,
conversation-state tracking, intent detection, and multi-participant support in
an event-driven design.
psyche-conversation-sentiment-tracking#
Conversation-level sentiment tracking via signal fusion (src/index.ts):
aggregates facial/vocal/text/engagement/atmosphere signals into smoothed
temporal sentiment arcs with turning-point detection, intervention
recommendations, phase analysis, speaker congruence, and cross-conversation
comparison.
@psyche/attention-tracker#
Attention tracking/analysis (src/index.ts): attention-state tracking, shift
and lapse detection with recovery, collective multi-participant attention,
attention heatmaps, engagement scoring, topic-attention correlation, pattern
detection, and fatigue recommendations.
psyche-engagement-detector#
Multi-dimensional engagement detection (src/index.ts): gaze, behavioral,
conversational, temporal, and content engagement combined with configurable
weights into smoothed scores, trend detection, and signal detection (attention
loss, interest spikes, fatigue) with session history.
psyche-engagement-recovery#
Disengagement detection and recovery orchestration (src/index.ts): a
disengagement detector feeding recovery-strategy selection, with Zod schemas for
engagement snapshots and recovery config.
@psyche/context-monitoring#
Realtime context monitoring (src/index.ts): signal collection/aggregation,
pattern and anomaly detection, state tracking, threshold-based alerting, and
analytics, with Zod schemas for signals and configuration.
psyche-anticipatory-responses#
Anticipatory response modeling (src/index.ts): an anticipation modeler over
incoming signals (with AnticipationSignalSchema) to pre-compute likely
responses, plus utility/constant helpers.
psyche-follow-up-initiation#
Follow-up initiation (src/index.ts): types and logic for deciding when and how
the assistant should proactively initiate a follow-up. A compact lib (one barrel
over a types module and engine).
psyche-proactive-suggestions#
Proactive suggestion engine (src/index.ts): signal analysis, trigger-based
suggestion generation, suppression/deduplication, user-feedback tracking,
pattern learning, and analytics, with Zod schemas.
psyche-proactive-summarization#
Proactive content summarization (src/index.ts): a content collector feeding
trigger detection, format selection, user-preference learning, and delivery
orchestration, with ContentItemInputSchema/config schemas.
psyche-proactive-assistance-triggers#
Proactive-assistance trigger detection (src/index.ts): a need detector over
indicator inputs (NeedIndicatorInputSchema) plus constants, utilities, and
factory functions for firing assistance offers.
psyche-help-offer-triggers#
Help-offer triggering (src/index.ts): a help-signal aggregator over
HelpSignalInputSchema inputs with HelpOfferConfigSchema to decide when to
surface help. A small, focused trigger lib.
psyche-memory-core#
MemGPT-style hierarchical memory (src/index.ts): multi-tier storage with
automatic paging and capacity management via a VirtualContextManager (add,
search, context-window assembly, background monitoring). The anchor of the
memory-* cluster.
psyche-memory-in-context#
8K-token in-context memory (src/index.ts): per-message extraction of key
facts, topics, and entities, with buildContext assembling an optimized,
token-estimated context for the LLM.
psyche-memory-working#
32K-token working memory (src/index.ts): LRU eviction with auto promotion/
demotion between tiers, importance-weighted entries, and statistics via a
WorkingMemoryManager.
psyche-memory-archival#
Unlimited archival memory (src/index.ts): vector-based semantic retrieval,
importance/tagging, and consolidation, built around an injectable embedding
provider (createMockEmbeddingProvider for offline tests; real providers in
production).
psyche-memory-consolidation#
Memory consolidation (src/index.ts): time-based decay (exponential, power-law,
stepped), multi-factor importance scoring, similar-memory merging, low-value
pruning, and automated consolidation scheduling.
psyche-memory-persistence#
Cross-session memory persistence (src/index.ts): multi-backend storage
(memory, file, PostgreSQL, Redis, S3), gzip/brotli compression, AES-256-GCM
encryption at rest, serialization, session management, and integrity
verification.
psyche-memory-embeddings#
Bridge between Psyche memory and @sophia/indexing embeddings (src/index.ts):
memory-specific embedding strategies, tier-aware TTL caching, a batch pipeline
with progress, and similarity search over memory blocks. A cross-domain seam.
psyche-memory-retrieval#
Advanced memory retrieval (src/index.ts): hybrid semantic (vector) + keyword
(BM25/TF-IDF) search, re-ranking (RRF, MMR, importance, recency, hybrid-score),
and deduplication, via a MemoryRetrievalService.
psyche-memory-tools#
LLM-callable MemGPT-style memory tools (src/index.ts): tool
schema/definitions, an execution engine, and backend integration that let an LLM
manage its own memory blocks.
psyche-computer-use-core#
Provider-neutral native desktop control (src/index.ts): separately injected
planning/vision bindings, governed screenshots, native action/result types, and
an observe-reason-act agent. Every run requires exact app/window/action/network/
task-file authority, process-confinement attestation, risk-based confirmation,
fresh-frame and focus interlocks, step/action/rate/time/token/interrupt budgets,
and non-model end-state verification. The library is not a browser driver and is
not yet admitted to Eve/Drawer; ADR-0077 owns browser separation and ADR-0080
owns the per-run native control boundary.
psyche-browser-automation#
Playwright-backed browser automation (src/index.ts): navigation/interaction,
form filling, the Page Object pattern, network interception/mocking,
screenshots, cookie/storage management, and retry/wait utilities via
createBrowserAutomation.
psyche-screen-analysis#
Screen analysis (src/index.ts, ~94K LOC, the largest lib): screenshot capture
across providers, Tesseract.js OCR, heuristic UI-element detection, document
classification, and screenshot comparison/change detection.
psyche-sandbox#
Sandboxed execution for agents (src/index.ts): process/Docker/VM isolation
backends, CPU/memory/time resource limits, network restrictions, filesystem
access control, and security policies via createSandbox.
psyche-action-safety#
Action validation and dangerous-action blocking (src/index.ts): a
SafetyChecker, confirmation workflows, rate limiting, and a policy engine that
classify and gate agent actions (e.g. blocking rm -rf /).
psyche-tool-registry#
MCP tool registry (src/index.ts): tool registration, discovery, and metadata
management, with parameter/return types, external-format conversion, search, and
versioning over registry events.
psyche-tool-execution#
Tool-call execution pipeline (src/index.ts): tool-call parsing, an execution
pipeline with handlers, and result handling for AI agent systems, with
event/config and parser types.
psyche-knowledge-ingestion#
RAG ingestion pipeline (src/index.ts): multiple chunking strategies
(fixed-size, semantic, recursive, token-based), an embedding-service abstraction
with caching/batching, vector-index management, document versioning, and
progress reporting via createIngestionPipeline.
psyche-knowledge-context#
RAG context assembly (src/index.ts): a ContextAssembler with strategies, a
TokenBudgetManager, SourceAttribution, and a ContextFormatter for
citation-aware context construction within a token budget.
psyche-knowledge-retrieval#
RAG retrieval (src/index.ts): semantic vector search (cosine/euclidean/dot/
manhattan), BM25/TF-IDF keyword search, hybrid retrieval (RRF, weighted,
cascade), and re-ranking (MMR, weighted, recency decay) with query/result
caching.
psyche-sophia-integration#
Integration bridging Sophia's document-ingestion pipeline into Psyche knowledge
management (src/index.ts): a SophiaIngestionAdapter, full pipeline
orchestration, and bidirectional document/chunking/embedding adapters. A
cross-domain seam.
psyche-sophia-search-integration#
Integration bridging Sophia search into Psyche (src/index.ts): a
PsycheSearchAdapter, a fluent PsycheQueryBuilder, a citation-producing
PsycheRAGIntegration, and a result mapper translating Sophia results to Psyche
form.
psyche-conferencing-core#
Platform-agnostic video-conferencing abstraction (src/index.ts): the
IConferenceSession interface, video injection modes (replace/overlay/PIP/
background), audio injection/mixing (replace/blend/ducking/sidechain) with EQ,
compression, and noise gating, plus createMockConferenceSession for testing.
The base every conferencing adapter implements.
psyche-zoom-integration#
Zoom Meeting SDK adapter (src/index.ts): a meeting-lifecycle wrapper,
JWT/OAuth auth, a REST client, raw audio/video access, and an
IConferenceSession implementation over conferencing-core. Requires real Zoom
SDK credentials.
psyche-teams-integration#
Microsoft Teams adapter (src/index.ts): Bot Framework + Real-time Media
Platform integration for joining meetings with raw audio/video and participant
management, exposing createTeamsConferenceSession. Requires Teams tenant/app
credentials.
psyche-meet-integration#
Google Meet adapter (src/index.ts): browser-automation (Puppeteer) + Calendar
API integration for joining meetings with raw audio/video, via
createMeetConferenceSession. Requires Google OAuth credentials.
psyche-webex-integration#
Cisco Webex adapter (src/index.ts): Webex SDK + REST API integration for
joining meetings with raw audio/video and participant management, via
createWebexConferenceSession. Requires a Webex access token.
psyche-recall-integration#
Recall.ai meeting-bot integration (src/index.ts): a single multi-platform bot
(Zoom/Meet/Teams/Webex) with auth manager, API client, and bot SDK
(createRecallBotSDK) for realtime audio/video streaming. Requires a Recall.ai
API key.
psyche-participant-manager#
Multi-participant stream management (src/index.ts, ~85K LOC, one of the
largest libs): participant tracking, gallery pagination with active-speaker
follow, processing queues, and feature degradation for large meetings via
createParticipantTracker/
createGalleryManager/createProcessingQueueManager/createDegradationManager.
@psyche/participation-tracker#
Meeting participation tracking (src/index.ts): speaker analysis, balance
metrics, a MeetingDynamicsAnalyzer, a ParticipationTrendAnalyzer, and the
main ParticipationTracker, correlating participation with engagement.
psyche-participant-language-preference#
Per-participant language preference management (src/index.ts): source-priority
preference resolution, dialect identifiers, auto-detection integration, fallback
chains, conflict resolution, meeting language-policy enforcement, change
history, and translation routing.
psyche-meeting-summarizer#
AI meeting summarization (src/index.ts): processes transcripts, participant
data, events, and engagement metrics into structured summaries (topics, action
items, decisions, questions, engagement analysis) with input/output models.
psyche-caption-streaming#
Realtime translated-caption stream management (src/index.ts): caption-segment
ingestion, multi-language output channels, speaker attribution, word-level
timing, translation integration, buffering, latency tracking, rendering,
synchronization, and transcript aggregation.
psyche-language-detection#
Language detection (src/index.ts): text statistical analysis, script
detection, n-gram profiling, audio spectral/phonemic analysis, multi-language
content detection, language-change tracking, and realtime streaming detection,
with Zod schemas.
psyche-cultural-adaptation#
Translation-aware cultural adaptation (src/index.ts): idiom detection/
adaptation, cultural-reference handling, formality-register mapping, honorific
systems, measurement conversion, sensitivity filtering, and dialect selection
over regional databases.
psyche-dialect-handling#
Dialect/accent handling (src/index.ts): regional variant detection, rule-based
text adaptation between dialects, accent-profile management, formality
registers, multi-dialect conversation coordination, and metrics, with Zod
schemas.
psyche-translation-memory#
Translation memory + glossary management (src/index.ts): fuzzy matching via
Levenshtein distance, quality-weighted retrieval, LRU/LFU/quality/oldest
eviction, glossary term enforcement, TMX export, and project-based orchestration
via createTranslationMemoryToolkit.
psyche-translation-quality#
Translation-quality monitoring (src/index.ts): automatic metric computation
(BLEU, METEOR, TER, chrF), threshold alerting, per-language-pair tracking,
trend/ degradation analysis, feedback integration, reference corpora, quality
gates, and benchmarking, with Zod schemas.
psyche-translation-streaming#
Realtime translation streaming (src/index.ts): chunked processing pipelines,
multi-protocol transport, circular buffering with backpressure, adaptive
quality, latency optimization, exponential-backoff reconnection, and metrics,
with Zod schemas.
@psyche/tavus-client#
TypeScript API client for the Tavus CVI (Conversational Video Interface)
platform (src/index.ts): replica/persona/conversation operations with response
caching, request batching, and usage monitoring via createTavusClient. The
base the other Tavus libs build on.
@psyche/tavus-conversation#
Tavus conversation management (src/index.ts): configuration presets (e.g.
CUSTOMER_SERVICE_PRESET), recording management, transcript handling,
analytics, and pluggable storage (createInMemoryStorage) via
createConversationManager.
@psyche/tavus-persona-manager#
Tavus persona management (src/index.ts): persona templating with variable
substitution (e.g. CUSTOMER_SERVICE_TEMPLATE), versioning, cloning, and
configuration validation via createPersonaManager.
@psyche/tavus-replica-manager#
Tavus replica lifecycle management (src/index.ts): versioning, quality
validation, workflow automation, and metadata storage via
createReplicaManager/ createReplicaId.
@psyche/tavus-pipeline#
Full Tavus CVI pipeline (src/index.ts): WebRTC + Daily.co room management,
data channel communication, and conversation-lifecycle management via
createCVIPipeline, integrating turn-taking, perception, and tool calling.
@psyche/tavus-pipecat#
Pipecat-inspired frame-based pipeline architecture for Tavus (src/index.ts):
frame types, processors, pipelines, and services for realtime conversational AI.
@psyche/tavus-phoenix#
Phoenix-4 rendering-pipeline integration for Tavus (src/index.ts): WebRTC
video streaming, frame synchronization, and quality management via
createRenderPipeline/ createQualityManager with QUALITY_PRESETS.
@psyche/tavus-perception#
Raven-1 visual-perception integration for Tavus (src/index.ts): emotion
detection, video forwarding, a perception handler, LLM context, a
facial-expression pipeline, caching/filtering/analytics, screen-content
analysis, gesture recognition, and a learning optimizer, with hybrid
Tavus+Psyche signals.
@psyche/tavus-turntaking#
Sparrow-1 turn-taking integration for Tavus (src/index.ts): floor manager,
turn detector, speaker adaptation, prosodic signals, syllable detection,
hesitation handling, cultural timing, and multi-speaker support in hybrid
Tavus+Psyche mode.
@psyche/tavus-expression#
Expression/emotion control bridge for Tavus (src/index.ts): a
TavusExpressionController (extends EventEmitter) providing realtime
expression updates, lip sync, and quality monitoring.
@psyche/tavus-tts#
TTS integration for Tavus (src/index.ts): Cartesia and ElevenLabs voice
selection, configuration, presets, and persona-TTS management via
createTavusTTSManager/getTTSConfigForUseCase/buildPersonaTTSUpdate.
@psyche/tavus-llm#
Custom-LLM integration for Tavus (src/index.ts): an OpenAI-compatible API
proxy for Claude — OpenAI↔Claude message translation, streaming SSE translation,
a /chat/completions proxy server, system-prompt sync, and tool-calling — via
createProxyServer/buildTavusLLMConfig.
@psyche/tavus-knowledge#
Knowledge-base/RAG integration for Tavus (src/index.ts): document management,
retrieval configuration, and cross-system synchronization of knowledge between
Tavus and Psyche.
@psyche/tavus-memories#
Memory management/synchronization for Tavus (src/index.ts): cross-conversation
persistence, tag management, and Psyche memory integration.
@psyche/tavus-objectives#
Objectives and guardrails management for Tavus (src/index.ts): workflow
configuration, compliance templates, and trigger handling for conversation
objectives.
@psyche/tavus-tools#
Tool-calling integration for Tavus (src/index.ts): tool-event handling,
execution, security/audit, and built-in tools, with branded tool/execution
types.
@psyche/tavus-hybrid#
Hybrid-mode orchestration between Psyche-native and Tavus avatar systems
(src/index.ts): a unified avatar interface, mode switching, health monitoring,
failover, and graceful degradation — the bridge that lets the two stacks run
interchangeably.
@psyche/tavus-bridge#
Video-conferencing bridge for Tavus (libs/psyche/tavus-bridge, ~48K LOC across
the package; src/index.ts is the 641-line barrel): connects Tavus avatar
streams to virtual camera/audio devices (WebRTC, virtual-camera/-audio types)
for use with Zoom, Meet, Teams, etc.
@psyche/aiops-anomaly#
AIOps anomaly detection (src/index.ts): statistical detectors (Z-score,
modified Z-score, IQR, Grubbs, MAD, EWMA), time-series
trend/seasonality/forecasting, CUSUM changepoint detection, metric correlation
(Pearson/Spearman/cross-corr), root-cause inference, and alert management.
@psyche/log-metrics#
Log-based metrics (src/index.ts): multi-format parsing (JSON, logfmt, CLF,
Combined, Syslog, custom regex), level classification, timestamp normalization,
pattern matching, rule-based metric/label extraction, time-windowed aggregation,
and analytics.
@psyche/ml-log-analysis#
ML log analysis (src/index.ts): log clustering (Drain, cosine, Jaccard,
Levenshtein, token-frequency), template extraction, TF-IDF vectorization,
frequency anomaly detection, classification, sequential-pattern mining, and
root-cause inference.
@psyche/predictive-alerting#
Predictive alerting (src/index.ts): forecasting (linear regression,
exponential smoothing, Holt's, Holt-Winters, moving average, ARIMA-like), trend
detection, capacity/exhaustion forecasting, lead-time alert rules, notification
routing with rate limiting, suppression/maintenance windows, and accuracy
tracking.
@psyche/slo-alerting#
SLO-based alerting (src/index.ts, cites Google SRE): SLI
definition/measurement, SLO management, error-budget tracking, multi-window
burn-rate alerting, and Prometheus-compatible alert-rule generation.
@psyche/latency-analysis#
Latency analysis (src/index.ts): component-level latency tracking/breakdown,
histograms with streaming percentile estimation, statistical regression
detection, latency-threshold/anomaly alert rules, and trace critical-path
analysis.
psyche-predictive-performance-modeling#
Predictive performance modeling (src/index.ts): multi-metric performance
tracking, model training (linear regression, exponential smoothing,
Holt-Winters, moving average, ensemble), degradation/capacity-exhaustion/anomaly
forecasting, risk assessment, and orchestrated monitoring sessions.
psyche-quality-trend-analysis#
Domain-agnostic quality-trend analysis (src/index.ts): multi-metric
time-series tracking, configurable smoothing, linear-regression trending,
degradation/ improvement and changepoint detection, forecasting, ensemble
scoring, alerting, and orchestrated monitoring sessions.
@psyche/disaster-recovery#
Kubernetes disaster recovery (src/index.ts): backup scheduling/execution/
verification/retention, recovery with validation+rollback, cross-site
replication with lag monitoring, RPO/RTO/SLA tracking, failover-plan
orchestration, DR test planning, and posture scoring.
@psyche/k8s-federation#
Kubernetes multi-cluster federation (src/index.ts): cluster registration/
lifecycle/health, federated resource distribution with pluggable placement,
cross-cluster service discovery, multi-strategy traffic routing (round-robin,
weighted, latency, geo, failover), and automatic failover/recovery.
@psyche/k8s-security#
Kubernetes security (src/index.ts): IRSA (IAM Roles for Service Accounts) for
AWS EKS, Pod Security Standards validation, Network Policy management, and RBAC
configuration/analysis.
@psyche/database-sharding#
Database sharding (src/index.ts): strategies (hash, range, consistent-hash,
directory, geographic, composite), a consistent-hash ring with virtual nodes
(FNV-1a/DJB2/Murmur-like), shard-key extraction, query routing (single/
scatter-gather/broadcast/targeted), migration, rebalancing, split/merge, and
hotspot detection.
@psyche/timeseries-optimization#
Time-series optimization (src/index.ts): downsampling, compression, retention
policies, time-bucketing, query optimization, storage tiering, materialized-view
management, and LRU caching, with Zod schemas.
@psyche/human-evaluation#
Continuous human-evaluation pipeline (src/index.ts): evaluation sessions with
counterbalancing, multiple rating types (Likert/binary/ranking/continuous),
attention checks, inter-rater reliability (ICC, Fleiss' Kappa, Krippendorff's
Alpha), significance testing, trend analysis, and quality alerts.
psyche-crowdsourced-evaluation#
Crowdsourced evaluation at scale (src/index.ts): worker management with
qualification tiers, HIT lifecycle, quality control (gold standards, spam
detection), response aggregation (majority vote, weighted average, Dawid-Skene
EM, median, trimmed mean), cost tracking, and campaign orchestration.
@psyche/nps#
Net Promoter Score collection/analysis (src/index.ts): survey management,
response recording, NPS calculation, and an analytics engine producing reports
via createNPSSurveyManager/createNPSAnalyticsEngine/calculateNPS.
@psyche/satisfaction#
Customer satisfaction surveys (CSAT, CES) for AI evaluation (src/index.ts):
survey creation/publishing, response recording, and CSAT/CES calculation via
createSatisfactionSurveyManager/calculateCSAT/calculateCES.
psyche-accessibility#
Accessibility library for Psyche agents (src/index.ts, ~30K LOC): WCAG 2.2
compliance — contrast checking (4.5:1), pointer alternatives for gestures,
minimum target sizes (24×24), redundant-entry prevention — plus captioning,
screen-reader, motor, and hearing accessibility.
psyche-mobile-sdk#
Mobile SDK surface (src/index.ts): iOS and Android SDK modules, cross-platform
wrappers, white-label capabilities, edge/offline deployment, and multi-agent
collaboration (Phase 24.18), re-exported from per-platform modules.
psyche-e2e-tests#
End-to-end test suite (*.e2e.test.ts, ~4K LOC across four specs): vitest tests
exercising avatar rendering, voice pipeline (STT→LLM→TTS with <500ms latency),
conferencing, and performance, using mock providers to simulate without live API
calls. A test project, not a runtime library.
psyche-integration-tests#
Python integration test suite (tests/test_*.py, ~3.3K LOC across three
modules): pytest tests covering cross-service event flows, pipeline integration,
and service communication for the Python foundation. A test project, not a
runtime library.
@psyche/training-data#
Phase 85–86 flywheel producer for the avatar/assistant domain
(libs/psyche/training-data/src), split into two pipelines rather than the
single-pipeline shape its siblings use: PsycheAvatarSignalPipeline captures
avatar-animation signals (typed FlameFrame 3DMM coefficients and VisemeFrame
lip-sync frames) and PsychePerceptionSignalPipeline captures perception
signals, both under an explicit PsycheTrainingConsent and a shared
PsycheTrainingSink seam.