# Psyche Tavus Integration Service

AI avatar management service for the Psyche AI Virtual Assistant Platform.

Part of the Oshun Platform.

## Overview

The Tavus Integration Service provides comprehensive integration with Tavus's
SOTA avatar models:

- **Phoenix-4**: Gaussian-diffusion rendering model for realistic avatars
- **Sparrow-1**: Turn-taking and dialogue model for natural conversations
- **Raven-1**: Multimodal perception model

This service enables Psyche AI assistants to conduct real-time video
conversations with human-like avatars.

## Features

### Replica Management

- Create digital twins from training videos
- Training status monitoring with progress callbacks
- Video upload pipeline with validation
- S3 and local storage support
- Quality assessment for training videos

### Persona Management

- System prompt configuration builder
- Template library for common personas (Sales, Support, Education, etc.)
- Guardrails and objectives system
- Persona-replica associations
- Template-based persona creation

### Conversation Management

- Real-time video conversation creation
- Audio-only mode support
- Custom greetings and context
- Session management and monitoring
- Recording and captions support

### Video Generation

- Script-to-video generation
- Audio-to-video synchronization
- Background customization
- Batch video processing

### Webhook Handling

- Signature verification
- Event parsing and routing
- Retry logic with exponential backoff
- Dead letter queue for failed events
- Priority-based handler registration

## Tech Stack

- **Language**: Python 3.11+
- **Framework**: FastAPI + Uvicorn
- **HTTP Client**: aiohttp (async)
- **Encryption**: cryptography (Fernet)
- **S3 Integration**: aiobotocore (optional)
- **Testing**: pytest + pytest-asyncio

## Project Structure

```
src/tavus_integration/
├── __init__.py              # Package exports
├── main.py                  # FastAPI application
├── api/
│   ├── __init__.py          # API module exports
│   ├── client.py            # Tavus API client with rate limiting
│   ├── auth.py              # API key management and encryption
│   └── webhooks.py          # Webhook receiver and handlers
├── replica/
│   ├── __init__.py          # Replica module exports
│   └── manager.py           # Replica management and video upload
└── persona/
    ├── __init__.py          # Persona module exports
    └── manager.py           # Persona management and templates
```

## API Endpoints

### Health

- `GET /health` - Health check
- `GET /ready` - Readiness check
- `GET /stats` - Service statistics

### Replicas

- `POST /replicas` - Create a new replica
- `GET /replicas` - List all replicas
- `GET /replicas/{id}` - Get replica by ID
- `DELETE /replicas/{id}` - Delete replica

### Personas

- `POST /personas` - Create a new persona
- `GET /personas` - List all personas
- `GET /personas/{id}` - Get persona by ID
- `DELETE /personas/{id}` - Delete persona
- `GET /personas/templates` - List persona templates

### Conversations

- `POST /conversations` - Create conversation
- `GET /conversations` - List conversations
- `GET /conversations/{id}` - Get conversation
- `POST /conversations/{id}/end` - End conversation

### Videos

- `POST /videos` - Generate video
- `GET /videos` - List videos
- `GET /videos/{id}` - Get video
- `DELETE /videos/{id}` - Delete video

### Webhooks

- `POST /webhooks/tavus` - Receive Tavus webhooks
- `GET /webhooks/stats` - Webhook statistics

## Development

### Using Nx

```bash
# Install dependencies
nx install psyche-tavus-integration

# Development server
nx serve psyche-tavus-integration

# Production server
nx serve-prod psyche-tavus-integration

# Build
nx build psyche-tavus-integration

# Linting
nx lint psyche-tavus-integration
nx lint-fix psyche-tavus-integration

# Formatting
nx format psyche-tavus-integration
nx format-check psyche-tavus-integration

# Type checking
nx typecheck psyche-tavus-integration

# Testing
nx test psyche-tavus-integration
nx test-cov psyche-tavus-integration

# Docker
nx docker-build psyche-tavus-integration
nx docker-run psyche-tavus-integration
```

### Direct Commands

```bash
cd apps/psyche/tavus-integration

# Install dependencies
poetry install

# With S3 support
poetry install --extras s3

# Development server
poetry run uvicorn tavus_integration.main:app --reload --port 8013

# Run tests
poetry run pytest

# Lint
poetry run ruff check src tests
```

## Configuration

### Environment Variables

| Variable               | Description                           | Default    |
| ---------------------- | ------------------------------------- | ---------- |
| `TAVUS_API_KEY`        | Tavus API key                         | Required   |
| `TAVUS_ENVIRONMENT`    | API environment (production/sandbox)  | production |
| `TAVUS_WEBHOOK_SECRET` | Webhook signature verification secret | -          |
| `TAVUS_MASTER_KEY`     | Master key for API key encryption     | -          |
| `CORS_ORIGINS`         | Allowed CORS origins                  | \*         |
| `PORT`                 | Service port                          | 8013       |
| `HOST`                 | Service host                          | 0.0.0.0    |

### S3 Video Upload

For S3-based video uploads, configure:

| Variable                | Description                   |
| ----------------------- | ----------------------------- |
| `AWS_ACCESS_KEY_ID`     | AWS access key                |
| `AWS_SECRET_ACCESS_KEY` | AWS secret key                |
| `AWS_DEFAULT_REGION`    | AWS region                    |
| `TAVUS_S3_BUCKET`       | S3 bucket for training videos |

## Usage Examples

### Create a Replica

```python
import httpx

async with httpx.AsyncClient() as client:
    response = await client.post(
        "http://localhost:8013/replicas",
        json={
            "name": "My Avatar",
            "video_url": "https://example.com/training-video.mp4",
            "callback_url": "https://my-app.com/webhooks/tavus",
        }
    )
    replica = response.json()
    print(f"Created replica: {replica['replica_id']}")
```

### Create a Persona

```python
response = await client.post(
    "http://localhost:8013/personas",
    json={
        "name": "Sales Assistant",
        "role": "Sales Representative",
        "organization": "Acme Corp",
        "tone": "professional",
        "personality_traits": ["helpful", "knowledgeable"],
        "expertise_areas": ["product features", "pricing"],
        "default_replica_id": "replica_123",
    }
)
```

### Start a Conversation

```python
response = await client.post(
    "http://localhost:8013/conversations",
    json={
        "persona_id": "persona_456",
        "replica_id": "replica_123",
        "conversation_name": "Sales Demo",
        "custom_greeting": "Hi! I'm here to help you learn about our products.",
    }
)
conversation = response.json()
print(f"Join URL: {conversation['conversation_url']}")
```

### Generate a Video

```python
response = await client.post(
    "http://localhost:8013/videos",
    json={
        "replica_id": "replica_123",
        "script": "Welcome to our product demonstration...",
        "video_name": "Product Demo",
    }
)
```

## Persona Templates

Built-in templates for common use cases:

- **sales_representative** - Sales and business development
- **customer_support** - Customer support and issue resolution
- **interview_screener** - Job interview screening
- **educational_tutor** - Educational tutoring

## Docker

### Build

```bash
nx docker-build psyche-tavus-integration
```

### Run

```bash
nx docker-run psyche-tavus-integration
```

The container:

- Runs on port 8013
- Uses Python 3.11 slim image
- Includes health checks
- Non-root user (psyche)

## License

Proprietary - Oshun Platform