# Local Development Setup Guide

This guide walks you through setting up the Lilith platform for local
development.

## Overview

The Lilith platform is a microservices-based system with 76+ services
orchestrated via Docker Compose. This guide covers everything needed to get a
complete development environment running.

**Time to complete**: ~15-30 minutes (depending on download speeds)

---

## Prerequisites

### Required Software

| Software           | Minimum Version | Verification Command     |
| ------------------ | --------------- | ------------------------ |
| **Docker**         | Latest stable   | `docker --version`       |
| **Docker Compose** | v2.0+           | `docker compose version` |
| **Node.js**        | 18.0.0+         | `node --version`         |
| **npm**            | 8.0+            | `npm --version`          |
| **Git**            | 2.30+           | `git --version`          |

### System Requirements

| Resource       | Minimum | Recommended |
| -------------- | ------- | ----------- |
| **RAM**        | 8 GB    | 16 GB       |
| **Disk Space** | 20 GB   | 50 GB       |
| **CPU Cores**  | 4       | 8           |

### Port Availability

Ensure these port ranges are available:

- **3000-3050**: Application services
- **5433**: PostgreSQL
- **6380**: Redis
- **9000-9301**: MinIO, Prometheus, Elasticsearch
- **16686**: Jaeger UI
- **19530**: Milvus

---

## Quick Start (Single Command)

For the fastest setup, run:

```bash
# Clone the repository
git clone https://github.com/lilith/platform.git
cd platform

# Install dependencies
npm install

# Deploy everything (databases, services, monitoring)
npm run deploy
```

This single command:

1. Starts all infrastructure (PostgreSQL, Redis, Elasticsearch, Neo4j, Milvus,
   MinIO)
2. Waits for health checks to pass
3. Starts all application services
4. Starts observability stack (Prometheus, Grafana, Jaeger)

**First run takes 5-10 minutes** as Docker images are pulled.

---

## Step-by-Step Setup

### Step 1: Clone the Repository

```bash
git clone https://github.com/lilith/platform.git
cd platform
```

### Step 2: Install Dependencies

```bash
npm install
```

This installs root workspace dependencies. Individual service dependencies are
managed within each service directory.

### Step 3: Start Infrastructure

The platform uses Docker Compose with multiple infrastructure services:

```bash
# Start just infrastructure (databases + monitoring)
docker compose -f docker-compose.dev.yml up -d postgres redis elasticsearch neo4j milvus minio
```

**Infrastructure Services**:

| Service       | Port                       | Purpose                                         |
| ------------- | -------------------------- | ----------------------------------------------- |
| PostgreSQL    | 5433                       | Relational data (users, content, conversations) |
| Redis         | 6380                       | Cache, sessions, pub/sub                        |
| Elasticsearch | 9201                       | Full-text search, logging                       |
| Neo4j         | 7475 (HTTP), 7688 (Bolt)   | Knowledge graph                                 |
| Milvus        | 19530                      | Vector database (embeddings)                    |
| MinIO         | 9000 (API), 9001 (Console) | S3-compatible object storage                    |

### Step 4: Verify Infrastructure Health

```bash
# Run health checks
npm run infrastructure:health
```

Expected output:

```
✓ PostgreSQL: healthy
✓ Redis: healthy
✓ Elasticsearch: healthy
✓ Neo4j: healthy
✓ Milvus: healthy
✓ MinIO: healthy
```

If any service fails, see [Troubleshooting](#troubleshooting) below.

### Step 5: Start Application Services

```bash
# Start all services
docker compose -f docker-compose.dev.yml up -d
```

Or use the npm script:

```bash
npm run deploy
```

### Step 6: Verify Service Health

```bash
# Check all service health
npm run health

# Or wait for all services to be healthy (up to 5 min)
npm run health:wait
```

### Step 7: Seed Development Data

```bash
# Seed databases with sample data
npm run seed:dev
```

This populates:

- Sample users and profiles
- Demo content (meditations, lectures)
- Knowledge base documents
- AI templates and personas
- Sample notifications

---

## Service Architecture

### Port Assignments

**Core Services (3000-3012)**:

| Port | Service      | Description                              |
| ---- | ------------ | ---------------------------------------- |
| 3000 | BFF          | Backend-for-Frontend API Gateway         |
| 3001 | Conversation | Chat and message management              |
| 3002 | AI           | LLM orchestration and multi-agent system |
| 3003 | Auth         | Authentication and authorization         |
| 3004 | Content      | Courses, curricula, versioning           |
| 3005 | Media        | Asset storage and streaming              |
| 3006 | Notification | Push and email notifications             |
| 3007 | Moderation   | Content safety and filtering             |
| 3008 | RAG          | Retrieval-augmented generation           |
| 3009 | TTS          | Text-to-speech synthesis                 |
| 3010 | STT          | Speech-to-text recognition               |
| 3011 | Ingestion    | Content pipeline processing              |
| 3012 | Knowledge    | Knowledge graph management               |

**Extended Services (3013-3043)**: Additional domain services for meditation,
community, blockchain, etc.

**Observability Stack**:

| Port  | Service         | Description           |
| ----- | --------------- | --------------------- |
| 9090  | Prometheus      | Metrics collection    |
| 9092  | Grafana         | Metrics visualization |
| 9093  | AlertManager    | Alert routing         |
| 16686 | Jaeger          | Distributed tracing   |
| 8082  | Redis Commander | Redis UI              |
| 8083  | pgAdmin         | PostgreSQL UI         |

---

## Environment Configuration

### Default Development Values

The development environment uses sensible defaults. No `.env` file is required
for basic development.

**Default Credentials (Development Only)**:

| Service    | Username         | Password       |
| ---------- | ---------------- | -------------- |
| PostgreSQL | `lilith`         | `dev_password` |
| Neo4j      | `neo4j`          | `dev_password` |
| MinIO      | `minioadmin`     | `minioadmin`   |
| JWT Secret | `dev-secret-key` | -              |

### Environment Variables

Key environment variables (with defaults):

```bash
# Database
POSTGRES_HOST=postgres
POSTGRES_PORT=5432
POSTGRES_DB=lilith_dev
POSTGRES_USER=lilith
POSTGRES_PASSWORD=dev_password

# Cache
REDIS_HOST=redis
REDIS_PORT=6379

# Vector Database
MILVUS_ADDRESS=milvus:19530

# Knowledge Graph
NEO4J_URI=bolt://neo4j:7687
NEO4J_USER=neo4j
NEO4J_PASSWORD=dev_password

# Object Storage
MINIO_ENDPOINT=minio:9000
MINIO_ACCESS_KEY=minioadmin
MINIO_SECRET_KEY=minioadmin

# AI Providers (optional for development)
OPENAI_API_KEY=sk-test-key
ANTHROPIC_API_KEY=sk-ant-test-key

# Security
JWT_SECRET=dev-secret-key
SAFETY_MODE=strict
```

### Custom Configuration

To override defaults, create a `.env.local` file:

```bash
# .env.local
OPENAI_API_KEY=sk-your-real-key
ANTHROPIC_API_KEY=sk-ant-your-real-key
```

---

## Database Setup

### PostgreSQL Initialization

The database is automatically initialized with:

```sql
-- Extensions
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
CREATE EXTENSION IF NOT EXISTS "pgcrypto";
CREATE EXTENSION IF NOT EXISTS "btree_gin";
CREATE EXTENSION IF NOT EXISTS "pg_stat_statements";

-- Schemas
CREATE SCHEMA IF NOT EXISTS auth;
CREATE SCHEMA IF NOT EXISTS content;
CREATE SCHEMA IF NOT EXISTS conversation;
CREATE SCHEMA IF NOT EXISTS media;
CREATE SCHEMA IF NOT EXISTS notification;
CREATE SCHEMA IF NOT EXISTS audit;
```

### Manual Database Access

```bash
# Connect to PostgreSQL
docker exec -it lilith-postgres psql -U lilith -d lilith_dev

# Connect to Redis
docker exec -it lilith-redis redis-cli

# Connect to Neo4j (via browser)
open http://localhost:7475
```

### Reset Databases

```bash
# Full reset with backup
npm run reset:all

# Force reset (no backup)
npm run reset:force
```

---

## Service Startup Order

Services must start in a specific order due to dependencies:

### Phase 1: Infrastructure (Required First)

```
PostgreSQL ─┬─► Redis ─┬─► Elasticsearch
            │          │
            ├─► Neo4j  ├─► Milvus
            │          │
            └─► MinIO  └─► etcd (Milvus dependency)
```

### Phase 2: Core Services

```
Auth ─────────────────────────────────►
                                        │
Conversation ───► Content ───► Media ──►├──► BFF
                                        │
Notification ───► Moderation ──────────►│
                                        │
AI ───► RAG ───────────────────────────►
```

### Phase 3: Observability

```
Prometheus ───► Grafana ───► AlertManager
                │
Jaeger ─────────┘
```

---

## Development Workflow

### Running Individual Services

For focused development on a single service:

```bash
# Navigate to service
cd services/conversation

# Install dependencies
npm install

# Start in development mode
npm start

# Or with custom port
PORT=3001 npm start
```

### Hot Reload Development

```bash
# Smart hot reload (watches for changes)
npm run dev:hot-smart

# Full hot reload
npm run dev:hot

# Minimal (core services only)
npm run dev:hot-minimal
```

### Running Tests

```bash
# Run all E2E tests
npm test

# CI mode (minimal output)
npm run test:ci

# Local development validation
npm run test:local-dev

# Individual service tests
cd services/auth
npm test
```

### Viewing Logs

```bash
# Follow all logs
npm run deploy:logs

# Specific service logs
docker logs -f lilith-bff

# Log dashboard
npm run logs:dashboard
```

### Checking Status

```bash
# Service status
npm run deploy:status

# Health report
npm run health:report

# Database connectivity
npm run database:connectivity
```

---

## Useful URLs

After successful deployment:

| URL                            | Purpose            |
| ------------------------------ | ------------------ |
| http://localhost:3000/health   | BFF health check   |
| http://localhost:3000/graphiql | GraphQL playground |
| http://localhost:9092          | Grafana dashboards |
| http://localhost:16686         | Jaeger tracing     |
| http://localhost:8082          | Redis Commander    |
| http://localhost:8083          | pgAdmin            |
| http://localhost:9001          | MinIO Console      |
| http://localhost:7475          | Neo4j Browser      |

---

## Troubleshooting

### Common Issues

#### 1. Port Already in Use

**Symptom**: `Error: bind: address already in use`

**Solution**:

```bash
# Find process using the port
lsof -i :3000

# Kill the process
kill -9 <PID>

# Or use a different port
PORT=3100 npm start
```

#### 2. Docker Memory Issues

**Symptom**: Services crash or restart repeatedly

**Solution**:

1. Increase Docker memory allocation to 8GB+ in Docker Desktop settings
2. Reduce running services:

```bash
# Start only essential services
docker compose -f docker-compose.dev.yml up -d postgres redis bff auth
```

#### 3. Database Connection Refused

**Symptom**: `ECONNREFUSED` errors

**Solution**:

```bash
# Check if database is running
docker ps | grep postgres

# Check logs
docker logs lilith-postgres

# Restart database
docker compose -f docker-compose.dev.yml restart postgres
```

#### 4. Milvus Fails to Start

**Symptom**: Milvus container exits immediately

**Solution**:

```bash
# Milvus requires etcd - ensure it's running
docker compose -f docker-compose.dev.yml up -d etcd

# Wait for etcd
sleep 30

# Then start Milvus
docker compose -f docker-compose.dev.yml up -d milvus
```

#### 5. Health Checks Timeout

**Symptom**: `npm run health:wait` times out

**Solution**:

```bash
# Check which services are failing
npm run health

# View logs of failing service
docker logs lilith-<service-name>

# Restart specific service
docker compose -f docker-compose.dev.yml restart <service-name>
```

#### 6. Permission Denied on Scripts

**Symptom**: `Permission denied` when running scripts

**Solution**:

```bash
chmod +x scripts/*.sh
```

### Health Check Timeouts

Expected health check times:

| Service       | Timeout |
| ------------- | ------- |
| PostgreSQL    | 60s     |
| Redis         | 30s     |
| Elasticsearch | 60s     |
| Neo4j         | 60s     |
| Milvus        | 90s     |
| MinIO         | 30s     |

If services take longer, check Docker resource allocation.

### Resetting Everything

If all else fails:

```bash
# Stop all containers
npm run deploy:stop

# Remove all volumes (WARNING: deletes data)
docker compose -f docker-compose.dev.yml down -v

# Remove Docker cache
docker system prune -af

# Start fresh
npm run deploy
```

---

## Next Steps

After successful setup:

1. **Explore the API**: Visit http://localhost:3000/graphiql
2. **Read Architecture**: See
   [System Architecture](../architecture/system-architecture.md)
3. **Run Tests**: `npm test`
4. **Check Contributing Guide**: See [CONTRIBUTING.md](../../CONTRIBUTING.md)

---

## Related Documentation

- [System Architecture](../architecture/system-architecture.md)
- [API Reference](../reference/typescript-sdk.md)
- [GraphQL API](../reference/graphql-api.md)
- [Authentication](../reference/authentication.md)

---

## Support

- **GitHub Issues**: Report setup problems
- **Slack**: #dev-help channel
- **Documentation**: https://docs.lilith.ai
