Eve library

Environment Configuration Management

Guide

Revision 1

On this page

Overview#

This document describes the comprehensive environment configuration management system for the Lilith platform, supporting multiple deployment environments (development, staging, production) with environment-specific configurations, security policies, and deployment automation.

Architecture#

Environment Types#

  1. Development (dev): Local development with debugging features enabled
  2. Staging: Pre-production testing environment
  3. Production (prod): Live production environment with full security

Directory Structure#

text
config/
├── environments/
│   ├── development/
│   │   ├── .env                    # Environment variables
│   │   ├── services/               # Service-specific configs
│   │   │   ├── bff/
│   │   │   ├── ai/
│   │   │   └── ...
│   │   ├── vault/                 # Vault configurations
│   │   ├── monitoring/            # Monitoring configs
│   │   ├── networking/            # Network policies
│   │   └── security/              # Security policies
│   ├── staging/
│   │   └── [same structure as development]
│   └── production/
│       └── [same structure as development]
├── active -> environments/development    # Symlink to active environment
└── .env.active                             # Current environment file

Configuration Components#

1. Environment Variables (.env)#

Each environment has its own .env file with:

Development Environment#

bash
NODE_ENV=development
ENVIRONMENT=development
DEBUG=true
HOT_RELOAD=true
LOG_LEVEL=debug

# URLs
API_BASE_URL=http://localhost:8080
WEB_BASE_URL=http://localhost:3000
VAULT_ADDR=http://localhost:8200

# Database
DB_HOST=localhost
DB_PORT=5433
DB_NAME=lilith_dev
DB_USER=lilith
DB_PASSWORD=dev_password

# Resources (generous)
CPU_LIMIT=2
MEMORY_LIMIT=4Gi

Production Environment#

bash
NODE_ENV=production
ENVIRONMENT=production
DEBUG=false
LOG_LEVEL=warn

# URLs
API_BASE_URL=https://api.lilith.app
WEB_BASE_URL=https://lilith.app
VAULT_ADDR=https://vault.lilith.app

# Database
DB_HOST=postgres-prod.internal
DB_PORT=5432
DB_NAME=lilith_prod
DB_USER=lilith_prod
DB_PASSWORD=

# Security
ENCRYPTION_ENABLED=true
AUDIT_LOGGING_ENABLED=true
RATE_LIMITING_ENABLED=true

# Resources (optimized)
CPU_LIMIT=1
MEMORY_LIMIT=1Gi

2. Docker Compose Overrides#

Each environment gets its own Docker Compose override file:

Development (docker-compose.development.yml)#

yaml
services:
  bff:
    environment:
      - NODE_ENV=development
      - DEBUG=true
    volumes:
      - ./services/bff/src:/app/src
      - ./config/environments/development/.env:/app/.env:ro
    command: ['npm', 'run', 'dev']

  # Development tools
  nginx-proxy:
    image: nginx:alpine
    ports:
      - '8081:80'

Production (docker-compose.production.yml)#

yaml
services:
  bff:
    environment:
      - NODE_ENV=production
      - LOG_LEVEL=warn
    restart: unless-stopped
    deploy:
      replicas: 2
      resources:
        limits:
          memory: 1Gi
        reservations:
          memory: 512Mi
      update_config:
        parallelism: 1
        delay: 10s
        failure_action: rollback

3. Vault Configuration#

Environment-specific Vault configurations:

Development Vault (config/environments/development/vault/vault.hcl)#

hcl
ui = true
api_addr = "http://0.0.0.0:8200"
cluster_addr = "http://0.0.0.0:8201"

backend "file" {
  path = "/vault/data"
}

listener "tcp" {
  address = "0.0.0.0:8200"
  tls_disable = true
  cluster_address = "0.0.0.0:8201"
}

log_level = "debug"

Production Vault (config/environments/production/vault/vault.hcl)#

hcl
ui = true
api_addr = "https://vault.lilith.app"
cluster_addr = "https://vault.lilith.app:8201"

backend "file" {
  path = "/vault/data"
}

listener "tcp" {
  address = "0.0.0.0:8200"
  tls_disable = false
  cluster_address = "0.0.0.0:8201"
}

log_level = "info"

4. Service Configurations#

Each service gets environment-specific configuration:

Service Configuration JSON (config/environments/production/services/bff/config.json)#

json
{
  "service": "bff",
  "environment": "production",
  "version": "1.0.0",
  "features": {
    "healthChecks": true,
    "metrics": true,
    "tracing": true,
    "debug": false
  },
  "endpoints": {
    "health": "/health",
    "metrics": "/metrics",
    "readiness": "/ready"
  },
  "resources": {
    "cpu": {
      "request": "100m",
      "limit": "500m"
    },
    "memory": {
      "request": "256Mi",
      "limit": "1Gi"
    }
  },
  "environmentSpecific": {
    "debug": false,
    "logLevel": "warn",
    "performanceMonitoring": true,
    "errorReporting": true
  }
}

Usage#

1. Environment Setup#

Generate All Configurations#

bash
# Generate configurations for all environments
./scripts/config-manager.sh all

# Generate for specific environment
./scripts/config-manager.sh generate development
./scripts/config-manager.sh generate staging
./scripts/config-manager.sh generate production

Validate Configuration#

bash
# Validate specific environment
./scripts/config-manager.sh validate production

# This checks:
# • Required files exist
# • JSON configurations are valid
# • Docker compose files are valid
# • No missing service configurations

2. Environment Switching#

Quick Switch#

bash
# Switch to production environment
./scripts/config-manager.sh switch production

# Switch to development
./scripts/config-manager.sh switch dev

Using Environment Switch Script#

bash
# Switch environments
./scripts/switch-env.sh prod
./scripts/switch-env.sh staging
./scripts/switch-env.sh dev

# Show current environment
./scripts/switch-env.sh show

3. Deployment#

Development Deployment#

bash
# Switch to development environment
./scripts/switch-env.sh dev

# Deploy development services
./scripts/deploy/development/deploy.sh

Staging Deployment#

bash
# Switch to staging environment
./scripts/switch-env.sh staging

# Deploy staging services
./scripts/deploy/staging/deploy.sh

Production Deployment#

bash
# Switch to production environment
./scripts/switch-env.sh production

# Deploy production services
./scripts/deploy/production/deploy.sh

4. Health Monitoring#

Environment-Specific Health Checks#

bash
# Check current environment
./scripts/switch-env.sh show

# Run health check for current environment
./scripts/deploy/$(echo $ENVIRONMENT | tr '[:upper:]' '[:lower:]')/health-check.sh

Configuration Management Workflow#

1. Development Workflow#

  1. Local Setup

    bash
    ./scripts/config-manager.sh generate development
    ./scripts/switch-env.sh dev
    
  2. Development Services

    bash
    docker compose -f docker-compose.yml -f docker-compose.development.yml up -d
    
  3. Feature Development

    • Modify service code
    • Test locally
    • Validate configuration changes

2. Testing Workflow#

  1. Staging Setup

    bash
    ./scripts/config-manager.sh generate staging
    ./scripts/switch-env.sh staging
    
  2. Deploy to Staging

    bash
    ./scripts/deploy/staging/deploy.sh
    
  3. Integration Testing

    • Run automated tests
    • Validate service integration
    • Performance testing

3. Production Workflow#

  1. Production Setup

    bash
    ./scripts/config-manager.sh generate production
    ./scripts/switch-env.sh production
    
  2. Pre-deployment Validation

    bash
    ./scripts/config-manager.sh validate production
    
  3. Production Deployment

    bash
    ./scripts/deploy/production/deploy.sh
    
  4. Post-deployment Verification

    bash
    ./scripts/deploy/production/health-check.sh
    

Security Considerations#

1. Configuration Security#

Secrets Management#

  • Never commit secrets to version control
  • Use Vault for all sensitive data
  • Environment-specific secret paths in Vault
  • Automatic rotation for critical secrets

Configuration Validation#

  • JSON schema validation for service configs
  • Docker compose validation before deployment
  • Environment variable validation for required settings

2. Network Security#

Development Environment#

  • Localhost access only
  • Debug endpoints enabled
  • Mock services for external APIs

Staging Environment#

  • Internal network access only
  • Real services (with staging data)
  • Security features enabled

Production Environment#

  • Full security features enabled
  • Load balancer with SSL termination
  • Network policies and service mesh
  • External access through controlled endpoints

3. Access Control#

Environment Access#

bash
# Development: All developers
# Staging: Development and QA teams
# Production: DevOps and authorized personnel only

Service Configuration#

bash
# Production services:
# - No debug endpoints
# - Limited log levels
# - Error reporting only
# - Performance monitoring enabled

Monitoring and Observability#

1. Environment Monitoring#

Service Health#

bash
# Health check endpoints for each service
GET /health
GET /ready
GET /metrics

Configuration Monitoring#

bash
# Track configuration changes
# Validate environment consistency
# Monitor service restarts

2. Alerting#

Environment-Specific Alerts#

  • Development: Error notifications
  • Staging: Performance and error alerts
  • Production: Critical alerts with escalation

Configuration Drift Detection#

bash
# Compare configurations
./scripts/config-manager.sh compare development staging
./scripts/config-manager.sh compare staging production

Best Practices#

1. Configuration Management#

Version Control#

  • Configuration files in version control
  • Secrets never committed
  • Environment templates for consistency

Change Management#

  • Review process for production changes
  • Staging validation before production
  • Rollback plans for critical changes

2. Environment Consistency#

Standardization#

  • Consistent structure across environments
  • Standard naming conventions
  • Required fields validation

Testing#

  • Configuration validation in CI/CD
  • Environment testing before deployment
  • Rollback testing procedures

3. Operational Excellence#

Automation#

  • Automated deployment pipelines
  • Configuration validation scripts
  • Health monitoring and alerting

Documentation#

  • Comprehensive documentation for each environment
  • Runbooks for common operations
  • Troubleshooting guides for issues

Troubleshooting#

1. Common Issues#

Configuration Validation Failures#

bash
# Check configuration syntax
./scripts/config-manager.sh validate production

# Check JSON files
jq empty config/environments/production/services/*/config.json

Environment Switch Failures#

bash
# Check directory permissions
ls -la config/environments/
ls -la config/environments/active

# Check symlink
ls -la config/.env.active

Deployment Failures#

bash
# Check environment variables
cat config/environments/active/.env

# Check docker compose files
docker compose -f docker-compose.yml -f docker-compose.production.yml config

2. Debug Configuration#

Enable Debug Mode#

bash
# Add debug logging
export DEBUG_CONFIG=true
export LOG_LEVEL=debug

# Check configuration loading
./scripts/deploy/$(echo $ENVIRONMENT | tr '[:upper:]' '[:lower:]')/deploy.sh --dry-run

Validate Service Configuration#

bash
# Check service config files
cat config/environments/$(echo $ENVIRONMENT | tr '[:upper:]' '[:lower:]')/services/bff/config.json

# Validate JSON
jq empty config/environments/$(echo $ENVIRONMENT | tr '[:upper:]' '[:lower:]')/services/bff/config.json

3. Health Monitoring#

Service Health#

bash
# Check service health
curl -f http://localhost:8080/health

# Check all services
./scripts/health-monitor.sh all

Configuration Health#

bash
# Check current environment
./scripts/switch-env.sh show

# Validate configuration
./scripts/config-manager.sh validate $ENVIRONMENT

Advanced Features#

1. Configuration Templates#

Template Generation#

bash
# Generate new environment template
./scripts/config-manager.sh generate custom-environment

# Customize based on existing environment
cp -r config/environments/production config/environments/custom-environment
# Modify as needed

2. Environment-Specific Features#

Feature Flags#

json
{
  "features": {
    "experimentalFeatures": {
      "development": true,
      "staging": false,
      "production": false
    },
    "advancedMonitoring": {
      "development": false,
      "staging": true,
      "production": true
    }
  }
}

Resource Scaling#

json
{
  "resources": {
    "autoScaling": {
      "development": false,
      "staging": true,
      "production": true
    },
    "maxReplicas": {
      "development": 1,
      "staging": 2,
      "production": 5
    }
  }
}

3. Multi-Region Support#

Region Configuration#

bash
# Add region-specific configurations
mkdir -p config/environments/production/us-east-1
mkdir -p config/environments/production/us-west-2

# Region-specific overrides
echo "REGION=us-east-1" >> config/environments/production/us-east-1/.env
echo "CLUSTER=lilith-prod-use1" >> config/environments/production/us-east-1/.env

Integration with CI/CD#

1. GitHub Actions#

Environment Configuration#

yaml
name: Deploy to Production

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Generate Production Config
        run: ./scripts/config-manager.sh generate production

      - name: Validate Configuration
        run: ./scripts/config-manager.sh validate production

      - name: Deploy to Production
        run: ./scripts/deploy/production/deploy.sh

2. Environment Variables in CI/CD#

Secure Configuration Loading#

bash
# Load environment-specific configuration
source config/environments/$ENVIRONMENT/.env

# Use configuration in deployment
docker compose -f docker-compose.yml -f docker-compose.$ENVIRONMENT.yml up -d

3. Testing Pipeline#

Multi-Environment Testing#

yaml
# Staging tests
- name: Run Staging Tests
  run: |
    ./scripts/switch-env.sh staging
    ./scripts/deploy/staging/deploy.sh
    ./scripts/deploy/staging/health-check.sh
    npm run test:e2e:staging

# Production validation
- name: Production Validation
  run: |
    ./scripts/config-manager.sh validate production
    ./scripts/deploy/production/deploy.sh --dry-run

Migration Guide#

1. Existing Environment Migration#

Backup Current Configuration#

bash
# Create backup directory
mkdir -p backups/config-migration-$(date +%Y%m%d)

# Backup current configurations
cp -r docker-compose.* backups/config-migration-$(date +%Y%m%d)/
cp -r .env* backups/config-migration-$(date +%Y%m%d)/ 2>/dev/null || true

Migrate to New System#

bash
# Generate new configurations
./scripts/config-manager.sh all

# Validate new configurations
for env in development staging production; do
    ./scripts/config-manager.sh validate $env
done

# Test migration
./scripts/switch-env.sh development
# Test services work with new configuration

2. Service Migration#

Update Service Configuration Loading#

javascript
// In service startup
import config from './config.json';

// Load environment-specific overrides
const env = process.env.ENVIRONMENT || 'development';
const envConfigPath = `config/environments/${env}/services/${process.env.SERVICE_NAME}/config.json`;

try {
  const envConfig = require(envConfigPath);
  config = { ...config, ...envConfig };
} catch (error) {
  console.warn(`No environment-specific config found, using defaults`);
}

Configuration Validation#

javascript
// Validate required configuration
const requiredFields = ['service', 'environment', 'endpoints'];
for (const field of requiredFields) {
  if (!config[field]) {
    throw new Error(`Missing required configuration: ${field}`);
  }
}

References#

  • Configuration Scripts: scripts/config-manager.sh
  • Environment Switch: scripts/switch-env.sh
  • Deployment Scripts: scripts/deploy/*/
  • Health Monitoring: scripts/health-monitor.sh
  • Vault Setup: docs/deploy/VAULT_SETUP.md
  • Secret Rotation: docs/deploy/VAULT_MIGRATION.md