Oshun ECS Architecture — Infrastructure

This document describes the Amazon ECS (Elastic Container Service) architecture used to run Oshun's containerized services.

Overview#

Oshun uses AWS ECS with Fargate for running containerized microservices. The architecture prioritizes reliability, scalability, and cost efficiency.

text

┌─────────────────────────────────────────────────────────────────────────────┐
│                              AWS Cloud                                       │
│  ┌───────────────────────────────────────────────────────────────────────┐  │
│  │                           VPC (10.x.0.0/16)                            │  │
│  │  ┌─────────────────────────────────────────────────────────────────┐  │  │
│  │  │                      Public Subnets                              │  │  │
│  │  │  ┌─────────────┐  ┌─────────────────────────────────────────┐  │  │  │
│  │  │  │ NAT Gateway │  │      Application Load Balancer          │  │  │  │
│  │  │  └─────────────┘  │  (HTTPS → HTTP, WAF, SSL Termination)   │  │  │  │
│  │  │                    └──────────────────┬──────────────────────┘  │  │  │
│  │  └───────────────────────────────────────│─────────────────────────┘  │  │
│  │                                          │                             │  │
│  │  ┌───────────────────────────────────────│─────────────────────────┐  │  │
│  │  │                      Private Subnets  │                          │  │  │
│  │  │  ┌────────────────────────────────────┴───────────────────────┐ │  │  │
│  │  │  │                      ECS Cluster                            │ │  │  │
│  │  │  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐         │ │  │  │
│  │  │  │  │   API       │  │   Worker    │  │  Frontend   │         │ │  │  │
│  │  │  │  │   Service   │  │   Service   │  │   Service   │         │ │  │  │
│  │  │  │  │  (Fargate)  │  │  (Fargate)  │  │  (Fargate)  │         │ │  │  │
│  │  │  │  └─────────────┘  └─────────────┘  └─────────────┘         │ │  │  │
│  │  │  └────────────────────────────────────────────────────────────┘ │  │  │
│  │  └─────────────────────────────────────────────────────────────────┘  │  │
│  └───────────────────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────────────────┘

Components#

ECS Cluster#

The ECS cluster uses AWS Fargate as the compute engine, eliminating the need to manage EC2 instances.

Location: infra/terraform/ecs/

Key features:

Container Insights enabled for enhanced monitoring
CloudWatch log groups for container logs
Service discovery via AWS Cloud Map
Execute command enabled for debugging

Capacity providers:

Environment	Providers	Usage
Staging	FARGATE, SPOT	Cost optimization with spot
Production	FARGATE only	Reliability over cost savings

ECS Services#

Services run as Fargate tasks within the cluster.

Location: infra/terraform/ecs-services/

Service	Port	CPU	Memory	Min Tasks	Max Tasks
API	3000	512	1024	2	10
Worker	3001	512	1024	2	10
Frontend	3002	256	512	2	6

Networking#

VPC Configuration:

Staging: 2 Availability Zones, single NAT Gateway
Production: 3 Availability Zones, NAT Gateway per AZ

Subnets:

Type	CIDR Pattern	Purpose
Public	10.x.1-3.0/24	ALB, NAT Gateways
Private	10.x.11-13.0/24	ECS tasks, databases

Security Groups:

text

ALB Security Group
├── Inbound: 443 from 0.0.0.0/0 (HTTPS)
├── Inbound: 80 from 0.0.0.0/0 (HTTP → redirect to HTTPS)
└── Outbound: All traffic to VPC

ECS Service Security Group
├── Inbound: Service port from ALB Security Group
├── Outbound: 443 to 0.0.0.0/0 (AWS APIs, external services)
└── Outbound: Database ports to DB Security Group

Application Load Balancer#

Location: infra/terraform/alb/

The ALB handles traffic routing, SSL termination, and health checks.

Features:

HTTPS only (HTTP redirects to HTTPS)
ACM certificate for SSL/TLS
WAF integration for DDoS protection (production)
Access logs to S3 (production)
Cross-zone load balancing enabled

Target Groups:

Each service has a dedicated target group with:

Health check path: /health
Health check interval: 15-30 seconds
Deregistration delay: 30 seconds
Stickiness: Disabled (stateless services)

Container Images#

Images are stored in Amazon ECR (Elastic Container Registry).

Location: infra/terraform/ecr/

Repositories:

oshun-api - API service
oshun-worker - Background worker service
oshun-frontend - Frontend service
oshun-inference - AI inference service

Image policies:

Environment	Tag Mutability	Scan on Push	Max Images
Staging	MUTABLE	Yes	10
Production	IMMUTABLE	Yes	50

Scaling#

Auto Scaling#

Services use target tracking scaling based on CPU utilization.

Configuration:

hcl

# Staging
autoscaling_min        = 1
autoscaling_max        = 3
autoscaling_cpu_target = 70

# Production
autoscaling_min        = 2
autoscaling_max        = 10
autoscaling_cpu_target = 60

Scale-out behavior:

Scale out: When average CPU > target for 3 consecutive minutes
Scale in: When average CPU < target - 10% for 15 minutes
Cooldown: 60 seconds between scaling actions

Manual Scaling#

For expected traffic spikes:

bash

aws ecs update-service \
  --cluster oshun-production \
  --service oshun-api \
  --desired-count 5

Deployment#

Deployment Strategy#

Rolling deployment (default):

hcl

deployment_maximum_percent         = 200
deployment_minimum_healthy_percent = 100

This ensures zero-downtime deployments by:

Starting new tasks (up to 200% capacity)
Waiting for health checks to pass
Draining old tasks
Terminating old tasks

Blue/Green deployment (optional):

For services requiring instant rollback capability:

hcl

deployment_controller {
  type = "CODE_DEPLOY"
}

See deployment.md for details.

Health Checks#

ALB health checks:

hcl

health_check {
  path                = "/health"
  healthy_threshold   = 2
  unhealthy_threshold = 3
  timeout             = 5
  interval            = 15
  matcher             = "200"
}

ECS health checks:

dockerfile

HEALTHCHECK --interval=30s --timeout=5s --start-period=60s --retries=3 \
  CMD curl -f http://localhost:3000/health || exit 1

Monitoring#

CloudWatch Metrics#

Container Insights metrics:

CpuUtilized / CpuReserved - CPU usage
MemoryUtilized / MemoryReserved - Memory usage
NetworkRxBytes / NetworkTxBytes - Network I/O
StorageReadBytes / StorageWriteBytes - Disk I/O
RunningTaskCount - Number of running tasks

CloudWatch Alarms#

Location: infra/terraform/cloudwatch/

Alarm	Threshold	Period	Action
High CPU	> 80% (staging)	5 min	SNS alert
High CPU	> 70% (prod)	5 min	SNS + PagerDuty
High Memory	> 80%	5 min	SNS alert
High Error Rate	> 1% (prod)	5 min	SNS + PagerDuty
High Latency	> 500ms (prod)	5 min	SNS alert
Unhealthy Host Count	> 0	1 min	SNS alert

Logging#

Container logs are sent to CloudWatch Logs.

Log groups:

/ecs/oshun-{environment}/{service} - Service logs
/ecs/oshun-{environment}/execution - ECS agent logs

Log retention:

Staging: 14 days
Production: 90 days

Querying logs:

bash

# Recent API errors
aws logs filter-log-events \
  --log-group-name /ecs/oshun-production/api \
  --filter-pattern "ERROR" \
  --start-time $(date -d '1 hour ago' +%s000)

# CloudWatch Logs Insights query
aws logs start-query \
  --log-group-name /ecs/oshun-production/api \
  --start-time $(date -d '1 hour ago' +%s) \
  --end-time $(date +%s) \
  --query-string 'fields @timestamp, @message | filter @message like /ERROR/'

Security#

IAM Roles#

Task execution role:

Pull images from ECR
Write logs to CloudWatch
Retrieve secrets from Secrets Manager
Read parameters from SSM Parameter Store

Task role:

Application-specific permissions
Access to S3 buckets
Access to SQS queues
X-Ray tracing (production)

Secrets Management#

Secrets are stored in AWS Secrets Manager and injected as environment variables.

hcl

secrets = [
  {
    name      = "DATABASE_URL"
    valueFrom = "arn:aws:secretsmanager:us-east-1:123456789:secret:oshun/database-url"
  }
]

Network Security#

Tasks run in private subnets (no public IPs)
Outbound internet via NAT Gateway
Security groups restrict traffic to required ports
VPC Flow Logs enabled for auditing

Cost Optimization#

Staging Environment#

Single NAT Gateway (vs. one per AZ)
FARGATE_SPOT capacity provider
Smaller task sizes (256 CPU, 512 Memory)
Shorter log retention (14 days)
No WAF or enhanced monitoring

Production Environment#

Reserved capacity (Savings Plans) for predictable baseline
Auto-scaling for variable load
Right-sized tasks based on actual usage
S3 Intelligent-Tiering for logs

Cost Monitoring#

bash

# Estimate monthly cost
aws ce get-cost-forecast \
  --time-period Start=$(date +%Y-%m-01),End=$(date -d 'next month' +%Y-%m-01) \
  --granularity MONTHLY \
  --metric UNBLENDED_COST \
  --filter '{"Dimensions": {"Key": "SERVICE", "Values": ["Amazon Elastic Container Service"]}}'

Disaster Recovery#

Backup Strategy#

Task definitions stored in Terraform (version controlled)
Container images in ECR with cross-region replication (optional)
Secrets replicated via Terraform (optional KMS cross-region)

Recovery Procedures#

Service failure:

Auto-healing via ECS (unhealthy tasks replaced)
Auto-scaling for capacity issues
Manual scaling if needed

AZ failure:

Tasks automatically redistributed across remaining AZs
ALB health checks stop routing to failed AZ
No manual intervention required (production)

Region failure:

Deploy to secondary region using Terraform
Update DNS to point to new region
Restore data from cross-region backups

Troubleshooting#

Common Issues#

Task fails to start:

bash

# Check stopped task reason
aws ecs describe-tasks \
  --cluster oshun-production \
  --tasks <task-id>

# Common causes:
# - Image pull failure (check ECR permissions)
# - Resource constraints (increase CPU/memory)
# - Health check failure (check /health endpoint)

High CPU/Memory:

bash

# Get service metrics
aws cloudwatch get-metric-statistics \
  --namespace AWS/ECS \
  --metric-name CPUUtilization \
  --dimensions Name=ClusterName,Value=oshun-production Name=ServiceName,Value=oshun-api \
  --start-time $(date -d '1 hour ago' -u +%Y-%m-%dT%H:%M:%SZ) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%SZ) \
  --period 300 \
  --statistics Average

Deployment stuck:

bash

# Check deployment status
aws ecs describe-services \
  --cluster oshun-production \
  --services oshun-api \
  --query 'services[0].deployments'

# Force new deployment
aws ecs update-service \
  --cluster oshun-production \
  --service oshun-api \
  --force-new-deployment

Useful Commands#

bash

# List all services
aws ecs list-services --cluster oshun-production

# Get service details
aws ecs describe-services --cluster oshun-production --services oshun-api

# List running tasks
aws ecs list-tasks --cluster oshun-production --service-name oshun-api

# Execute command in container
aws ecs execute-command \
  --cluster oshun-production \
  --task <task-id> \
  --container api \
  --interactive \
  --command "/bin/sh"

# Get task definition
aws ecs describe-task-definition --task-definition oshun-api:latest

Overview#

Components#

ECS Cluster#

ECS Services#

Networking#

Application Load Balancer#

Container Images#

Scaling#

Auto Scaling#

Manual Scaling#

Deployment#

Deployment Strategy#

Health Checks#

Monitoring#

CloudWatch Metrics#

CloudWatch Alarms#

Logging#

Security#

IAM Roles#

Secrets Management#

Network Security#

Cost Optimization#

Staging Environment#

Production Environment#

Cost Monitoring#

Disaster Recovery#

Backup Strategy#

Recovery Procedures#

Troubleshooting#

Common Issues#

Useful Commands#

Related Documentation#