This directory contains the local and production-like Docker Compose topology for Music Shield:
- Shield API server on
:8080 - BullMQ scan workers
- PostgreSQL with pgvector for reference metadata and embedding tables
- Redis for scan queue, cache, and perceptual-hash state
- Qdrant for vector similarity search
Development#
cp infra/themis/music-shield/.env.example infra/themis/music-shield/.env
docker compose --env-file infra/themis/music-shield/.env \
-f infra/themis/music-shield/docker-compose.yml \
--profile dev up -d --build
Health check:
curl http://localhost:8080/health
Prometheus metrics:
curl http://localhost:8080/metrics
Authenticated endpoints use x-api-key; the development default is
themis-dev-key.
Monitoring#
Run Prometheus and Grafana alongside either runtime profile:
docker compose --env-file infra/themis/music-shield/.env \
-f infra/themis/music-shield/docker-compose.yml \
--profile dev --profile monitoring up -d --build
Prometheus is available on http://localhost:9090 by default and scrapes
/metrics from the Shield API. Grafana is available on http://localhost:3000;
the provisioned dashboard is Themis / Themis Music Shield.
Monitoring assets live under monitoring/:
prometheus.ymlscrapes the dev and production-like API service names.alert-rules.ymldefines alerts for p95 scan latency above 60 seconds, engine health failures, newly detected false negatives, and queue depth above 1000.grafana/dashboards/themis-music-shield.jsontracks scan volume, latency percentiles, risk distribution, queue depth, cache hit rate, engine health, and FP/FN rates.
Production-Like#
docker compose --env-file infra/themis/music-shield/.env \
-f infra/themis/music-shield/docker-compose.yml \
--profile prod up -d --build
Before using the prod profile, set strong values for:
THEMIS_SHIELD_API_KEYSTHEMIS_REFERENCE_DB_PASSWORD- image tag and registry variables
- port bindings if the host already runs PostgreSQL, Redis, or Qdrant
Runtime Notes#
The Shield API package is library-first, so the Dockerfile builds the Nx
@themis/shield-api target and runs explicit runtime entrypoints:
- API:
libs/themis/shield-api/dist/main.js - worker:
libs/themis/shield-api/dist/worker.js
Full scans are queued through BullMQ when THEMIS_SHIELD_ASYNC_SCANS=true.
POST /scan/quick runs in-process for low-latency smoke testing. The reference
database initializes from
libs/themis/reference-db/drizzle/0000_themis_reference_db_initial.sql.
Kubernetes#
Kustomize manifests live in infra/themis/music-shield/kubernetes:
base/contains production-oriented API, worker, PostgreSQL/pgvector, Redis, Qdrant, HPA, PVC, ingress, secret-template, network-policy, ServiceMonitor, and PrometheusRule resources.overlays/development/lowers replica counts and uses thelocalimage tag.overlays/production/keeps production replica defaults and expands persistent storage requests.
Apply only after replacing base/secret-template.yaml with sealed, external, or
cluster-managed secrets:
kubectl apply -k infra/themis/music-shield/kubernetes/overlays/production
The worker HPA expects an external metrics adapter that exposes
bullmq_queue_depth with a queue=themis.shield.scan label. The API
ServiceMonitor expects Prometheus Operator CRDs in the cluster and scrapes
/metrics on the http service port.
Reference Database Seeding#
After PostgreSQL is initialized, run the idempotent reference seed pipeline:
THEMIS_REFERENCE_DATABASE_URL=postgresql://themis_shield:themis_shield_dev@localhost:55432/themis_shield \
pnpm --filter @themis/reference-db run seed -- \
--musicbrainz-user-agent=OshunThemisSeeder/1.0 \
--lakh-midi-json=/data/themis/lakh-midi \
--public-domain-json=/data/themis/public-domain
The seed command reports JSON progress lines, creates support tables for bundled common phrases, protected drum breaks, and copyright cases, and safely updates existing rows on re-run. The initial production bootstrap is expected to take 4-8 hours depending on MusicBrainz rate limits and local MIDI/audio throughput.