Eve library

Demeter Troubleshooting Guide

Guide

Revision 1

On this page

Common failure modes across the Demeter API, tests, and local dev, with the concrete cause and fix. Ordered roughly by how often they bite.

API won't start#

Symptom: buildServer() throws on boot; process exits immediately.

  • Env validation failed. src/config.ts validates the environment with Zod and throws on the first missing/invalid variable (e.g. absent JWT_SECRET, malformed DATABASE_URL, or DB_POOL_MIN > DB_POOL_MAX). Read the thrown message — it names the offending variable. Fix the env, restart.
  • Fastify plugin version mismatch. A plugin pinned to the wrong major (e.g. fastify: '4.x' on a 5.x app) makes buildServer() throw at registration. All Demeter plugins must declare fastify: '5.x'. (This was a real bug fixed during the persistence work.)

Database#

Symptom: /health/ready reports checks.database.status !== "ok", or queries hang/error.

  • Postgres unreachable. Confirm the dev stack is up (docker compose -f docker/docker-compose.dev.yml ps) and DATABASE_URL points at it. The readiness probe surfaces DB latency; a connection error there is the fastest signal.
  • Pool exhaustion under load. If requests queue and connectionTimeoutMillis (5s) trips, raise DB_POOL_MAX. The pool drains cleanly on shutdown via the onClose hook; a "Cannot use a pool after calling end" error means something is querying after shutdown began.
  • Missing tables / "relation does not exist". Migrations weren't applied to the target DB. Run pnpm --filter @demeter/api db:migrate.

Integration tests#

Symptom: integration specs fail or skip.

  • Suite skipped entirely. The harness skipIf(!dbAvailable)s when it can't reach demeter_test. Bring up Postgres and ensure DEMETER_TEST_DATABASE_URL (default postgresql://oshun:oshun_dev@localhost:5432/demeter_test) resolves.
  • "relation already exists" / stale schema after editing a migration. The harness only applies new migration files; it never re-applies a changed one. Reset the test schema so it replays from scratch:
    bash
    psql "$DEMETER_TEST_DATABASE_URL" -c 'DROP SCHEMA public CASCADE; CREATE SCHEMA public;'
    
  • Table-count smoke test fails. server.integration.spec.ts asserts the table count (29). A new migration that adds/removes a table must update that assertion in the same change.
  • Auth rate-limit flakiness across tests. Integration specs use distinct per-test source IPs (10.X.*) to dodge the per-user/IP rate limiter; reusing an IP across many requests can trip a 429. Give each test its own IP, or set RATE_LIMIT_ENABLED=false for throughput-style tests.

Auth at runtime#

  • 401 on a valid-looking request. The access token expired — clients must refresh (rotation issues a new refresh token each time; store the newest).
  • Logged out of all devices unexpectedly. A revoked refresh token was replayed, so its whole family was revoked (reuse-detection working as designed). The client should treat this as "re-authenticate".
  • OAuth login fails in tests. Don't call live providers — inject deterministic verifiers via buildServer({ verifiers }) (see ADR-0005).

Rate limiting#

Symptom: 429 Too Many Requests.

  • Expected under burst load per the per-user/IP tier. For load/perf testing set RATE_LIMIT_ENABLED=false (the perf harness does this). Don't disable it in production.

Redis#

  • Rate limiting/caching degraded. If REDIS_URL is unreachable, the readiness probe reports it. Confirm Redis is up in the dev stack and the URL is correct.

Mobile test suite (this CI box)#

Symptom: jest-expo mobile tests fail to run with Could not locate module react-native/... or an expo "winter" runtime require-outside-scope error.

  • This is a known environment limitation, not a code defect: react-native is hoisted to the workspace root (so the app-local node_modules/react-native path the jest config expects is absent), and the expo winter runtime is incompatible with the current jest-runtime sandbox. It blocks the entire mobile suite. Mobile tests are written and type-check (tsc --noEmit) but must be executed in a properly provisioned mobile CI environment.