# Service Level Objectives

Owner: platform team Last reviewed: 2026-05-28

Production SLOs for the V1 services. Each row gates the alert routing in
[alert-routing.md](alert-routing.md) and the budget calculations in the Grafana
"SLO" dashboards.

This doc is policy. Per-service implementation lives in:

- `infra/monitoring/<domain>/grafana/dashboards/*-slo.json` — Grafana SLO panel
  definitions.
- `infra/monitoring/<domain>/prometheus/alerts.yml` — alerting rules (latency,
  error rate).
- `libs/shared/observability` — instrumentation helpers.

Eve's cross-service assistant path additionally uses the source-verified,
per-plane contract in
[`EVE_SOTA_RELIABILITY_SLO_CONTRACT_2026-09.md`](../audits/EVE_SOTA_RELIABILITY_SLO_CONTRACT_2026-09.md).
That contract is stricter where a service average could conceal failed tasks,
late or duplicate effects, missing cost, grounding failure, or a data-boundary
violation. Missing plane telemetry is `UNKNOWN`, never an availability pass.

## SLO budget rules

- Each SLO has an **error budget** = `(1 − target) × measurement window`.
- We measure on a 30-day rolling window.
- If a service burns >50% of its budget in a single week, the domain freezes
  non-critical deploys until the budget is restored.
- Budget burns trigger a postmortem ([on-call.md](on-call.md) §Postmortem).

## Targets

### User-facing surfaces

| Service                 | Availability | Latency p95                 | Latency p99 | Notes                                                                           |
| ----------------------- | ------------ | --------------------------- | ----------- | ------------------------------------------------------------------------------- |
| oshun-shell             | 99.95%       | 200 ms (TTFB)               | 500 ms      | Front-door shell; outage = full product outage                                  |
| iris/api                | 99.9%        | 400 ms                      | 1.0 s       | Assistant chat — most critical user surface                                     |
| lilith/svc-conversation | 99.9%        | 250 ms (text)               | 600 ms      | WebSocket message processing                                                    |
| aphrodite/payment       | 99.95%       | 600 ms                      | 1.5 s       | Checkout flow; budget = revenue                                                 |
| aphrodite/streaming     | 99.5%        | n/a (streaming)             | n/a         | RTMP availability; viewer-side latency in `aphrodite-streaming-viewer-slo.json` |
| nyx/api                 | 99.5%        | 200 ms                      | 500 ms      | Astronomy data API; mostly cacheable                                            |
| tara/api                | 99.5%        | 300 ms                      | 750 ms      | Meditation session APIs                                                         |
| veritas/api             | 99.5%        | 500 ms                      | 1.5 s       | News/content; high read volume                                                  |
| yemaya/api              | 99.9%        | 300 ms                      | 800 ms      | Identity / tenant / settings                                                    |
| metis/api-gateway       | 99.95%       | 50 ms (forwarding overhead) | 150 ms      | Edge gateway                                                                    |

### Internal / async surfaces

| Service                   | Availability | Job p95 | Job p99 | Notes                                    |
| ------------------------- | ------------ | ------- | ------- | ---------------------------------------- |
| lilith/svc-ai (inference) | 99.5%        | 4 s     | 10 s    | Model inference; latency varies by model |
| lilith/svc-tts            | 99.5%        | 2 s     | 5 s     | TTS synthesis                            |
| isis/generation-api       | 99.5%        | 30 s    | 90 s    | Image / video generation                 |
| veritas/ingestion         | 99.5%        | 60 s    | 300 s   | RSS / sitemap collectors                 |
| veritas/video             | 99.5%        | 5 min   | 15 min  | Video generation pipeline                |
| yemaya/workers            | 99.5%        | 30 s    | 120 s   | Notification / email workers             |
| oshun/bff (worker mode)   | 99.9%        | 1 s     | 3 s     | Cross-domain orchestration               |

### Data / storage surfaces

| Service                       | Availability | Notes                                                           |
| ----------------------------- | ------------ | --------------------------------------------------------------- |
| Postgres primary (per domain) | 99.95%       | RDS Multi-AZ; managed by AWS, but app-level health check counts |
| Redis (per domain)            | 99.9%        | ElastiCache with replica fail-over                              |
| MinIO / S3 (uploads)          | 99.99%       | Vendor SLA; app-level GET-success counts                        |
| Kafka (event bus)             | 99.9%        | When enabled per profile                                        |

## How we measure

- **Availability** — `(successful_requests / total_requests)` where successful =
  HTTP 2xx-4xx (4xx is a client error, not a service failure). Computed from
  Prometheus
  `rate(http_requests_total{status=~"5.."}[5m]) / rate(http_requests_total[5m])`.
- **Latency** — Prometheus
  `histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket[5m])) by (le))`
  for p95; `0.99` for p99.
- **Job duration** — Worker tasks emit a custom histogram
  `oshun_job_duration_seconds` with the job-type label.

## Excluded events

We exclude from SLO accounting:

- Synthetic monitoring traffic.
- Health-check pings (`/health`, `/healthz`).
- Internal admin endpoints (`/v1/admin/*`).
- Scheduled maintenance windows (set via the maintenance toggle in PagerDuty).

## Reviewing targets

The SLO doc is reviewed quarterly. Each domain owner brings:

- Prior-quarter actuals (uptime, p95/p99) from Grafana.
- Burn-rate alert frequency.
- Whether the target was hit, missed by <10%, or missed by >10%.
- Proposed target adjustments (with rationale).

Permanent target changes require sign-off from the domain lead and the platform
team. Tightening targets is encouraged when the service has been comfortably
within them for 2+ quarters; loosening targets requires evidence of
architectural limits and a plan to re-tighten.