Eve library

ADR 0002 — Server-Sent Events for coach chat streaming

ADR

Revision 1

On this page

Status: Accepted (2026-05-06)

Context#

The original POST /v1/coach/chat endpoint returns the full coach response in a single JSON payload. The mobile and web chat UIs masked the abrupt arrival with a client-side typewriter that revealed the words one at a time over a setInterval. That's a reasonable UX choice but it's dishonest: the function was named simulateStreaming, the comment said "streaming", and yet there was no server stream — just a delay.

Three options:

  1. Leave the typewriter, rename it to be honest. (We initially shipped this — see prior commit.)
  2. Implement Server-Sent Events end-to-end so the rhythm is genuinely driven by the server.
  3. Implement WebSockets.

Decision#

Add POST /v1/coach/chat/stream. The endpoint:

  1. Parses + persists the user message and the full coach response up front, before opening the stream. A disconnected client can recover the canonical record via /v1/coach/conversations + the message list.
  2. Switches reply.raw into raw mode, writes Content-Type: text/event-stream, and emits these event types:
    • meta — conversationId + messageId.
    • token — one event per word, trailing whitespace preserved.
    • final — full message + suggestions + followUp.
    • done — sentinel.
  3. Honors a configurable per-token delay (COACH_STREAM_INTERVAL_MS env var, default 50 ms). 0 disables the delay (used in tests).
  4. Aborts cleanly on reply.raw.on('close') — no extra writes after the client disconnects.

The mobile + web clients consume the stream with fetch + ReadableStreamDefaultReader rather than EventSource so we get AbortSignal support and avoid the EventSource Authorization-header limitation.

Consequences#

  • The "AI is thinking" rhythm is genuinely driven by the server, not faked client-side.
  • Tests run instantly because streamIntervalMs: 0 is wired through the route plugin options.
  • Per-message persistence happens before the stream begins, so we can always reconstruct a session even if the network drops mid-stream.
  • One consequence to monitor: the user message is persisted before the coach response is generated — if the response generator throws, the conversation will contain a user message with no reply. The route handler currently propagates the error and the message remains in the log; future work could mark the message with pending=true and clean up on next interaction.

Why not WebSockets#

WebSockets would let the client push back (e.g. "stop generating") but the response generator is synchronous (no LLM behind it) — there's nothing to interrupt. SSE is also strictly simpler from an infrastructure standpoint: it survives plain HTTP/2 reverse proxies and needs no upgrade handshake.

Reference#

  • Endpoint: apps/arete/api/src/routes/coach.ts — search for /chat/stream.
  • Mobile consumer: apps/arete/mobile/src/hooks/useCoach.ts — streamChatMessage.
  • Web consumer: apps/arete/web/src/api/hooks/useCoach.ts — streamChatMessage.
  • Tests: 7 SSE-specific cases in __tests__/coach.test.ts.