Status: Accepted (2026-05-06)
Context#
The original POST /v1/coach/chat endpoint returns the full coach response in a
single JSON payload. The mobile and web chat UIs masked the abrupt arrival with
a client-side typewriter that revealed the words one at a time over a
setInterval. That's a reasonable UX choice but it's dishonest: the function
was named simulateStreaming, the comment said "streaming", and yet there was
no server stream — just a delay.
Three options:
- Leave the typewriter, rename it to be honest. (We initially shipped this — see prior commit.)
- Implement Server-Sent Events end-to-end so the rhythm is genuinely driven by the server.
- Implement WebSockets.
Decision#
Add POST /v1/coach/chat/stream. The endpoint:
- Parses + persists the user message and the full coach response up front,
before opening the stream. A disconnected client can recover the canonical
record via
/v1/coach/conversations+ the message list. - Switches
reply.rawinto raw mode, writesContent-Type: text/event-stream, and emits these event types:meta— conversationId + messageId.token— one event per word, trailing whitespace preserved.final— full message + suggestions + followUp.done— sentinel.
- Honors a configurable per-token delay (
COACH_STREAM_INTERVAL_MSenv var, default 50 ms). 0 disables the delay (used in tests). - Aborts cleanly on
reply.raw.on('close')— no extra writes after the client disconnects.
The mobile + web clients consume the stream with fetch +
ReadableStreamDefaultReader rather than EventSource so we get AbortSignal
support and avoid the EventSource Authorization-header limitation.
Consequences#
- The "AI is thinking" rhythm is genuinely driven by the server, not faked client-side.
- Tests run instantly because
streamIntervalMs: 0is wired through the route plugin options. - Per-message persistence happens before the stream begins, so we can always reconstruct a session even if the network drops mid-stream.
- One consequence to monitor: the user message is persisted before the coach
response is generated — if the response generator throws, the conversation
will contain a user message with no reply. The route handler currently
propagates the error and the message remains in the log; future work could
mark the message with
pending=trueand clean up on next interaction.
Why not WebSockets#
WebSockets would let the client push back (e.g. "stop generating") but the response generator is synchronous (no LLM behind it) — there's nothing to interrupt. SSE is also strictly simpler from an infrastructure standpoint: it survives plain HTTP/2 reverse proxies and needs no upgrade handshake.
Reference#
- Endpoint:
apps/arete/api/src/routes/coach.ts— search for/chat/stream. - Mobile consumer:
apps/arete/mobile/src/hooks/useCoach.ts—streamChatMessage. - Web consumer:
apps/arete/web/src/api/hooks/useCoach.ts—streamChatMessage. - Tests: 7 SSE-specific cases in
__tests__/coach.test.ts.