# Eve voice providers

With `OPENROUTER_API_KEY` in the BFF process, Eve defaults to:

| Leg            | Model                          | Default voice / format                                 |
| -------------- | ------------------------------ | ------------------------------------------------------ |
| Speech input   | `microsoft/mai-transcribe-2`   | Recorded audio → transcript and available word timings |
| Spoken replies | `qwen/qwen-audio-3.0-tts-plus` | `longanlingxin`, MP3                                   |

The microphone captures up to one minute, puts recognized speech in the composer
for review, and sends the reviewed text through the existing assistant turn
pipeline. Enabling **Voice output** reads replies aloud alongside their chat
messages. Selecting these provider defaults does not enable playback for a
member. Server voice remains subject to the existing billing-plan and
session-ownership checks. No client receives the OpenRouter credential. The web
client forwards the browser's preferred locale instead of forcing US English.

While capture is active, the panel states that the recording remains on the
device until Stop. During server transcription it states that the microphone is
off and the recording is being sent to Eve's speech provider; the browser
fallback names browser speech recognition instead. Recognition never submits a
turn: the words stay editable in the text composer until the member presses
Send. A reported confidence below 0.75 adds a review warning.

Starting the microphone while speech is playing is a barge-in control: active
server audio or browser synthesis stops before recording. Closing the panel
cancels the recorder or browser recognizer. Transcript responses carry
`Cache-Control: no-store`; request audio and provider response bodies are not
logged, and normalization temporary files are removed on every exit.

## Configuration and rollback

Both model pins live in `model-registry.ts`; `voice-config.ts` resolves them.
Set configuration before starting the BFF, then restart it to reload bindings.

- `OSHUN_ASSISTANT_STT_PROVIDER`: `openrouter`, `deepgram`, or `whisper`.
- `OSHUN_ASSISTANT_STT_MODEL`: overrides the OpenRouter or Deepgram model.
  Whisper retains `whisper-1`. An OpenRouter override must support verbose JSON.
- `OSHUN_ASSISTANT_STT_ENDPOINT`: OpenRouter/Whisper API root, or full Deepgram
  listen URL. Unset uses the selected vendor's endpoint.
- `OSHUN_ASSISTANT_TTS_PROVIDER`: `openrouter`, `elevenlabs`, or `cartesia`.
- `OSHUN_ASSISTANT_TTS_MODEL`: overrides the OpenRouter model.
- `OSHUN_ASSISTANT_TTS_VOICE_ID`: provider-specific voice identifier. The Qwen
  Plus catalog also offers `longanlufeng`. Change the voice when changing
  models. Set the provider explicitly when retaining a legacy provider's voice
  id.
- `OSHUN_ASSISTANT_TTS_ENDPOINT`: OpenRouter API root; independent of STT.

Explicit providers take precedence. Unknown providers or explicit providers with
missing keys fail closed. Without an OpenRouter key, existing automatic
selection remains: Deepgram then Whisper for STT; ElevenLabs then Cartesia for
TTS. Cartesia still requires a voice id. Runtime failures do not resend content
to a different vendor. Unconfigured routes return the existing 503
browser-fallback signal; provider failures return safe 502 errors to the
existing recovery UI.

MAI does not guarantee a confidence score. Missing confidence, duration, or word
timings remain absent. TTS requests MP3 explicitly because OpenRouter's default
raw PCM cannot be played directly by the chat audio element. Both clients bound
provider requests to 60 seconds and avoid logging vendor response bodies or
credentials.

WebM and MP4 recordings require `ffmpeg` on the BFF host (or
`OSHUN_FFMPEG_PATH`). The Node service image installs it for `@oshun/bff`. MAI
rejected both raw containers in live verification, so the BFF converts them to
16 kHz mono PCM WAV using a 15-second timeout and one decoder thread. Private
temporary recordings are deleted on every exit; recordings above the bounded
decoded-duration limit are rejected, never silently truncated. Ogg, MP3 and WAV
pass through unchanged. Missing ffmpeg produces a safe provider-error response.

## Verification

`voice-config.spec.ts`, `openrouter-stt.spec.ts`, `openrouter-tts.spec.ts`, and
`assistant-voice-route.spec.ts` cover selection, request/response contracts,
normalization, provider failures, authentication, ownership, and plan limits.
`normalize-voice-recording.spec.ts` requires ffmpeg and exercises real WebM/MP4
decoding, temporary-file cleanup, and input/duration limits.
`apps/oshun/web/e2e/assistant-voice.spec.ts` covers microphone capture,
transcript review, text plus speech replies, and disabled playback. Its speech
endpoint responses are doubled; its browser capture APIs are real.

`openrouter-voice.live.spec.ts` is an opt-in billed test: set `EVE_VOICE_LIVE=1`
and provide `OPENROUTER_API_KEY` only to the test process. Set
`EVE_VOICE_EVIDENCE_DIR` to an external directory to retain the generated MP3s,
transcripts, timings, and word-error measurements. Run it with one Vitest worker
using the BFF's Vitest config.

On 2026-09-06, both live English round trips passed. Qwen generated MP3 audio
(24 kHz, mono, independently decoded by ffprobe); MAI returned the intended
sentences with numeric formatting (`nine` → `9`, `ten minute` → `10-minute`) and
word timings. This is a functional smoke test, not a comparative accuracy,
accent, or subjective voice-quality benchmark. Live format checks also passed
for WebM and MP4 after conversion, and Ogg without conversion.

Local verification on that date passed: 132 backend tests, 27 web tests, both
live round trips, and all three Chromium voice flows with the shared axe
accessibility fixture. The BFF production bundle, focused BFF/web typechecks,
scoped repository lint, and formatting checks also passed. Browser playback is
instrumented; MP3 decoding is checked separately by ffprobe.

## Task 8.7 depth decision (2026-09-12)

ADR-0084 retains the shipped asynchronous, review-before-send contract and
declines live duplex. The admitted live measurement exercised the pinned MAI
binding 15 times: three each for synthetic US, British RP, Caribbean, US with
pink background noise, and silence. Worst WER was 0.200 for clean US and 0.222
for the noisy cell; British and Caribbean fixtures were 0.000. All three silence
transcripts remained empty. Nearest-rank p95 was 776.294 ms against the
predeclared 5,000 ms budget.

These fixtures are reproducible functional quality evidence, not demographic
accent-fairness evidence or a substitute for consented-human-speech research.
Server TTS retains a plain exact-text caption because the provider supplies no
audio timing contract; the UI does not fabricate word synchronization. The
decision record and receipts live under
`docs/audits/eve-sota-voice-depth-decision/`.

API references:
[OpenRouter STT](https://openrouter.ai/docs/guides/overview/multimodal/stt),
[OpenRouter TTS](https://openrouter.ai/docs/guides/overview/multimodal/tts).
