With OPENROUTER_API_KEY in the BFF process, Eve defaults to:
| Leg | Model | Default voice / format |
|---|---|---|
| Speech input | microsoft/mai-transcribe-2 |
Recorded audio → transcript and available word timings |
| Spoken replies | qwen/qwen-audio-3.0-tts-plus |
longanlingxin, MP3 |
The microphone captures up to one minute, puts recognized speech in the composer for review, and sends the reviewed text through the existing assistant turn pipeline. Enabling Voice output reads replies aloud alongside their chat messages. Selecting these provider defaults does not enable playback for a member. Server voice remains subject to the existing billing-plan and session-ownership checks. No client receives the OpenRouter credential. The web client forwards the browser's preferred locale instead of forcing US English.
While capture is active, the panel states that the recording remains on the device until Stop. During server transcription it states that the microphone is off and the recording is being sent to Eve's speech provider; the browser fallback names browser speech recognition instead. Recognition never submits a turn: the words stay editable in the text composer until the member presses Send. A reported confidence below 0.75 adds a review warning.
Starting the microphone while speech is playing is a barge-in control: active
server audio or browser synthesis stops before recording. Closing the panel
cancels the recorder or browser recognizer. Transcript responses carry
Cache-Control: no-store; request audio and provider response bodies are not
logged, and normalization temporary files are removed on every exit.
Configuration and rollback#
Both model pins live in model-registry.ts; voice-config.ts resolves them.
Set configuration before starting the BFF, then restart it to reload bindings.
OSHUN_ASSISTANT_STT_PROVIDER:openrouter,deepgram, orwhisper.OSHUN_ASSISTANT_STT_MODEL: overrides the OpenRouter or Deepgram model. Whisper retainswhisper-1. An OpenRouter override must support verbose JSON.OSHUN_ASSISTANT_STT_ENDPOINT: OpenRouter/Whisper API root, or full Deepgram listen URL. Unset uses the selected vendor's endpoint.OSHUN_ASSISTANT_TTS_PROVIDER:openrouter,elevenlabs, orcartesia.OSHUN_ASSISTANT_TTS_MODEL: overrides the OpenRouter model.OSHUN_ASSISTANT_TTS_VOICE_ID: provider-specific voice identifier. The Qwen Plus catalog also offerslonganlufeng. Change the voice when changing models. Set the provider explicitly when retaining a legacy provider's voice id.OSHUN_ASSISTANT_TTS_ENDPOINT: OpenRouter API root; independent of STT.
Explicit providers take precedence. Unknown providers or explicit providers with missing keys fail closed. Without an OpenRouter key, existing automatic selection remains: Deepgram then Whisper for STT; ElevenLabs then Cartesia for TTS. Cartesia still requires a voice id. Runtime failures do not resend content to a different vendor. Unconfigured routes return the existing 503 browser-fallback signal; provider failures return safe 502 errors to the existing recovery UI.
MAI does not guarantee a confidence score. Missing confidence, duration, or word timings remain absent. TTS requests MP3 explicitly because OpenRouter's default raw PCM cannot be played directly by the chat audio element. Both clients bound provider requests to 60 seconds and avoid logging vendor response bodies or credentials.
WebM and MP4 recordings require ffmpeg on the BFF host (or
OSHUN_FFMPEG_PATH). The Node service image installs it for @oshun/bff. MAI
rejected both raw containers in live verification, so the BFF converts them to
16 kHz mono PCM WAV using a 15-second timeout and one decoder thread. Private
temporary recordings are deleted on every exit; recordings above the bounded
decoded-duration limit are rejected, never silently truncated. Ogg, MP3 and WAV
pass through unchanged. Missing ffmpeg produces a safe provider-error response.
Verification#
voice-config.spec.ts, openrouter-stt.spec.ts, openrouter-tts.spec.ts, and
assistant-voice-route.spec.ts cover selection, request/response contracts,
normalization, provider failures, authentication, ownership, and plan limits.
normalize-voice-recording.spec.ts requires ffmpeg and exercises real WebM/MP4
decoding, temporary-file cleanup, and input/duration limits.
apps/oshun/web/e2e/assistant-voice.spec.ts covers microphone capture,
transcript review, text plus speech replies, and disabled playback. Its speech
endpoint responses are doubled; its browser capture APIs are real.
openrouter-voice.live.spec.ts is an opt-in billed test: set EVE_VOICE_LIVE=1
and provide OPENROUTER_API_KEY only to the test process. Set
EVE_VOICE_EVIDENCE_DIR to an external directory to retain the generated MP3s,
transcripts, timings, and word-error measurements. Run it with one Vitest worker
using the BFF's Vitest config.
On 2026-09-06, both live English round trips passed. Qwen generated MP3 audio
(24 kHz, mono, independently decoded by ffprobe); MAI returned the intended
sentences with numeric formatting (nine → 9, ten minute → 10-minute) and
word timings. This is a functional smoke test, not a comparative accuracy,
accent, or subjective voice-quality benchmark. Live format checks also passed
for WebM and MP4 after conversion, and Ogg without conversion.
Local verification on that date passed: 132 backend tests, 27 web tests, both live round trips, and all three Chromium voice flows with the shared axe accessibility fixture. The BFF production bundle, focused BFF/web typechecks, scoped repository lint, and formatting checks also passed. Browser playback is instrumented; MP3 decoding is checked separately by ffprobe.
Task 8.7 depth decision (2026-09-12)#
ADR-0084 retains the shipped asynchronous, review-before-send contract and declines live duplex. The admitted live measurement exercised the pinned MAI binding 15 times: three each for synthetic US, British RP, Caribbean, US with pink background noise, and silence. Worst WER was 0.200 for clean US and 0.222 for the noisy cell; British and Caribbean fixtures were 0.000. All three silence transcripts remained empty. Nearest-rank p95 was 776.294 ms against the predeclared 5,000 ms budget.
These fixtures are reproducible functional quality evidence, not demographic
accent-fairness evidence or a substitute for consented-human-speech research.
Server TTS retains a plain exact-text caption because the provider supplies no
audio timing contract; the UI does not fabricate word synchronization. The
decision record and receipts live under
docs/audits/eve-sota-voice-depth-decision/.
API references: OpenRouter STT, OpenRouter TTS.