Disciplines · Audits

Eve Task 8.4 — Mobile Assistant Transport Contract

The compared arms run the same authenticated BFF session and governed agent pipeline:

5sections5 minread
Markdown source
On this page

Status: implementation and measurement harness present; closure remains open until retained Android measurements and both mobile-runtime automation cells pass. This document is not a runtime receipt.

Decision boundary#

The compared arms run the same authenticated BFF session and governed agent pipeline:

  • streaming: Expo SDK 57 WinterCG fetch, Accept: text/event-stream, shared turn.* parser, provisional deltas, authoritative turn.complete.
  • buffered: legacy Accept: application/json, identical BFF turn pipeline, one terminal JSON response.

The current normal build remains visibly buffered while closure evidence is absent. EXPO_PUBLIC_OSHUN_MOBILE_ASSISTANT_TRANSPORT=streaming selects the candidate measurement arm; buffered selects the control. A closure commit must change the default to the admitted winner (or explicitly retain buffered if the result is inconclusive). A transient failure never flips that flag or silently changes the experiment.

Expo SDK 57 documents that expo/fetch is installed as the native global fetch and that response.body.getReader() supports streaming on Android and iOS: https://docs.expo.dev/versions/v57.0.0/sdk/expo/. This supersedes the older local assumption that React Native necessarily buffers every response.

Preregistered admission thresholds#

Run at least 20 completed sequential turns per arm on the same lower-tier Android device/profile, alternating arm order between repeated trials. Keep the prompt set, BFF/provider binding, account tier, thermal starting posture, and network path fixed. The comparator admits streaming only when every check passes:

Boundary Admission threshold
TTFT streaming p95 is at most 70% of buffered p95
total latency streaming p95 is at most 115% of buffered p95
native memory streaming p95 PSS is no more than buffered p95 + max(12 MiB, 15%)
energy streaming estimated mAh is no more than buffered + 10%, with a 0.001 mAh floor
reader wakeup proxy no streaming turn exceeds 64 native response chunks
cancel at least one real mobile cancellation settles in at most 250 ms p95
unreliable network at least one Android airplane-mode interruption and recovery receipt
lifecycle at least one background/foreground interruption and recovery receipt
duplicates zero automatic replays in both arms
transcript every completed request commits exactly one assistant turn; interrupted user turns remain visible
automation the Task 8.4 Maestro lifecycle flow passes

Missing power data, fewer than 20 completed samples, a different device profile, or any failed hard invariant produces inconclusive; it does not preserve the default by fiat. The executable authority is apps/oshun/mobile/scripts/compare-assistant-transport-benchmarks.mjs.

Winning runtime contract#

The shipping candidate follows these rules:

  1. One composer submission owns one AbortController and one stable local turn key. While it is active, composer and quick-prompt resubmission are disabled and controller-side duplicate submissions are rejected as defense in depth.
  2. turn.delta text is provisional. It updates the existing pending bubble; it never appends a transcript record.
  3. Only a valid turn.complete appends the single authoritative assistant bubble. Its text replaces the provisional text in place.
  4. A non-2xx response received before the SSE sink opens is safe to route once through the deterministic BFF message endpoint. An aborted, incomplete, malformed, timed-out, or network-broken stream is ambiguous and is never replayed automatically. The buffered comparison arm falls back only for an allowlisted BFF preflight reason retained in the HTTP error (agent not configured or budget exhausted); an unclassified/terminal 5xx is never replayed because its status alone cannot identify the sink boundary.
  5. Stop, background, sheet close, and an observed offline transition abort the in-flight request. The member's user turn stays in the transcript. Foreground or network recovery presents a reconciled state and waits for an explicit new submission.
  6. Transport state lives in the existing transcript-state band. It exposes connecting, streaming, buffered fallback, confirmed, interrupted, offline, cancelled, and reconnected states without creating a transport dashboard.
  7. Per-turn metric lines are emitted only by an E2E-enabled build. They contain timings and counts, never prompt or reply content.

The no-replay rule is deliberately stricter than the previous mobile behavior, which sent every failed agent turn to /message. Once an SSE request has started, a disconnect can occur after tools or durable persistence; without a server-owned idempotency/status protocol, a second POST cannot prove it is not a duplicate.

Automated execution#

Start the BFF with a real or scripted streaming agent, then start the dev client with E2E hooks and the selected arm, for example:

bash
EXPO_PUBLIC_OSHUN_BFF_URL=http://127.0.0.1:4010 \
EXPO_PUBLIC_E2E_DISABLE_AUTOFILL=true \
EXPO_PUBLIC_OSHUN_MOBILE_ASSISTANT_TRANSPORT=streaming \
  pnpm --dir apps/oshun/mobile exec expo start --dev-client --port 8081

Run the cross-platform lifecycle contract:

bash
pnpm --dir apps/oshun/mobile run e2e:matrix:assistant-transport

The flow uses Maestro's native Home/foreground commands on Android and iOS. Its airplane-mode cell is Android-only, matching Maestro's platform contract: https://docs.maestro.dev/reference/commands-available/setairplanemode.

For retained Android performance receipts, compile/start Metro once per arm and run:

bash
apps/oshun/mobile/scripts/run-assistant-transport-android-benchmark.sh \
  streaming docs/audits/eve-sota-evidence/phase-08/task-8-4-streaming-android.json

apps/oshun/mobile/scripts/run-assistant-transport-android-benchmark.sh \
  buffered docs/audits/eve-sota-evidence/phase-08/task-8-4-buffered-android.json

node apps/oshun/mobile/scripts/compare-assistant-transport-benchmarks.mjs \
  docs/audits/eve-sota-evidence/phase-08/task-8-4-streaming-android.json \
  docs/audits/eve-sota-evidence/phase-08/task-8-4-buffered-android.json \
  docs/audits/eve-sota-evidence/phase-08/task-8-4-decision.json

The Android runner refuses to overwrite a receipt, samples package PSS while Maestro drives the benchmark turns, snapshots batterystats before the streaming-only lifecycle fault cells, retains only content-free transport sample lines (not device-wide logcat) plus hashed Maestro/resource data, verifies the compiled arm from runtime samples, and disables airplane mode during cleanup. Lifecycle outcomes stay in the receipt but are excluded from the latency, memory, and energy comparison population. The comparison command exits nonzero on incomplete data or a failed threshold.

Required closure receipts#

Task 8.4 remains unchecked until all of the following are retained and verified:

  • streaming and buffered Android raw/derived receipts with at least 20 completed samples each and usable package-energy data;
  • an admitted comparison decision;
  • passing Android unreliable-network, cancel, duplicate, lifecycle, and transcript-continuity automation;
  • passing iOS current-simulator cancel, lifecycle, and transcript-continuity automation (airplane mode is not applicable on iOS);
  • focused unit, mobile component, BFF integration, lint, and typecheck gates;
  • the task evidence manifest at docs/audits/eve-sota-evidence/phase-08/task-8-4.json covering static contract, automated behavior, service integration, mobile runtime, fault recovery, and performance quality.

The current host has no attached Android device/AVD, no installed emulator or system image, no /dev/kvm, and no iOS simulator. Therefore no mobile-runtime, battery, or memory result is asserted in this implementation-only change, and the ledger checkbox must remain open.