Phase: 180.C.20.01 Status: Decision — direct CDP is the verified default; Browser Use is an opt-in, policy-gated, fail-loud provider behind the same interface. Scope: Which backend the Bellona remote-host browser surface routes a task plan to, and the reliability / privacy / latency / cost tradeoffs of each.
This is a real comparison run against the same task fixtures the deterministic
browser surface already ships — the five 180.C.19.09 task templates in
libs/bellona/remote-protocol/src/browser-task-templates.ts:
| Template id | Fixture role | Mutation posture |
|---|---|---|
task-template:production-board-update |
open board → focus card → set status → submit | safe-mutation + a form submission |
task-template:asset-marketplace-search |
navigate → type query (pressEnter) → wait → snapshot | safe-mutation (submit-capable type) + read |
task-template:documentation-lookup |
navigate → search → wait → extract | safe-mutation + read |
task-template:account-free-research |
navigate → wait → snapshot | read-only throughout |
task-template:authenticated-admin-workflow |
grant profile → navigate → gated admin action | authenticated profile + operator-approval-gated mutation |
Each template expands (via planRemoteHostBrowserTaskTemplate) into an ordered
sequence of typed, schema-validated browser command argument-sets. That expansion
is the provider-agnostic plan all three backends consume unchanged — the
planner does not change to swap backends (see 180.C.20.02,
libs/bellona/remote-protocol/src/browser-agent-provider.ts).
Honesty note (measurement). Direct-CDP numbers below are anchored to the deterministic adapter that is built and exercised on-box by the host test suite (
apps/bellona/remote-host/src/browser-adapter.test.ts,browser-agent-direct-cdp.test.ts). The Browser Use cloud figures are projected / uninstrumented — no Browser Use cloud account or API key is wired in this environment, so its provider is an injectable seam that fails loudprovider_not_configureduntil creds are supplied (by design; we do not fabricate a run). Browser Use local figures are likewise projected from the publishedbrowser-usearchitecture (an LLM agent loop driving Playwright over CDP), not measured here. Where a row is projected it is marked (projected).
1. The three backends#
1.1 Direct CDP (first-party, direct_cdp)#
The existing CDP/Playwright host browser adapter
(apps/bellona/remote-host/src/browser-adapter.ts). Each plan step is a typed
Oshun command (browser.navigate, browser.click_element, browser.submit, …)
dispatched to a deterministic handler that drives Playwright over a
loopback-only CDP endpoint against an isolated managed profile. There is
no LLM in the loop: the plan fully determines the actions. The
createRemoteHostBrowserDirectCdpProvider wrapper
(apps/bellona/remote-host/src/browser-agent-direct-cdp.ts) exposes this adapter
through the common provider interface, so it is selectable as the deterministic
default with zero third-party dependencies.
1.2 Browser Use — local (browser_use_local)#
The open-source browser-use Python package run on the same box: an LLM agent
loop that observes the page (accessibility tree + screenshots) and decides the
next Playwright action. The model call is local-network/remote depending on which
LLM is configured, but the browser execution stays on-box. In Oshun this is a
RemoteBrowserUseRuntime wired into createRemoteBrowserUseProvider({ mode: 'browser_use_local', runtime }). Without a wired runtime the provider fails loud
provider_not_configured.
1.3 Browser Use — cloud (browser_use_cloud)#
The hosted Browser Use Cloud API: the task plan / objective is sent off-box, a
remote browser executes it, and step events + screenshots stream back. In Oshun
this is the same provider with mode: 'browser_use_cloud' over a runtime that
calls the cloud API. Execution leaves the box (offBoxExecution: true),
which is the dominant privacy consideration.
A fourth mode,
mcp, exists in the interface for "another MCP browser provider" but is out of scope for this measurement; it is gated identically to the Browser Use modes (any non-direct_cdpmode is third-party).
2. Reliability#
| Dimension | direct_cdp | browser_use_local (projected) | browser_use_cloud (projected) |
|---|---|---|---|
| Determinism | Deterministic — the plan fully specifies the steps; same plan → same actions | Non-deterministic — an LLM picks the next action each step | Non-deterministic (same loop, remote) |
| Repeatability of the 5 fixtures | Repeatable; asserted in browser-adapter.test.ts (62 tests) + browser-agent-direct-cdp.test.ts |
Varies run-to-run with model + page changes | Varies run-to-run |
| Failure mode | Fails loud with structured evidence (DOM-failure evidence, trace.zip) | Agent may "succeed" on the wrong element or loop; needs a verifier | Same, plus network/availability failures |
| Selector robustness to page drift | Brittle to large redesigns (selector-bound), but predictable | More robust to cosmetic drift (semantic targeting) | Same as local |
| Recovery | Compensating actions per mutating command (180.C.19.*) | Agent self-correction (unbounded without guardrails) | Same |
| Auditability of what ran | Every step is a typed command in the audit log | Steps normalized to Oshun events (180.C.20.03), but the decision was the model's | Same |
Reading. Direct CDP wins decisively on determinism, repeatability, and auditability — exactly the properties an MVP and a regression suite need. Browser Use's value is robustness to page drift and handling under-specified objectives, at the cost of determinism. For the five fixtures, which are fully specified, the non-determinism is a net negative; Browser Use earns its keep on tasks the templates can't pre-express.
3. Privacy#
| Dimension | direct_cdp | browser_use_local | browser_use_cloud |
|---|---|---|---|
| Page content leaves the box | No | Only to the configured LLM (prompt context) | Yes — page DOM/screenshots sent to the cloud |
| Authenticated profile exposure | First-party; isolated managed profile by default | Third-party agent drives the profile in-process | Third-party + off-box |
| Credential entry | Blocked on personal profiles unless explicitly approved; values redacted from evidence (180.C.19.06) | Must be gated — agent could be baited into typing secrets | Must be gated; higher blast radius |
| Screenshot/trace redaction | Host applies redaction masks before persistence | Normalized step screenshots marked redaction: pending until host redacts |
Same |
| Step-note leakage | n/a (no free-text agent notes) | Agent "thoughts" can contain secrets → redacted by redactRemoteBrowserAgentText before persistence (180.C.20.05) |
Same |
Reading. Direct CDP is the only backend where neither page content nor
credentials leave the box. Browser Use local sends page context to whatever LLM
is configured; Browser Use cloud sends it to a third party. This is why the
180.C.20.04 policy gate refuses to hand a non-direct_cdp provider an
authenticated/granted profile, a sensitive origin, a file upload, or a form
submission without an explicit, plan-bound operator approval, and why
direct_cdp is the only mode exempt from that gate.
4. Latency#
| Step type | direct_cdp (on-box, measured class) | browser_use_local (projected) | browser_use_cloud (projected) |
|---|---|---|---|
| navigate (load) | network-bound page load; no model call | page load + 1 LLM decision (~1–5 s) | page load + LLM + round-trip to cloud |
| click / type / submit | low-ms Playwright action + the page's own latency | + 1 LLM decision per step | + LLM + cloud round-trip per step |
| snapshot / extract | direct DOM/text read | model may re-observe before extracting | + cloud round-trip |
Whole production-board-update (4 steps) |
~ sum of 4 deterministic actions + page latencies | 4 actions + 4 model decisions | + per-step cloud round-trips |
Direct CDP adds no model latency: a step costs a Playwright action plus the
page's own latency. The host enforces explicit per-class ceilings — read-only
standard (≤120 s), safe-mutation short (≤30 s), file-transfer (≤120 s) — in
browser.ts. Browser Use adds one LLM decision per step (local) plus a
network round-trip per step (cloud). For a 4-step plan that is 4× the model
latency stacked on top of the same page latencies. For under-specified objectives
where Browser Use replaces many human iterations, that per-step cost can still be
a net win; for the pre-expanded templates it is pure overhead.
5. Cost#
| Dimension | direct_cdp | browser_use_local | browser_use_cloud |
|---|---|---|---|
| Per-step marginal $ | ~0 (CPU/Playwright only) | LLM tokens per step (prompt = page context + history) | LLM tokens + hosted-browser minutes |
| Fixed $ | none beyond the host | a local model endpoint (self-host or API) | a Browser Use Cloud subscription / API key |
| Scaling shape | linear in CPU; cheap to fan out | linear in tokens; page-context prompts are large | linear in tokens + browser-minutes |
| Cost predictability | High (deterministic step count) | Lower (step count varies with the agent loop) | Lower |
Direct CDP is effectively free per step. Browser Use's cost scales with tokens (the page context is a large prompt) and, for cloud, with hosted-browser minutes. A deterministic 4-step template costs ~0 on direct CDP and 4 model calls on Browser Use.
6. Decision#
-
Direct CDP is the verified default. It is deterministic, on-box, auditable, free per step, and already built + tested on this box. The deterministic MVP runs entirely through
direct_cdpwith no third-party provider registered —selectRemoteBrowserAgentProviderpicks it as the default and refuses to silently downgrade a deterministic task to a third-party agent (no_silent_downgrade). -
Browser Use is an opt-in provider behind the same interface, selected only by explicit provider id/mode, for tasks the templates cannot pre-express (under-specified objectives, heavy page drift). It is wired as an injectable seam that fails loud
provider_not_configureduntil a runtime/credentials are supplied — we never fabricate a Browser Use run. -
Third-party providers are policy-gated. A non-
direct_cdpprovider is refused an authenticated/granted profile, a sensitive origin, a file upload, or a form submission without an explicit, plan-bound operator approval (180.C.20.04). The approval must bind to the exact task run (approvedTaskId, single-use), the exact plan hash, the exact provider (id+mode), be unexpired (expiresAtchecked againstnow), and enumerate every detected reason + every detected sensitive origin — so it cannot be replayed against a later run, reused after expiry, retargeted to another provider, or silently widened. Exposure is classified from each step's command name + re-parsed arguments (never the planner's risk labels), the active origin is tracked across steps so a click/type/submit on an already-sensitive page (or alaunch_managedthat opens directly on a sensitive origin) is caught, and an unknown command or unparseable arguments is denied as maximally dangerous. -
Whatever runs, runs through Oshun's schemas. Every Browser Use / external step event is normalized (180.C.20.03) into Oshun command progress, a timeline event, an optional screenshot/trace artifact, and an audit entry — all validated against the existing protocol schemas. An event that cannot be normalized is quarantined fail-loud, never dropped into a fabricated success. Credential-ish substrings in agent step notes are redacted before persistence.
When to reach for each#
- Direct CDP — the five templates and any task that can be expressed as a deterministic plan; all regression/CI runs; anything touching credentials, sensitive origins, uploads, or submissions on an authenticated profile.
- Browser Use local — exploratory or under-specified objectives where page context staying on-box matters and a local model is available.
- Browser Use cloud — only when neither of the above is viable, with explicit operator approval, accepting that page content leaves the box.
7. Evidence pointers#
- Provider interface, selection, normalization, policy gate, Browser Use seams:
libs/bellona/remote-protocol/src/browser-agent-provider.ts - Adversarial tests (cannot bypass schema/policy/approval/redaction/audit):
libs/bellona/remote-protocol/src/browser-agent-provider.test.ts - First-party direct-CDP provider over the real adapter handlers:
apps/bellona/remote-host/src/browser-agent-direct-cdp.ts(+.test.ts) - The fixtures (task templates) this evaluation runs against:
libs/bellona/remote-protocol/src/browser-task-templates.ts - Prompt-injection corpus the gates are also proven against:
libs/bellona/remote-protocol/src/browser-prompt-injection.ts