[earakely-scale]

What and why

A new task step, browser_connect_cdp, hands a deployed agent a CDP address to an env's website browser. Browser Use (M8.18) and the CDP benchmark arms need agentenv to do this. M8.1 gave agents cdp-config/v1, M8.6 the browser's cdp operation, and M8.7 the relay's guard, but no step connected them. This is FD-4964 (M8.17). Its four decisions are recorded there.

What the step does (task_steps/browser_connect_cdp.py), placed after deploy_agent (decision 1):

1. Asks the browser for the agent's address with browser/v1 cdp {user}, through the env with its control token, on the card of the browser app it finds among the env's children. A browser computer's driver takes its app's name, so the step never asks for the child by name. browser names the app; unnamed, it's the env's only website browser.
2. Rewrites the address to one the agent's sandbox reaches, with the same reachable_url and host_url_for that deploy_agent uses for MCP URLs. A browser deployed on its own then needs no gateway. An address anywhere else, at a gateway's public address for instance, is kept as it is (M8.11).
3. Adds it to the agent's cdp-config/v1 as chromium, or the name given, with metadata {browser}. M8.15 and M8.16 add context (decision 4).
4. Hides every tool the env serves from the agent's role, unless keep_tools. It uses tool access's wildcard (tools: "*") on the env's disable-tool/v1. On a browser computer every tool drives the browser, so a CDP agent has one way in (decision 3).
   • The role is the agent's, default when its deploy_agent step has none (decision 2). So every other caller of that env without a role loses those tools too, including env_outcome_verifier, whose verify(mcp_url) calls the env with no role. A task with both should give the agent a role or set keep_tools. That cost was accepted on review for now; the step could later hide them only when the env is in the agent's env_ids.
   • The step records the role it hid them for. Role is the key only until M6.1's per-agent identity.
5. Records the connection in context.metadata["browser_connections"]: its env, agent, browser, name, role and what was hidden. The address isn't recorded. A connection under a name the agent already has replaces that record.

It refuses:

• an agent whose card has no cdp-config/v1. The card is only known once the agent is deployed, so this is at execution.
• an env with no website browser, or several and none named, or none by the name given.

A reset ends the address, and the task connects the agent again. reset_env is unchanged and knows nothing of CDP.

• What a reset does: a reset is a fresh container, which ends the agent's address and its hidden-tool rules.
• What the task does: it runs browser_connect_cdp again after reset_env, or gives the turn a retry_config whose retry_from_step_id reruns it.
• What happens otherwise: a turn on the old address fails.
• Why: this was decided on review at c361ca3, after an earlier version had reset_env reconnect recorded connections. Keeping reset_env free of CDP was preferred to it, and putting this on the task author is for now.

Two departures from FD-4964's text:

• user is the agent's agent_name, not its deploy_agent step id. Every step names a deployed agent by agent_name, and the step id isn't in the context.
• The step is registered as a built-in, and the episode rules count it among the steps that need a real agent (_needs_the_agent), since it calls the agent's own extension.

The echo agent (tst/data/a2a_agent) is the fixture:

• It now lists cdp-config/v1.
• A cdp <name> <url> line in its prompt has it open the URL in a new tab over raw CDP, through request.cdp_endpoint(name). It then adds cdp <name> <digest>: <title>, with <digest> the first 12 hex digits of its address's SHA-256, so a test can tell one address from another without the token.
• Its image gains websockets.

Docs: agents/AGENTS.md now says the step sends the endpoint, and that after a reset a task runs it again or reruns it with a retry_config.

Not in this PR:

• contexts and leases (M8.15, M8.16);
• the gateway route (M8.11);
• Browser Use itself (M8.18);
• chromium inside a desktop computer (M8.26).

Size: 453 lines added, 266 of them tests and the fixture agent.

How it was tested

• ruff check: clean.

• Unit tests: 11,938 passed. The new test_browser_connect_cdp.py mocks only the HTTP boundary:

  • The requests: the browser's cdp call with the control token; the agent's add with the address rewritten from localhost to host.docker.internal, as an agent on the local provider reaches it; and the disable call with {"role": "default", "tools": "*"}.
  • The task's choices: a role, keep_tools, a name, and a gateway address left as it is.
  • Each refusal: before any request.
  • Round trip: to_dict and from_dict.

  The episode spec test now covers the new type. Four tests fail on this Mac and on main alike: the two disk-recipe launcher tests, and the two optional-extras tests, because this venv's protocol metadata predates websockets in the computer extra. The new step imports with websockets refused.

• The fixture's CDP open, locally: run against Chrome 156 through the relay and guard, it opened the site twice and reported its title.

• Integration, on agent-environments-admin VMs (amd64 c6i.2xlarge, arm64 c7g.2xlarge), a new browser computer test. It deploys the echo agent on the local provider beside a browser computer and runs browser_connect_cdp on them:

  • the agent's turn opens the site over CDP;

  • the computer's MCP lists no tools to a caller with no role;

  • after reset_env, with the site served again, a turn on the old address fails. Once the task runs browser_connect_cdp again, the next turn opens the site through a fresh address, and the tools are hidden again;

  • the connection's record is checked.

  • At 9dea75f:

    • On amd64 the new test failed with child env 'chromium' does not offer 'cdp': the step asked for the child by name and got the driver's own card. Fixed in 5df21be, and the unit test's computer now takes its app's name, as a real one does.
    • On arm64 the echo agent didn't build (exec format error). tst/util/image_cache.build_or_reuse builds test images for linux/amd64, so that's true on main too, and CI is amd64 only. The arm64 VM got Ubuntu's qemu-user-static so the amd64 agent runs there emulated.

  • At 5df21be: browser_computer_test.py and the echo agent's test_agent_on_local_sandbox.py passed 7 of 7 on each arch.

• Chaos, two more browser computer tests, on both VMs and in CI's slow tier:

  • Restarts and a relaunch, on one browser computer. Opening data:text/html,<title>Home</title> over its address needs no server, so a driver kill can't take the page down with it. The echo agent opens it through the same address:
    • after the app server is killed and comes back;
    • after the driver is killed and comes back;
    • after Chromium relaunches.
  • A failed reconnect, by an injected fault (until c361ca3, which took the reconnect out of reset_env). A reset whose reconnect failed raised, and the agent's next turn failed closed with cdp.no_endpoint.
  • Two agents connected at once open the page twice each, concurrently, each through one address of its own.
  • At 270ab37, on both arches and in CI: every CDP check passed. The one failure was tool access after the driver restart: all 19 computer_* tools were visible to the agent's role again, because the driver keeps those rules only in memory. That's also true of modify_env_tool_access today. It's accepted for now and backlogged as FD-5246 (M8.55, persisting the rules). 6b9bc45 drops that one assertion.
  • At 6b9bc45: 9 of 9 passed on each arch: the whole browser computer suite, with the e2e and both chaos tests, plus test_agent_on_local_sandbox.py.

• Review fixes (Greptile, 3/5 on 9dea75f), at 5df21be, each with a test that fails without it:

  • P1: a reconnect failure after a reset let the next turn drive the ended address. Fixed at 5df21be by removing the connection from the agent. Moot since c361ca3, where reset_env no longer reconnects.
  • P1: a connection replaced under the same name came back on a reset of the env it was connected to before. The step now keeps one record per agent and name, which is still the case after c361ca3.

• Greptile at dc03433 (5/5) found the two-agent test gave both turns of an agent one prompt id. Fixed in 270ab37, and Greptile's review of that commit is 5/5 with no findings.

• CI on 6b9bc45:

  • integration-local-slow (2/12) failed in run_bundle_installed_agent_test.py. Its bundle copies the echo agent's agent.py under a Dockerfile of its own that installs only agentenv-protocol[agent], so the agent's new websockets import stopped it from starting.
  • fba2fe2 installs websockets there too. That test and test_prompt_agent_file_parts_local.py, which loads agent.py in process, passed on the amd64 VM.

• CI on fba2fe2: integration-local-slow (6/12) failed the same way in run_bundle_built_agents_test.py, whose four agent images also install only the agent extra. 919ca3b installs websockets there too. Those are the only two Dockerfiles in the tests that install the agent extra.

  • On the amd64 VM, run_bundle_built_agents_test.py passed 3 of 3.
  • run_bundle_installed_agent_test.py ran and passed its task, but missed its "building with docker" line there, because the VM already had the image from the previous run.

• CI on 919ca3b: every check passed (32), with the installer matrix's placeholder skipped as always. Greptile gives it 5/5 with no findings.

• At c361ca3 (reset_env decoupled): 9 of 9 passed on each arch, the whole browser computer suite with the e2e and chaos tests, plus test_agent_on_local_sandbox.py.

  • CI's slow tier failed once, in the chaos test: the turn right after the relaunch reported an empty title, within a second.
  • Cause: the echo agent stopped at the new tab's about:blank. That document is already complete before the tab commits its URL.
  • Reproducing it: a probe on the amd64 VM didn't reproduce it in 24 relaunch-then-turn rounds, with and without waiting for computer-ready first.
  • Fix: b98566c makes the agent wait for a complete document that isn't about:blank.

• At b98566c: 9 of 9 passed on each arch. In CI every check passed (32), with the installer matrix's placeholder skipped as always, and Greptile gives it 5/5 with no findings.

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge with the documented reset and tool-visibility limits.

Summary

Adds browser_connect_cdp to give an agent a browser address and optionally hide the env’s tools from its role. The latest change waits until the new tab leaves about:blank before reading its title.

• Tasks can give an agent its own address to an environment’s browser.

Diagram

sequenceDiagram
    participant Task
    participant Browser
    participant Agent
    Task->>Browser: Request CDP address for agent
    Browser-->>Task: HTTP and WebSocket addresses
    Task->>Task: Rewrite addresses for agent sandbox
    Task->>Agent: Add named CDP endpoint
    opt Tools not kept
        Task->>Browser: Hide env tools from agent role
    end
    Task->>Task: Replace saved connection record

Loading

_{Reviews (9) · Last reviewed commit: "test: the echo agent's CDP open waits pa..." · Reviewed by Greptile}