Browser Use Version

0.12.5

Bug Description, Steps to Reproduce, Screenshots

Individual CDP calls within handlers lack timeouts, causing indefinite hangs with remote browsers

Summary

When using browser-use with remote browsers via CDP (e.g., Browserless), individual CDP calls like Runtime.evaluate inside DOMWatchdog.on_BrowserStateRequestEvent can hang indefinitely if the CDP WebSocket connection becomes stale. There is no per-call timeout — the only safety net is the event-level timeout (default 30s), which wastes the entire timeout window before failing.

Environment

• browser-use 0.12.5
• Remote browser via CDP (Browserless)
• keep_alive=True

Observed behaviour

When running sequential agents on the same browser session with keep_alive=True, there are naturally idle periods between agent runs where no CDP traffic flows. After such an idle period, the next BrowserStateRequestEvent can hang:

1. DownloadsWatchdog completes but its CDP call (get_current_page_url) takes 3.0s instead of the usual 0.00-0.01s — an early signal the connection is degraded
2. DOMWatchdog enters _get_pending_network_requests
3. get_or_create_cdp_session(focus=True) succeeds but also takes 3.0s
4. The subsequent Runtime.evaluate call never returns — no error, no timeout, just an indefinite await
5. The handler blocks until the event-level timeout fires

The connection degradation is not deterministic — sometimes connections survive long idle periods, sometimes they become stale after relatively short gaps. No CDP errors or WebSocket close frames are logged when the connection degrades. The WebSocket appears open but is silently half-dead (sends succeed, responses never arrive).

This was identified by adding diagnostic instrumentation around the key methods in DOMWatchdog — wrapping _get_pending_network_requests, _capture_clean_screenshot, and _build_dom_tree_without_highlights with timing logs, and adding a heartbeat task to on_BrowserStateRequestEvent. This confirmed the handler was alive but stuck awaiting the CDP call indefinitely.

With local browsers this is unlikely to occur because the browser process is co-located. With remote browsers behind a WebSocket proxy, idle connections can degrade without either side detecting it.

Relation to #4374

This creates a catch-22 for remote browser users with keep_alive=True:

• Keep the event bus alive (workaround for Bug: Agent.close() with keep_alive=True drops CDP WebSocket connections, triggering unnecessary reconnection cycle #4374): The CDP WebSocket connection can degrade silently after idle periods, and the next Runtime.evaluate hangs indefinitely
• Let Agent.close() tear down the event bus (default behaviour): The CDP WebSocket handlers exit, session manager state is cleared, and the authenticated session is lost — exactly what Bug: Agent.close() with keep_alive=True drops CDP WebSocket connections, triggering unnecessary reconnection cycle #4374 describes

Both paths lead to broken sessions. The underlying issue is that browser-use assumes the CDP connection is always healthy if the WebSocket is open.

Root cause

CDP calls in handler code (e.g., _get_pending_network_requests, _build_dom_tree_without_highlights, _capture_clean_screenshot) await CDP protocol methods with no individual timeout. If the remote browser stops responding — due to WebSocket proxy recycling, idle timeout, or network conditions — these awaits block forever.

The relevant code path in dom_watchdog.py:

async def _get_pending_network_requests(self):
    cdp_session = await self.browser_session.get_or_create_cdp_session(focus=True)
    # This Runtime.evaluate has no timeout — hangs forever if CDP is stale
    result = await cdp_session.execute(
        Runtime.evaluate(expression=..., await_promise=True, ...)
    )

Reproduction

This is timing-dependent and requires a remote browser. The conditions are:

1. Connect to a remote browser via CDP with keep_alive=True
2. Apply the workaround from Bug: Agent.close() with keep_alive=True drops CDP WebSocket connections, triggering unnecessary reconnection cycle #4374 to prevent Agent.close() from tearing down the event bus (otherwise the CDP connection is killed on close and the session is lost before this issue can manifest)
3. Run an agent to completion (agent closes, browser session and CDP connection persist)
4. Wait with no CDP traffic (duration varies — sometimes 30s is enough, sometimes minutes are fine)
5. Start a new agent on the same browser session
6. The first BrowserStateRequestEvent has a chance of hanging indefinitely

The probability is non-deterministic and depends on the remote browser provider's WebSocket proxy behaviour and network conditions.

Failing Python Code

It is difficult to provide a minimal replication as it involves CDP and multiple browser instances.

It also requires addressing https://github.com/browser-use/browser-use/issues/4374 locally so that keep_alive actually works.

LLM Model

No response

Operating System & Browser Versions

No response

Full DEBUG Log Output