Skip to content

Bug: Screenshot blob in tool result poisons conversation context → API 400 on all subsequent turns #4742

@hellosimplerick

Description

@hellosimplerick

Browser Use Version

Version 147.0.7727.117 (Official Build) (x86_64)

Bug Description, Steps to Reproduce, Screenshots

Environment

macOS 26.3.1 (Build 25D2128), Intel x86_64
Python 3.11.3
Node v20.19.1
Claude Code 2.1.92 (desktop app)
browser-harness 0.1.0

What happened
I set up browser-harness following the README and successfully connected it to Chrome via CDP. Everything worked correctly through navigation and basic page interaction.
At the end of the session, Claude Code offered to star the browser-harness GitHub repo as a live demo that browser interaction was working. I confirmed ("yes"), and Claude Code ran 3 commands and read a file to complete the action.
Immediately after that step completed, every subsequent message in the session produced this error with no recovery possible:
API Error: 400 {"type":"error","error":{"type":"invalid_request_error","message":"Could not process image"},"request_id":"req_011CaQdtzxGnDEhJA72Z4BLw"}
The session was completely bricked. No message — including "hello" and "what does this error mean?" — could get through. The only resolution was starting a brand new Claude Code session.
Root cause (as diagnosed)
The browser action triggered a CDP Page.captureScreenshot call to verify state. The resulting screenshot was returned as a raw base64-encoded PNG blob inside the tool result payload and got stored in the Claude Code conversation history.
Claude Code re-sends the full conversation history on every turn. The Anthropic API rejected the inlined image blob on every subsequent request — either due to size, malformed encoding, or missing media_type metadata — producing the 400 error on every message indefinitely.
Steps to reproduce

Install and connect browser-harness to Chrome via CDP
Ask Claude Code to perform any browser action that triggers a page screenshot (e.g. "star this repo", "click this button", "verify this page loaded")
After the action completes, send any follow-up message
Observe: API Error: 400 - Could not process image on every message from that point forward

Expected behaviour
Tool results from CDP screenshot actions should not inline raw base64 image data into the conversation history.
Suggested fix (any of the following would resolve it)

Return a local file path or temp URL to the screenshot instead of the raw base64 blob
Strip screenshot data from tool results by default, only returning it when explicitly requested
Cap or compress the image to ensure it stays within API-accepted limits and includes correct media_type metadata

Impact
This is a hard session-killer. Once triggered, there is no recovery without abandoning the session entirely. Given that screenshot verification appears to be a core part of how browser-harness confirms actions, this will affect most non-trivial usage flows.
Workaround
Run /clear in Claude Code immediately after any browser action completes, before sending another message. This resets conversation context and prevents the bad block from persisting.

Failing Python Code

LLM Model

No response

Operating System & Browser Versions

No response

Full DEBUG Log Output

Error from within Claude Code:
API Error: 400 {"type":"error","error":{"type":"invalid_request_error","message":"Could not process image"},"request_id":"req_011CaQe8hfbkAuZnsR5UCGMu"}

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions