This is a useful direction. Browser agents are one of those places where the final answer is usually not enough to debug the failure.

For a local trace viewer, I would make a few things first-class:

• one stable run_id for the whole browser task
• step-level records: URL, title, action chosen, target element summary, model/tool output, latency, and error
• screenshot references, but not as the only evidence
• a diffable final result: what was extracted, clicked, submitted, or changed
• retry/repair attempts linked back to the failed step they were trying to fix
• version metadata: browser-use version, model/provider, prompt/template version, and config

The most useful viewer is probably not just “watch the run.” It is “compare this failed run to the last successful run and see where the path diverged.” That comparison view would be valuable for anyone trying to turn browser-use from a demo into a repeatable workflow.

Thanks, this is useful feedback. I agree that for Browser Use debugging the real workflow is often “where did this fail relative to the last passing run?”, not only replaying one trace.

I opened a BrowserTrace issue to track the first local version: aaronlab/browsertrace#369

For v0.1 I would keep it explicit and conservative:

• compare a failed run with a selected successful run, not an automatic global baseline
• use Browser Use / BrowserTrace / model / prompt / config metadata so the UI does not imply two unrelated runs are comparable
• highlight the first divergent step by URL/title/action/target element summary/model output/error
• show retry/repair attempts as linked evidence around the failed step
• include a diffable final result when extracted/clicked/submitted/changed output is available

The detail I am still sorting out is the Browser Use final-result shape: separate fields for extracted content, tool output, and final result, or one normalized summary. If you have a concrete pass/fail pair you would expect to compare, that would help make the UI less vague.

Follow-up: I shipped the first small version of the failed-vs-good comparison path in BrowserTrace v0.1.19.

It is intentionally explicit for now:

browsertrace compare <failed_run_id> <success_run_id>
browsertrace compare <failed_run_id> <success_run_id> --json

The first slice compares existing step fields (action, url, status, and error) and reports the first divergent step between two selected runs. It does not yet auto-pick a baseline, change the SQLite schema, or add a UI diff view.

Release notes:
https://github.com/aaronlab/browsertrace/releases/tag/v0.1.19

The next Browser Use-specific question is still the final-result shape: for a pass/fail pair, would you expect comparison to separate extracted content, tool output, final result, and retry/repair attempts, or normalize those into one summary first?

A concrete failed run + known-good run shape would help keep the UI honest. No stars/upvotes requested; I am looking for workflow feedback from people turning Browser Use runs into repeatable workflows.

Small follow-up: BrowserTrace v0.1.20 now exposes the same failed-vs-good comparison payload through the local UI server too:

GET /api/compare/<failed_run_id>/<success_run_id>

So the current Browser Use debugging path is:

browsertrace compare <failed_run_id> <success_run_id>
browsertrace compare <failed_run_id> <success_run_id> --json
curl http://127.0.0.1:3000/api/compare/<failed_run_id>/<success_run_id>

The API is meant for local dashboards, scripts, or automation preflight checks that want the first divergent action, URL, status, or error before opening the full trace UI.

Useful feedback is still very concrete: for real Browser Use failed-vs-good pairs, should the next comparison field be final result, extracted content, tool output, retry/repair attempts, selected element summaries, or version/config metadata?