🔍 Scout Research Report

Triggered by @pelikhan

Executive Summary

Replacing the MCP Playwright server with @playwright/cli is well-motivated: the CLI (released early 2026) cuts token usage by ~4x by saving snapshots to disk rather than streaming them into context. The migration requires changes across the gh-aw workflow engine — primarily replacing Docker MCP container config generation with a bash-based setup that installs and invokes playwright-cli commands. The existing tools: playwright: frontmatter key can be preserved as the user-facing API; only the internal rendering changes.

Click to expand detailed findings

Research Findings

Why Replace MCP with CLI?

Factor	MCP (current)	CLI (target)
Token cost per session	~114,000 tokens	~27,000 tokens (~76% less)
Schema overhead	~3,600 tokens upfront (26 tools)	~68 tokens (SKILL.md, loaded once)
Page snapshots	Streamed inline to context	Saved as YAML to disk; agent reads on demand
Screenshots	Returned as image bytes in context	Saved as PNG to `.playwright-cli/`; not in context
Setup	Docker container (`mcr.microsoft.com/playwright/mcp`)	`npm install -g` @playwright/cli`` + browser install
Requires filesystem	No	Yes (required)
Requires shell/bash	No	Yes (required)

The CLI approach is specifically designed for coding agents (Claude Code, Copilot, Cursor) with filesystem and shell access — exactly the environment gh-aw workflows run in.

Current MCP Architecture in gh-aw

The existing implementation has these key pieces:

pkg/workflow/mcp_config_playwright_renderer.go — Renders the MCP Gateway JSON config for the Playwright Docker container (mcr.microsoft.com/playwright/mcp), with Docker args --init --network host --security-opt seccomp=unconfined --ipc=host and entrypoint args --output-dir /tmp/gh-aw/mcp-logs/playwright --no-sandbox
pkg/workflow/playwright_tools.go — Defines the 20 MCP tool names allowed (browser_click, browser_navigate, browser_snapshot, browser_take_screenshot, etc.)
pkg/workflow/mcp_playwright_config.go — PlaywrightToolConfig struct (Version, Args) and expression-extraction helpers for secure arg handling
pkg/workflow/tools_types.go — ToolsConfig.Playwright *PlaywrightToolConfig field
pkg/workflow/mcp_renderer.go — Routes "playwright" key to RenderPlaywright renderer
pkg/workflow/mcp_renderer_builtin.go — renderPlaywrightTOML for Codex engine
pkg/workflow/codex_engine.go — Codex-specific playwright config
pkg/workflow/mcp_setup_generator.go — Includes Playwright in setup sequence

Playwright CLI Command Set

# Install
npm install -g `@playwright/cli`
playwright-cli install --skills   # installs SKILL.md into agent context

# Core commands agents use
playwright-cli open (url)          # open browser + navigate
playwright-cli snapshot            # → saves YAML to .playwright-cli/, returns filepath
playwright-cli click (ref)         # click element by ref ID from snapshot
playwright-cli fill (ref) (value)  # fill input
playwright-cli screenshot          # → saves PNG to .playwright-cli/
playwright-cli close               # close session

All output artifacts (snapshots, screenshots) land in .playwright-cli/ on disk.

Migration Approach

The user-facing tools: playwright: frontmatter key can remain unchanged. Internally, the rendering changes from "configure a Docker MCP container" to "install CLI + inject skill knowledge + grant bash access."

Two viable strategies:

Option A — Implicit bash integration (simpler)

When tools: playwright: is set, gh-aw auto-installs @playwright/cli in the setup step (like npm install -g @playwright/cli && npx playwright install chromium)
Runs playwright-cli install --skills to drop SKILL.md into agent context
Grants bash: ["playwright-cli *"] automatically
No MCP server in the generated config at all

Option B — New playwright-cli tool key (explicit)

Introduce tools: playwright-cli: as a distinct config key
Keep tools: playwright: as legacy (deprecated) pointing to MCP
Enables gradual migration and A/B testing

Recommended: Option A — transparent to workflow authors, simpler migration path.

Files to Change (Estimated Scope)

File	Change
`pkg/workflow/mcp_config_playwright_renderer.go`	Delete or repurpose — no longer generates Docker MCP config
`pkg/workflow/playwright_tools.go`	Remove MCP tool list (no longer needed for CLI)
`pkg/workflow/mcp_playwright_config.go`	Keep expression extractor (still needed for `args` passthrough to CLI)
`pkg/workflow/tools_types.go`	Keep `PlaywrightToolConfig`; possibly add `Browser` field
`pkg/workflow/mcp_renderer.go`	Remove `"playwright"` case (CLI is not an MCP server)
`pkg/workflow/mcp_renderer_builtin.go`	Remove `renderPlaywrightTOML`
`pkg/workflow/mcp_setup_generator.go`	Add CLI install step for playwright
`pkg/workflow/codex_engine.go`	Update playwright handling
`actions/setup/md/playwright_prompt.md`	Update output path for CLI artifacts
`docs/src/content/docs/reference/playwright.md`	Update docs to describe CLI usage
`pkg/cli/workflows/test-playwright-args.md`	Update workflow test
`pkg/workflow/playwright_mcp_integration_test.go`	Rewrite for CLI
`pkg/cli/mcp_inspect_playwright_integration_test.go`	Update/remove MCP-specific tests

Key Technical Considerations

No Docker required — CLI runs directly on the runner; removes the Docker socket dependency for playwright specifically
--no-sandbox still needed — Chromium still needs --no-sandbox on GitHub Actions runners; this becomes a CLI flag: playwright-cli open --no-sandbox (url)
Output directory — Snapshots/screenshots go to .playwright-cli/ by default (configurable); gh-aw could redirect to /tmp/gh-aw/playwright-cli/ via env var or session flag
args field passthrough — Current PlaywrightToolConfig.Args could map to CLI flags (e.g., --browser chromium)
Skill injection — playwright-cli install --skills creates a SKILL.md that must be injected into the agent's system prompt (similar to the existing playwright_prompt.md)
Network firewall — Network domain restrictions still apply; CLI uses the same Chromium and respects the workflow's network: allowed: list

Recommendations

Adopt Option A (transparent swap): keep tools: playwright: frontmatter unchanged; replace the Docker MCP container with CLI setup under the hood
Preserve the args field: map existing args: ["--browser", "chromium"] to CLI flags
Run playwright-cli install --skills in the setup step and inject the resulting SKILL.md into the agent prompt
Redirect output to /tmp/gh-aw/playwright-cli/ to keep artifacts in the existing log path convention
Update integration tests: replace MCP tool name assertions with CLI bash command patterns
Deprecate version field: the MCP Docker image used a single image (no version); CLI follows npm semver — consider adding version support to pin @playwright/cli@x.y.z

Key Sources

[@playwright/cli npm package]((www.npmjs.com/redacted)
[Playwright CLI vs MCP — testdino.com]((testdino.com/redacted)
[CLI vs MCP Technical Comparison — joanmedia.dev]((www.joanmedia.dev/redacted)
[Playwright CLI Token-Efficient Alternative — testcollab.com]((testcollab.com/redacted)
[Playwright MCP Shifting to CLI — kualitatem.com]((www.kualitatem.com/redacted)

Suggested Next Steps

Create a spike branch: implement Option A for one engine (e.g., Claude) end-to-end
Add playwright-cli install to mcp_setup_generator.go setup step
Remove the MCP renderer path for playwright and replace with bash + SKILL injection
Update playwright_prompt.md with CLI-specific output path and usage notes
Rewrite integration tests to validate CLI invocation instead of MCP tool names
Run the existing test-playwright-args.md workflow against the new implementation to validate

References:

§23229207573

🔭 Intelligence gathered by Scout · ◷

Playwright cli #21529

Description

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions