Enable browser action tests in E2E suite and CI by linyows · Pull Request #149 · linyows/probe

linyows · 2026-01-13T08:54:16Z

Summary

Enable browser action tests in the E2E test suite using probe's native features to manage Xvfb, making tests work in both CI and local environments without external GitHub Action dependencies.

Key Changes

1. Enable Browser Tests in E2E Suite (`testdata/e2e-test.yml`)

Uncommented and enhanced browser tests
Added Xvfb management using probe's native features:
- background: true - Start Xvfb in background
- retry literal - Wait for Xvfb readiness with retries
- skipif: vars.ci == "" - Run Xvfb only in CI environments
- Set DISPLAY=:99 environment variable for browser tests
- Cleanup Xvfb after tests complete

2. Update CI Workflow (`.github/workflows/end-to-end-test.yml`)

Install xvfb and x11-utils packages for browser support
Use browser-actions/setup-chrome to install Chrome
Use probe-action (no xvfb-action wrapper needed)

3. Improve Browser Examples (`examples/browser.yml`)

Add CI environment variable: vars.ci: "{{ CI ?? '' }}"
Conditional headless mode: headless: "{{ vars.ci != '' }}"
- CI: headless mode enabled (works with Xvfb)
- Local: headless mode disabled (displays browser window)
Replace slow pkg.go.dev with www.google.com for faster tests
Increase timeout to 30s for network latency

Implementation Highlights

Self-Contained Solution:

No external GitHub Actions dependencies for Xvfb management
Uses only probe's native features (background, retry, skipif literals)
Works seamlessly in both CI and local environments

Environment-Aware:

CI environments automatically detect and use Xvfb
Local environments can run tests with visible browser window
Conditional execution using probe's expression system

Why Xvfb?:
Even in headless mode, Chrome requires an X11 display server for rendering. Xvfb (X Virtual FrameBuffer) provides a virtual display in CI environments where no physical display exists.

Test Results

All CI checks passing:

✅ Browser tests execute successfully
✅ Both "Get element text" and "Capture screenshot" jobs pass
✅ Color output preserved in CI logs

🤖 Generated with Claude Code

- Uncomment browser tests in testdata/e2e-test.yml - Add Chromium browser installation to CI workflow - Ensure browser tests can run in both local and CI environments Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Replace chromium-browser package installation with browser-actions/setup-chrome action to properly configure Chrome for chromedp in GitHub Actions. This resolves the "Missing X server or $DISPLAY" error by ensuring Chrome is correctly installed and configured for headless execution. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Replace probe-action with direct probe execution wrapped in xvfb-action to provide a virtual display for chromedp browser tests in CI. This resolves the "Missing X server or $DISPLAY" error by running tests under Xvfb (X Virtual Framebuffer). Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Increase timeout from 10s to 30s for the "Get element text" test to account for slower network access to external sites in CI environment. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Add 2-minute timeout to the browser test shell command to prevent premature termination when browser tests take longer to complete. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

The pkg.go.dev site can be slow to load in CI environments. Increase timeout to 60s to allow sufficient time for page load and element interactions. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Change the first browser test to use www.google.com instead of pkg.go.dev to avoid network timeout issues in CI. Simplify the test to just verify text extraction functionality with a title element check. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Add FORCE_COLOR=1 environment variable to preserve color output when running probe under xvfb-run, which disables TTY detection. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Change env.CI to CI for correct environment variable access. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

- Replace xvfb-action with probe-action for consistency - Install xvfb and x11-utils packages for browser tests - Add FORCE_COLOR=1 to preserve color output in CI Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Add CI variable to examples/browser.yml and set headless mode conditionally based on CI environment. This allows: - CI: headless=true (works with Xvfb) - Local: headless=false (displays browser window) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

No longer needed since we're using probe-action directly instead of xvfb-action which was wrapping the command with xvfb-run. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Apply timeout to allocator context to fix "websocket url timeout reached" error. The timeout now applies to both Chrome startup/WebSocket connection and chromedp operations. Previously, the timeout was only applied to chromedp operations, causing Chrome startup and WebSocket connection to use the default ~20s timeout. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Extend timeout to 1 minute to allow more time for Chrome startup, WebSocket connection, and external site access in CI environments. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

- Use example.com which is simpler, faster, and more reliable - Extract h1 element text ("Example Domain") instead of title - Add validation that extracted text is not empty Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Make the job name more concise while keeping it descriptive. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

github-actions · 2026-01-13T13:35:42Z

Code Metrics Report

	main (28add9a)	#149 (a233c14)	+/-
Coverage	48.7%	48.8%	+0.0%
Code to Test Ratio	1:1.1	1:1.1	-0.1
Test Execution Time	1m24s	22s	-1m2s

Details

  |                     | main (28add9a) | #149 (a233c14) |  +/-  |
  |---------------------|----------------|----------------|-------|
+ | Coverage            |          48.7% |          48.8% | +0.0% |
  |   Files             |             52 |             52 |     0 |
  |   Lines             |           5249 |           5251 |    +2 |
+ |   Covered           |           2560 |           2563 |    +3 |
- | Code to Test Ratio  |          1:1.1 |          1:1.1 |  -0.1 |
  |   Code              |          10573 |          10575 |    +2 |
  |   Test              |          12109 |          12109 |     0 |
+ | Test Execution Time |          1m24s |            22s | -1m2s |

Code coverage of files in pull request scope (55.6% → 56.3%)

Files	Coverage	+/-	Status
browser/client.go	46.9%	+0.6%	modified
mail/mock_server.go	73.1%	+1.0%	affected

Reported by octocov

github-actions · 2026-01-13T13:35:53Z

Code Metrics Report

	main (28add9a)	#149 (a233c14)	+/-
Coverage	48.7%	48.8%	+0.0%
Code to Test Ratio	1:1.1	1:1.1	-0.1
Test Execution Time	1m24s	22s	-1m2s

Details

  |                     | main (28add9a) | #149 (a233c14) |  +/-  |
  |---------------------|----------------|----------------|-------|
+ | Coverage            |          48.7% |          48.8% | +0.0% |
  |   Files             |             52 |             52 |     0 |
  |   Lines             |           5249 |           5251 |    +2 |
+ |   Covered           |           2560 |           2563 |    +3 |
- | Code to Test Ratio  |          1:1.1 |          1:1.1 |  -0.1 |
  |   Code              |          10573 |          10575 |    +2 |
  |   Test              |          12109 |          12109 |     0 |
+ | Test Execution Time |          1m24s |            22s | -1m2s |

Code coverage of files in pull request scope (55.6% → 56.3%)

Files	Coverage	+/-	Status
browser/client.go	46.9%	+0.6%	modified
mail/mock_server.go	73.1%	+1.0%	affected

Reported by octocov

Enable browser action tests in E2E suite and CI

610862a

- Uncomment browser tests in testdata/e2e-test.yml - Add Chromium browser installation to CI workflow - Ensure browser tests can run in both local and CI environments Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>