chore: add standalone desktop E2E automation#650
Conversation
- e2e/desktop/ with isolated npm dependencies (not in pnpm workspace) - Test modes: smoke, login, model switch, update, full - Supports nightly/beta/stable downloads and local unsigned builds - System screen recording + Playwright screenshots on failure - State snapshot capture (config, openclaw state, runtime ports) - CI workflow with download/build source options - Async E2E trigger from nightly and release workflows (non-blocking)
There was a problem hiding this comment.
💡 Codex Review
nexu/.github/workflows/desktop-release.yml
Lines 15 to 17 in d174bfe
This workflow now calls github.rest.actions.createWorkflowDispatch in trigger-e2e, but the workflow-level permissions only grant contents: write; with explicit permissions, omitted scopes are set to none, so GITHUB_TOKEN lacks Actions write permission and the dispatch call will fail (typically 403), meaning release runs will not actually trigger the follow-up E2E workflow.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
5 boundary condition tests: - Crash recovery: Force Quit simulation → verify launchd services survive → restart - Orphan cleanup: kill Electron only → verify stale sidecars detected and cleaned - Port conflict: occupy port 50800 → verify app handles EADDRINUSE - Stale state: fake runtime-ports.json → verify app ignores and does fresh start - Double launch: second instance → verify single-instance lock
…e per test - verify_services(): checks controller API, web server, openclaw health, launchd registration, TCP listeners, and electron process after every startup - resilience_update_residual: kill Electron with launchd services still running, verify new app teardowns residual services and starts fresh - resilience_reset: clean home directory before each resilience scenario to avoid stale login/config state leaking between tests - Main flow: reset home for smoke/login/resilience modes, preserve for model/update (test with existing state) - Remove IPC hook in login scenario that caused premature quit dialog
What
Add a standalone desktop E2E automation suite under
e2e/desktop/.Why
Desktop packaged app testing has been manual and fragile. Need automated verification of DMG install, codesign, cold start, login flow, model switching, and update scenarios — with full diagnostics capture for CI debugging.
How
e2e/desktop/with isolatedpackage.json(not in pnpm workspace, dependencies installed separately vianpm install)run-e2e.sh) + Playwright scenarios (packaged-e2e.mjs)smoke,login,model,update,fullNEXU_DESKTOP_E2E_SKIP_CODESIGN=true).mov) + Playwright screenshots + state directory snapshots on failure.github/workflows/desktop-e2e.yml) withdownload/buildsource optionsAffected areas
e2e/desktop/(new).github/workflows/desktop-e2e.yml(new).github/workflows/desktop-nightly.yml(added trigger-e2e job).github/workflows/desktop-release.yml(added trigger-e2e job)Checklist
bash -n,node --check)