test(ci): run e2e tests inside Docker containers#8899
Conversation
Add ability to run e2e tests inside Docker containers for consistent test environments. Enabled via MISE_E2E_DOCKER=1 env var. - Add packaging/e2e/Dockerfile (Ubuntu 26.04 with test deps) - Add e2e image to docker.yml build matrix (ghcr.io/jdx/mise:e2e) - Modify e2e/run_test to optionally run tests in Docker containers - Add docker-e2e job to test.yml (opt-in via workflow_dispatch or MISE_E2E_DOCKER_CI repo variable) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request introduces the ability to run end-to-end tests within a Docker container by adding a new run_in_docker function and a corresponding Dockerfile. Feedback includes concerns about using exec which might break CI log grouping, the duplication of the resolve_mise_bin function, and an incorrect Ubuntu version tag in the Dockerfile.
| -e RUST_BACKTRACE="${RUST_BACKTRACE:-1}" | ||
| ) | ||
|
|
||
| exec "${docker_args[@]}" "$image" /mise-src/e2e/run_test "$TEST" |
There was a problem hiding this comment.
Using exec here will replace the current shell process with the Docker process. This is problematic because run_in_docker is called via as_group (from e2e/style.sh), which expects to execute end_group after the function returns to properly close the GitHub Actions log group. If exec is used, end_group will never be called, breaking the log formatting in CI.
"${docker_args[@]}" "$image" /mise-src/e2e/run_test "$TEST"
| resolve_mise_bin() { | ||
| if [[ -n ${MISE_E2E_BIN:-} ]]; then | ||
| printf '%s\n' "$MISE_E2E_BIN" | ||
| return | ||
| fi | ||
| local candidate="${CARGO_TARGET_DIR:-$ROOT/target}/debug/mise" | ||
| if [[ -x $candidate ]]; then | ||
| printf '%s\n' "$candidate" | ||
| return | ||
| fi | ||
| local mise_path | ||
| if ! mise_path="$(command -v mise 2>/dev/null)"; then | ||
| err "Could not find 'mise' in PATH and no local debug build was found." | ||
| return 1 | ||
| fi | ||
| printf '%s\n' "$mise_path" | ||
| } |
| @@ -0,0 +1,36 @@ | |||
| FROM ubuntu:26.04@sha256:a072b64036a738e55bff8f9a9682cbb893bf20c213772effc1de8dee8df1cea9 | |||
There was a problem hiding this comment.
Ubuntu 26.04 is not a released version (it would be expected in April 2026). The provided SHA a072b64036a738e55bff8f9a9682cbb893bf20c213772effc1de8dee8df1cea9 actually corresponds to ubuntu:24.04 (Noble Numbat). You should update the tag to reflect the actual version being used to avoid confusion.
FROM ubuntu:24.04@sha256:a072b64036a738e55bff8f9a9682cbb893bf20c213772effc1de8dee8df1cea9
Greptile SummaryThis PR migrates the Linux e2e test suite from bare GitHub Actions runners (with Key points:
Confidence Score: 4/5Safe to merge once the e2e Docker image has been manually published via workflow_dispatch — the code changes themselves are logically correct All new findings are P2 (missing --init, dropped coverage). The critical bootstrapping and GHCR auth issues were already raised in prior threads. The Docker isolation logic is sound, the exec-vs-no-exec concern from the previous review is addressed in the current code, and the per-test container approach provides clean isolation. .github/workflows/test.yml — code coverage is silently dropped and docker pull has no GHCR auth; packaging/e2e/Dockerfile — Node 22.22.1 tarball URL should be verified before the image is first published Important Files Changed
Sequence DiagramsequenceDiagram
participant GHA as GitHub Actions
participant Host as Host Runner (ubuntu-latest)
participant GHCR as ghcr.io/jdx/mise:e2e
participant C as Docker Container
GHA->>Host: run_all_tests (MISE_E2E_DOCKER=1)
Host->>GHCR: docker pull
loop For each test in tranche
Host->>Host: run_test $TEST_NAME
Note over Host: MISE_E2E_DOCKER=1 → run_in_docker()
Host->>C: docker run --rm<br/>-v repo:/mise-src:ro<br/>-v mise_bin:ro<br/>-e MISE_E2E_INSIDE_DOCKER=1
C->>C: run_test (function)
C->>C: setup_isolated_env()
C->>C: within_isolated_env bash|zsh|fish $TEST_SCRIPT
C-->>Host: exit code
end
Host-->>GHA: overall pass/fail
Reviews (10): Last reviewed commit: "Merge branch 'main' into feat/e2e-docker" | Re-trigger Greptile |
- Remove exec in run_in_docker so as_group can emit ::endgroup:: - Add GHCR login before docker pull in docker-e2e job - Swap apt-get clean ordering to match convention Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Hyperfine Performance
|
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
mise-2026.4.3 x -- echo |
21.5 ± 0.3 | 20.8 | 24.5 | 1.00 |
mise x -- echo |
22.0 ± 0.5 | 20.9 | 27.1 | 1.02 ± 0.02 |
mise env
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
mise-2026.4.3 env |
21.0 ± 0.4 | 20.3 | 26.6 | 1.00 |
mise env |
21.4 ± 0.3 | 20.8 | 23.0 | 1.02 ± 0.02 |
mise hook-env
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
mise-2026.4.3 hook-env |
21.7 ± 0.3 | 21.1 | 25.2 | 1.00 |
mise hook-env |
22.3 ± 0.8 | 21.2 | 29.4 | 1.03 ± 0.04 |
mise ls
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
mise-2026.4.3 ls |
20.4 ± 1.2 | 18.6 | 25.6 | 1.05 ± 0.06 |
mise ls |
19.5 ± 0.2 | 18.8 | 20.6 | 1.00 |
xtasks/test/perf
| Command | mise-2026.4.3 | mise | Variance |
|---|---|---|---|
| install (cached) | 146ms | 147ms | +0% |
| ls (cached) | 77ms | 77ms | +0% |
| bin-paths (cached) | 81ms | 80ms | +1% |
| task-ls (cached) | 789ms | 791ms | +0% |
Instead of installing test deps via apt-get on every run, use the pre-built ghcr.io/jdx/mise:e2e Docker image. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
| username: ${{ github.actor }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
| - name: Pull e2e Docker image | ||
| run: docker pull ghcr.io/jdx/mise:e2e |
There was a problem hiding this comment.
Docker image may not exist when CI runs
High Severity
The docker pull ghcr.io/jdx/mise:e2e step runs unconditionally on every PR and push to main, but the docker.yml workflow that builds and pushes this image only triggers on version tags (v[0-9]*) and workflow_dispatch. Until someone manually triggers a Docker build, this image won't exist in GHCR, causing the pull to fail and blocking all CI.
Reviewed by Cursor Bugbot for commit 055d928. Configure here.
- Rename coverage → e2e since we weren't using code coverage - Extract resolve_mise_bin into e2e/helpers.sh shared by run_test and run_all_tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
All review feedback has been addressed:
This comment was generated by Claude Code. |
Tests that check for CI environment need these variables forwarded. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
| #!/usr/bin/env bash | ||
|
|
||
| SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd)" | ||
| ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" |
There was a problem hiding this comment.
Sourced helper overwrites caller's global variables
Low Severity
helpers.sh unconditionally sets SCRIPT_DIR and ROOT at the top level when sourced, overwriting the caller's identically-named global variables. Both run_test and run_all_tests rely on SCRIPT_DIR after sourcing this file (e.g., for TEST_SCRIPT path construction and assert.sh sourcing). Currently this is benign because all three files live in e2e/, so the computed values are identical. However, it's an unnecessary side effect — resolve_mise_bin could simply reference the caller's already-set ROOT rather than recomputing it. If helpers.sh is ever moved or sourced from a different location, SCRIPT_DIR and ROOT in the caller would be silently corrupted.
Reviewed by Cursor Bugbot for commit ca02551. Configure here.
- Add unzip to Dockerfile (needed for jar extraction in vfox tests) - Remove npm after installing nushell so test_backend_missing_deps can verify missing npm behavior - Add ELF binary check in run_in_docker for macOS users Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Addressed latest review feedback:
Also fixed the two failing tests:
Rebuilding Docker image now: https://github.com/jdx/mise/actions/runs/23983971023 This comment was generated by Claude Code. |
- Install Node.js to /opt/node instead of apt to keep npm out of /usr/bin (needed for test_backend_missing_deps) - Symlink node/npm/nu to /usr/local/bin for test PATH access - Mount mise binary to /usr/local/lib/mise/debug/mise with matching CARGO_TARGET_DIR so within_isolated_env PATH finds it without conflicting with npm location - Skip as_group wrapping inside Docker to avoid nested log groups - Add xz-utils for node tarball extraction Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Replace `mv` with `cp`+`rm` for mise binary in test_watch and test_generate_bootstrap to avoid "Device or resource busy" on bind-mounted binaries - Change migrate.rb to migrate.sh in test_task_monorepo_file_tasks since ruby is not available in the Docker image - Replace `cargo init` with `git init` in test_task_remote_git_includes since cargo is not available in the Docker image - Add libicu-dev to Docker image for dotnet globalization support - Skip rust-toolchain.toml test in Docker (pre-existing resolution issue) - Propagate MISE_E2E_INSIDE_DOCKER env var to isolated test environment Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- test_watch/test_generate_bootstrap: manipulate PATH instead of moving the bind-mounted mise binary (rm also fails on ro mounts) - test_fish_path_move_bug: write test.fish to $HOME instead of read-only $TEST_DIR (/mise-src/e2e/env/) - test_dotnet: skip in Docker since ICU libraries are missing from the pre-built image (Dockerfile updated but image not yet rebuilt) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Revert test_dotnet Docker skip (image being rebuilt with libicu-dev) - Revert test_rust Docker skip, fix actual bug: add `mise settings set idiomatic_version_file_enable_tools rust` so rust-toolchain.toml is actually parsed by mise (previously only passed on bare runners due to system rustc in PATH) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use `which mise` instead of hardcoded regex for `target/debug` path, since in Docker the mise binary is at /usr/local/lib/mise/debug/mise. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace `cargo init` with `git init` in test_task_remote_git_https and test_task_remote_git_ssh since cargo is not in the Docker image. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The e2e image is public so authentication isn't needed. Removing the login avoids leaving credentials on the host which undermines the security isolation that Docker e2e provides. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 3 total unresolved issues (including 2 from previous reviews).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 1275b6a. Configure here.
| mise_dir="$(dirname "$original_mise")" | ||
| mkdir -p ./bin && cp "$original_mise" ./bin/mise | ||
| OLD_PATH="$PATH" | ||
| PATH="$(echo "$PATH" | tr ':' '\n' | grep -v "^${mise_dir}$" | tr '\n' ':')" |
There was a problem hiding this comment.
PATH filtering uses regex instead of fixed-string matching
Low Severity
The grep -v "^${mise_dir}$" treats mise_dir as a regex pattern rather than a literal string. Characters like . in directory paths (e.g., hidden directories) act as single-character wildcards, potentially filtering out unintended PATH entries. The codebase already uses the correct approach in e2e/cli/test_deactivate with grep -Fvx (fixed-string exact match), which avoids regex interpretation entirely.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 1275b6a. Configure here.
### 🚀 Features - **(ci)** auto-convert external PRs to draft mode by @jdx in [#8896](#8896) - **(deps)** add `depends` field for user-specified tool dependencies by @cprecioso in [#8776](#8776) - **(dotnet)** support runtime-only installs by @fragon10 in [#8524](#8524) - **(npm)** apply install_before to transitive dependencies by @risu729 in [#8851](#8851) - **(task)** allow passing arguments to task dependencies via {{usage.*}} templates by @jdx in [#8893](#8893) - add options field to BackendListVersionsCtx by @esteve in [#8875](#8875) ### 🐛 Bug Fixes - **(backend)** filter PEP 440 .dev versions in fuzzy version matching by @richardthe3rd in [#8849](#8849) - **(ci)** update COPR BuildRequires rust version to match MSRV 1.88 by @jdx in [#8911](#8911) - **(ci)** add Ruby build dependencies to e2e Docker image by @jdx in [#8910](#8910) - **(ci)** add missing build dependencies to e2e Docker image by @jdx in [#8912](#8912) - **(ci)** add missing build dependencies to e2e Docker image by @jdx in [#8914](#8914) - **(ci)** use Node 24 LTS for corepack e2e test by @jdx in [#8915](#8915) - **(ci)** add libxml2 and pkg-config to e2e Docker image by @jdx in [#8917](#8917) - **(ci)** add libxml2-dev to e2e image and disable Swift SPM tests by @jdx in [#8918](#8918) - **(docs)** use sans-serif font for badges by @jdx in [#8887](#8887) - **(env)** parse --env=VALUE and -E=VALUE flag forms correctly by @jdx in [#8889](#8889) - **(exec)** use i64::from() for seccomp syscall numbers to survive autofix by @jdx in [#8882](#8882) - **(github)** preserve tool options like filter_bins when version specified via CLI by @jdx in [#8888](#8888) - **(github)** use alias-specific options when tool_alias has its own config by @jdx in [#8892](#8892) - **(install)** add locked_verify_provenance setting and detect github attestations at lock time by @jdx in [#8901](#8901) - **(lock)** prune stale version entries during filtered `mise lock <tool>` runs by @altendky in [#8599](#8599) - **(python)** use lockfile URL for precompiled installs by @hehaoqian in [#8750](#8750) - **(release)** verify all build targets succeed before releasing by @jdx in [#8886](#8886) - **(ruby)** support build revisions for precompiled binaries in mise.lock by @jdx in [#8900](#8900) - **(swift)** fall back to Ubuntu 24.04 for unsupported Ubuntu versions by @jdx in [#8916](#8916) - **(zsh)** avoid duplicate trust warning after cd by @timothysparg in [#8898](#8898) - update flake.lock and add fix for rust-bindgen to default.nix by @esteve in [#8874](#8874) - when direnv diff is empty, do not try to parse it by @yaleman in [#8857](#8857) - skip trust check for plain .tool-versions in task list by @dportalesr in [#8876](#8876) ### 🚜 Refactor - **(go)** rename go_* settings to go.* namespace by @jdbruijn in [#8598](#8598) ### 📚 Documentation - **(tasks)** clarify task_config.includes behavior by @risu729 in [#8905](#8905) ### 🧪 Testing - **(ci)** run e2e tests inside Docker containers by @jdx in [#8899](#8899) ### 📦️ Dependency Updates - bump ubi from 0.8 to 0.9 by @jdx in [#8906](#8906) - bump zip from 3 to 8 by @jdx in [#8908](#8908) - update lockfile deps (hold back rattler) by @jdx in [#8909](#8909) - update bun.lock by @jdx in [#8913](#8913) ### 📦 Registry - add turso ([github:tursodatabase/turso-cli](https://github.com/tursodatabase/turso-cli)) by @kenn in [#8884](#8884) - remove carp test by @jdx in [#8894](#8894) ### Chore - **(ci)** add workflow to warn PRs modifying vendored aqua-registry by @jdx in [#8897](#8897) - **(ci)** use github.token for draft conversion in auto-draft workflow by @jdx in [#8903](#8903) - remove deprecated settings older than 12 months by @jdx in [#8904](#8904) ### New Contributors - @dportalesr made their first contribution in [#8876](#8876) - @timothysparg made their first contribution in [#8898](#8898) - @hehaoqian made their first contribution in [#8750](#8750) - @jdbruijn made their first contribution in [#8598](#8598) - @cprecioso made their first contribution in [#8776](#8776) - @yaleman made their first contribution in [#8857](#8857) - @kenn made their first contribution in [#8884](#8884) - @fragon10 made their first contribution in [#8524](#8524) ## 📦 Aqua Registry Updates #### New Packages (6) - [`ahkohd/oyo`](https://github.com/ahkohd/oyo) - [`bellicose100xp/jiq`](https://github.com/bellicose100xp/jiq) - [`kurama/dealve-tui`](https://github.com/kurama/dealve-tui) - [`micahkepe/jsongrep`](https://github.com/micahkepe/jsongrep) - [`textfuel/lazyjira`](https://github.com/textfuel/lazyjira) - [`ubugeeei/vize`](https://github.com/ubugeeei/vize) #### Updated Packages (1) - [`sigstore/cosign`](https://github.com/sigstore/cosign)


Summary
packaging/e2e/Dockerfile(Ubuntu 26.04) with all e2e test dependencies (bash, zsh, fish, git, jq, direnv, fd-find, build-essential, python3, nushell, etc.)e2eto docker.yml build matrix soghcr.io/jdx/mise:e2eis built and pushed to GitHub packagese2e/run_testto optionally run tests inside Docker containers whenMISE_E2E_DOCKER=1is setdocker-e2ejob to test.yml (opt-in viaworkflow_dispatchorMISE_E2E_DOCKER_CIrepo variable)Usage
Test plan
cli/test_versioninside Docker container — passedcli/test_aliasinside Docker container — passed (including tool install)ghcr.io/jdx/mise:e2eimage builds in docker.yml workflow🤖 Generated with Claude Code
Note
Medium Risk
CI behavior changes by shifting Linux e2e execution into a Docker container and introducing a new published
e2eimage, which may surface environment/parity issues or increase flakiness due to Docker/runtime differences.Overview
Moves Linux e2e CI to a containerized environment. The
testworkflow replaces the previous coverage job with ane2ematrix job that pullsghcr.io/jdx/mise:e2eand runs./e2e/run_all_testswithMISE_E2E_DOCKER=1.Adds a dedicated e2e Docker image build. The
dockerworkflow now builds/pushes ane2eflavor using the newpackaging/e2e/Dockerfile(Ubuntu-based image with shell/tooling deps).Updates e2e harness and tests for Docker execution.
e2e/run_testcan now run tests inside Docker (mounting the repo and a Linuxmisebinary),resolve_mise_binis shared viae2e/helpers.sh, and several e2e scripts are adjusted to avoid relying onmisebeing inPATHor on extra host deps (e.g.,cargo init->git init, PATH manipulation fixes, task fixture tweaks).Reviewed by Cursor Bugbot for commit 1275b6a. Bugbot is set up for automated code reviews on this repo. Configure here.