Skip to content

fix(clp-package): Detect Docker plugin directory from docker info and resolve it if it's a symlink (fixes #1554).#1555

Merged
junhaoliao merged 4 commits into
y-scope:mainfrom
junhaoliao:compose-plugin-detect
Nov 5, 2025
Merged

fix(clp-package): Detect Docker plugin directory from docker info and resolve it if it's a symlink (fixes #1554).#1555
junhaoliao merged 4 commits into
y-scope:mainfrom
junhaoliao:compose-plugin-detect

Conversation

@junhaoliao

@junhaoliao junhaoliao commented Nov 5, 2025

Copy link
Copy Markdown
Member

Description

  • Prevent failures when Docker plugins are installed in non-standard locations or symlinked
    • Detect Docker Compose plugin path dynamically using docker info instead of hardcoded directory list
    • Resolve symlinks using readlink -f to get actual plugin directory
  • Add error handling when plugin path cannot be detected or resolved

Checklist

  • The PR satisfies the contribution guidelines.
  • This is a breaking change and that has been indicated in the PR title, OR this isn't a
    breaking change.
  • Necessary docs have been updated, OR no docs need to be updated.

Validation performed

Tested start-clp.sh which started successfully, on Ubuntu 22.04 WSL where the docker-compose plugin is symlinked to some target on the Windows host. No longer observed "clp_package_utils.general.DockerNotAvailableError: [Errno 2] Docker Compose is not installed or not functioning properly.":

junhao@ASUS-X870E:~/.docker/cli-plugins$ ll
total 16
drwxr-xr-x 2 junhao junhao 4096 Oct 30 12:35 ./
drwxr-xr-x 6 junhao junhao 4096 Oct 10 16:30 ../
lrwxrwxrwx 1 junhao junhao   66 Oct 30 12:35 docker-compose -> /mnt/c/Users/junhao/.docker/modules/cli-plugins-wsl/docker-compose*
lrwxrwxrwx 1 junhao junhao   64 Aug 26 12:07 docker-scout -> /mnt/c/Users/junhao/.docker/modules/cli-plugins-wsl/docker-scout*

Tested start-clp.sh which started successfully, on Debian 13 with non-Desktop Docker installation:

junhao@ASUS-X870E:/usr/libexec/docker/cli-plugins$ ls -l
total 151312
-rwxr-xr-x 1 root root 78355534 Oct  3 08:39 docker-buildx
-rwxr-xr-x 1 root root 76584394 Oct 22 13:52 docker-compose

Summary by CodeRabbit

  • Bug Fixes
    • More reliable Docker plugin detection with robust path validation.
    • Stricter error handling and fail-fast behaviour: missing or invalid plugins now produce clear errors and non-zero exit codes.

@junhaoliao junhaoliao requested a review from a team as a code owner November 5, 2025 05:36
@coderabbitai

coderabbitai Bot commented Nov 5, 2025

Copy link
Copy Markdown
Contributor

Walkthrough

Docker Compose plugin discovery now queries the Docker daemon via docker info to get the plugin path, validates and resolves that path with realpath, derives the plugin directory, exports CLP_DOCKER_PLUGIN_DIR, and returns/exists on failures instead of continuing after warnings. (49 words)

Changes

Cohort / File(s) Summary
Docker plugin discovery refactor
components/package-template/src/sbin/.common-env.sh
Replace iterative filesystem scanning with a docker info query to retrieve the Compose plugin path; add strict validation (existence check), resolve symlinked path with realpath, derive and export CLP_DOCKER_PLUGIN_DIR; emit errors and return/exit on failure rather than warning and continuing.

Sequence Diagram(s)

sequenceDiagram
    participant Script as .common-env.sh
    participant Docker as Docker Daemon

    rect rgb(220,240,255)
    Note over Script: Old flow — filesystem scan
    Script->>Script: Iterate candidate plugin dirs
    Script->>Script: Test plugin presence
    alt found
        Script->>Script: Export CLP_DOCKER_PLUGIN_DIR
    else not found
        Script->>Script: Possibly warn and continue
    end
    end

    rect rgb(240,240,220)
    Note over Script,Docker: New flow — daemon query
    Script->>Docker: Run `docker info` and parse plugin path
    Docker-->>Script: Return plugin path (or empty)
    alt plugin path returned
        Script->>Script: Check path exists
        Script->>Script: Resolve realpath -> resolved path
        Script->>Script: Derive plugin directory from resolved path
        Script->>Script: Export CLP_DOCKER_PLUGIN_DIR
    else no plugin path / invalid
        Script->>Script: Emit error and return/exit with non-zero
    end
    end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

  • Review parsing of docker info output for variations (locales, formatting).
  • Verify behaviour when realpath is unavailable and any fallback.
  • Check exit/return semantics to ensure callers handle non-zero correctly.
  • Confirm no unintended environment scope or variable name collisions.

Possibly related issues

Pre-merge checks and finishing touches

✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: detecting Docker plugin directory from docker info and resolving symlinks, directly related to the changeset modifications.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 44a3156 and ae7cc91.

📒 Files selected for processing (1)
  • components/package-template/src/sbin/.common-env.sh (1 hunks)
🧰 Additional context used
🧠 Learnings (11)
📓 Common learnings
Learnt from: junhaoliao
Repo: y-scope/clp PR: 0
File: :0-0
Timestamp: 2025-10-22T21:02:31.113Z
Learning: Repository y-scope/clp: Maintain deterministic CI/builds for Rust; add a check to verify Cargo.lock is in sync with Cargo.toml without updating dependencies (non-mutating verification in clp-rust-checks workflow).
📚 Learning: 2025-10-13T03:32:19.293Z
Learnt from: junhaoliao
Repo: y-scope/clp PR: 1414
File: tools/docker-images/clp-package/Dockerfile:20-24
Timestamp: 2025-10-13T03:32:19.293Z
Learning: In the clp repository's Dockerfiles (e.g., tools/docker-images/clp-package/Dockerfile), ENV directives should be split into separate lines for readability rather than consolidated to reduce layer count. This is especially true for PATH modifications, as agreed upon in PR #1166. Later ENV settings may depend on earlier ones (e.g., referencing CLP_HOME).

Applied to files:

  • components/package-template/src/sbin/.common-env.sh
📚 Learning: 2025-10-20T21:05:30.417Z
Learnt from: junhaoliao
Repo: y-scope/clp PR: 1413
File: tools/docker-images/clp-package/Dockerfile:22-24
Timestamp: 2025-10-20T21:05:30.417Z
Learning: In the clp repository's Dockerfiles, ENV directives should be consolidated into multi-line ENV statements when possible to reduce image layers. ENV statements should only be split into separate commands when consolidation is not possible due to dependencies (e.g., when later variables must reference earlier ones that need to be set first, or when PATH must be modified sequentially).

Applied to files:

  • components/package-template/src/sbin/.common-env.sh
📚 Learning: 2025-07-01T14:52:02.418Z
Learnt from: jackluo923
Repo: y-scope/clp PR: 1054
File: components/core/tools/docker-images/clp-env-base-musllinux_1_2-x86/build.sh:18-24
Timestamp: 2025-07-01T14:52:02.418Z
Learning: In the CLP project, consistency across platform build scripts is prioritized over defensive programming when it comes to git remote handling. All build.sh files in docker-images directories should follow the same pattern for git metadata injection.

Applied to files:

  • components/package-template/src/sbin/.common-env.sh
📚 Learning: 2025-07-07T17:43:04.349Z
Learnt from: jackluo923
Repo: y-scope/clp PR: 1054
File: components/core/tools/docker-images/clp-env-base-musllinux_1_2-aarch64/build.sh:3-5
Timestamp: 2025-07-07T17:43:04.349Z
Learning: In CLP project build scripts (specifically build.sh files in docker-images directories), maintain consistency with the established pattern of using separate `set -eu` and `set -o pipefail` commands rather than combining them into `set -euo pipefail`, to ensure uniform script structure across all platform build scripts.

Applied to files:

  • components/package-template/src/sbin/.common-env.sh
📚 Learning: 2025-08-09T04:07:27.083Z
Learnt from: Bill-hbrhbr
Repo: y-scope/clp PR: 1156
File: components/core/CMakeLists.txt:772-772
Timestamp: 2025-08-09T04:07:27.083Z
Learning: In the CLP project's CMakeLists.txt, when reviewing changes related to the ${zstd_TARGET} variable usage in test linking, the team is planning a refactoring PR to improve this mechanism. Guards for undefined target variables should be deferred to that separate PR rather than being added in focused dependency migration PRs.

Applied to files:

  • components/package-template/src/sbin/.common-env.sh
📚 Learning: 2025-07-23T09:54:45.185Z
Learnt from: Bill-hbrhbr
Repo: y-scope/clp PR: 1122
File: components/core/src/clp/clp/CMakeLists.txt:175-195
Timestamp: 2025-07-23T09:54:45.185Z
Learning: In the CLP project, when reviewing CMakeLists.txt changes that introduce new compression library dependencies (BZip2, LibLZMA, LZ4, ZLIB), the team prefers to address conditional linking improvements in separate PRs rather than expanding the scope of focused migration PRs like the LibArchive task-based installation migration.

Applied to files:

  • components/package-template/src/sbin/.common-env.sh
📚 Learning: 2025-08-25T06:29:59.610Z
Learnt from: Bill-hbrhbr
Repo: y-scope/clp PR: 1261
File: .github/workflows/clp-core-build.yaml:294-332
Timestamp: 2025-08-25T06:29:59.610Z
Learning: In the CLP project, Bill-hbrhbr prefers a "fail fast" approach for CI workflows - allowing potential command availability issues (like getconf in musllinux) to surface through CI failures rather than preemptively adding fallback logic, as they will fix issues when they occur.

Applied to files:

  • components/package-template/src/sbin/.common-env.sh
📚 Learning: 2025-10-22T21:14:12.225Z
Learnt from: junhaoliao
Repo: y-scope/clp PR: 1466
File: .github/workflows/clp-rust-checks.yaml:14-15
Timestamp: 2025-10-22T21:14:12.225Z
Learning: Repository y-scope/clp: In GitHub Actions workflows (e.g., .github/workflows/clp-rust-checks.yaml), YAML anchors/aliases are acceptable and preferred to avoid duplication; if actionlint flags an alias node (e.g., on push.paths) as an error, treat it as a tool limitation and do not require inlining unless the team asks to silence the warning.

Applied to files:

  • components/package-template/src/sbin/.common-env.sh
📚 Learning: 2025-10-22T21:02:31.113Z
Learnt from: junhaoliao
Repo: y-scope/clp PR: 0
File: :0-0
Timestamp: 2025-10-22T21:02:31.113Z
Learning: Repository y-scope/clp: Maintain deterministic CI/builds for Rust; add a check to verify Cargo.lock is in sync with Cargo.toml without updating dependencies (non-mutating verification in clp-rust-checks workflow).

Applied to files:

  • components/package-template/src/sbin/.common-env.sh
📚 Learning: 2025-09-15T22:20:40.750Z
Learnt from: quinntaylormitchell
Repo: y-scope/clp PR: 1125
File: components/job-orchestration/job_orchestration/scheduler/compress/compression_scheduler.py:267-291
Timestamp: 2025-09-15T22:20:40.750Z
Learning: For CLP compression jobs, the team has decided to fail the entire job immediately upon encountering any invalid input path, rather than continuing to process valid paths. This decision was made during PR #1125 development.

Applied to files:

  • components/package-template/src/sbin/.common-env.sh
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: package-image
🔇 Additional comments (5)
components/package-template/src/sbin/.common-env.sh (5)

34-36: LGTM!

The docker info template correctly isolates the compose plugin path with appropriate error suppression at the daemon-call level. The multi-line formatting improves readability.


38-41: LGTM!

The two-part validation (empty check and file existence check) appropriately catches both daemon failures and missing plugins. Early return prevents propagation of invalid paths.


43-47: LGTM!

The critical issue from prior review is properly fixed. Capturing readlink output separately and validating it before use prevents the silent fallback to current directory. The || true pattern correctly allows empty output capture without triggering set -e. Error message is clear and matches the agreed-upon suggestion.


49-53: LGTM!

The layered validation is appropriate for shell scripts. The explicit check for "." provides defense-in-depth even though readlink -f should always return absolute paths. The directory existence check (-d) provides a final sanity check. All three validations are non-redundant and necessary for robust error handling.


55-55: LGTM!

The export occurs only after all validation passes, and the conditional wrapper (line 33) respects any pre-set value. Variable naming is clear and follows project conventions.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 806a8c1 and 37da99a.

📒 Files selected for processing (1)
  • components/package-template/src/sbin/.common-env.sh (1 hunks)
🧰 Additional context used
🧠 Learnings (6)
📓 Common learnings
Learnt from: junhaoliao
Repo: y-scope/clp PR: 0
File: :0-0
Timestamp: 2025-10-22T21:02:31.113Z
Learning: Repository y-scope/clp: Maintain deterministic CI/builds for Rust; add a check to verify Cargo.lock is in sync with Cargo.toml without updating dependencies (non-mutating verification in clp-rust-checks workflow).
📚 Learning: 2025-10-13T03:32:19.293Z
Learnt from: junhaoliao
Repo: y-scope/clp PR: 1414
File: tools/docker-images/clp-package/Dockerfile:20-24
Timestamp: 2025-10-13T03:32:19.293Z
Learning: In the clp repository's Dockerfiles (e.g., tools/docker-images/clp-package/Dockerfile), ENV directives should be split into separate lines for readability rather than consolidated to reduce layer count. This is especially true for PATH modifications, as agreed upon in PR #1166. Later ENV settings may depend on earlier ones (e.g., referencing CLP_HOME).

Applied to files:

  • components/package-template/src/sbin/.common-env.sh
📚 Learning: 2025-10-20T21:05:30.417Z
Learnt from: junhaoliao
Repo: y-scope/clp PR: 1413
File: tools/docker-images/clp-package/Dockerfile:22-24
Timestamp: 2025-10-20T21:05:30.417Z
Learning: In the clp repository's Dockerfiles, ENV directives should be consolidated into multi-line ENV statements when possible to reduce image layers. ENV statements should only be split into separate commands when consolidation is not possible due to dependencies (e.g., when later variables must reference earlier ones that need to be set first, or when PATH must be modified sequentially).

Applied to files:

  • components/package-template/src/sbin/.common-env.sh
📚 Learning: 2025-07-07T17:43:04.349Z
Learnt from: jackluo923
Repo: y-scope/clp PR: 1054
File: components/core/tools/docker-images/clp-env-base-musllinux_1_2-aarch64/build.sh:3-5
Timestamp: 2025-07-07T17:43:04.349Z
Learning: In CLP project build scripts (specifically build.sh files in docker-images directories), maintain consistency with the established pattern of using separate `set -eu` and `set -o pipefail` commands rather than combining them into `set -euo pipefail`, to ensure uniform script structure across all platform build scripts.

Applied to files:

  • components/package-template/src/sbin/.common-env.sh
📚 Learning: 2025-07-01T14:52:02.418Z
Learnt from: jackluo923
Repo: y-scope/clp PR: 1054
File: components/core/tools/docker-images/clp-env-base-musllinux_1_2-x86/build.sh:18-24
Timestamp: 2025-07-01T14:52:02.418Z
Learning: In the CLP project, consistency across platform build scripts is prioritized over defensive programming when it comes to git remote handling. All build.sh files in docker-images directories should follow the same pattern for git metadata injection.

Applied to files:

  • components/package-template/src/sbin/.common-env.sh
📚 Learning: 2025-07-01T14:51:19.172Z
Learnt from: jackluo923
Repo: y-scope/clp PR: 1054
File: components/core/tools/scripts/lib_install/musllinux_1_2/install-packages-from-source.sh:6-8
Timestamp: 2025-07-01T14:51:19.172Z
Learning: In CLP installation scripts within `components/core/tools/scripts/lib_install/`, maintain consistency with existing variable declaration patterns across platforms rather than adding individual improvements like `readonly` declarations.

Applied to files:

  • components/package-template/src/sbin/.common-env.sh
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: package-image
  • GitHub Check: lint-check (macos-15)
  • GitHub Check: lint-check (ubuntu-24.04)
🔇 Additional comments (2)
components/package-template/src/sbin/.common-env.sh (2)

34-36: Dynamic Docker Compose plugin detection via docker info is well-implemented.

The query correctly uses the ClientInfo.Plugins format to extract the compose plugin path, which is more reliable than scanning hardcoded directories.


38-41: Early validation of plugin path existence is appropriate.

The check correctly validates that the plugin path returned by docker info both exists and is a regular file before attempting to resolve it.

Comment thread components/package-template/src/sbin/.common-env.sh
@junhaoliao junhaoliao marked this pull request as draft November 5, 2025 05:39
@junhaoliao junhaoliao marked this pull request as ready for review November 5, 2025 05:49
@kirkrodrigues kirkrodrigues changed the title fix(clp-package): Detect Docker plugin directory from docker info and resolve if it is a symlink (fixes #1554). fix(clp-package): Detect Docker plugin directory from docker info and resolve it if it's a symlink (fixes #1554). Nov 5, 2025
kirkrodrigues
kirkrodrigues previously approved these changes Nov 5, 2025

@kirkrodrigues kirkrodrigues left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor suggestion.

Comment thread components/package-template/src/sbin/.common-env.sh Outdated
Co-authored-by: kirkrodrigues <2454684+kirkrodrigues@users.noreply.github.com>
@junhaoliao junhaoliao merged commit ec509bf into y-scope:main Nov 5, 2025
20 checks passed
@junhaoliao junhaoliao deleted the compose-plugin-detect branch May 7, 2026 19:46
junhaoliao added a commit to junhaoliao/clp that referenced this pull request May 17, 2026
…nd resolve it if it's a symlink (fixes y-scope#1554). (y-scope#1555)

Co-authored-by: kirkrodrigues <2454684+kirkrodrigues@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants