Skip to content

Proper image-library overrides#88882

Merged
Felixoid merged 13 commits intoClickHouse:masterfrom
GSokol:proper_use_of_docker-library
Oct 27, 2025
Merged

Proper image-library overrides#88882
Felixoid merged 13 commits intoClickHouse:masterfrom
GSokol:proper_use_of_docker-library

Conversation

@GSokol
Copy link
Copy Markdown
Contributor

@GSokol GSokol commented Oct 22, 2025

See docker-library/official-images#20115 (comment)

Changelog category (leave one):

  • CI Fix or Improvement (changelog entry is not required)

@alexey-milovidov alexey-milovidov added the can be tested Allows running workflows for external contributors label Oct 22, 2025
@clickhouse-gh
Copy link
Copy Markdown
Contributor

clickhouse-gh bot commented Oct 22, 2025

Workflow [PR], commit [d912547]

Summary:

@clickhouse-gh clickhouse-gh bot added the pr-ci label Oct 22, 2025
@clickhouse-gh clickhouse-gh bot added the manual approve Manual approve required to run CI label Oct 23, 2025
@GSokol
Copy link
Copy Markdown
Contributor Author

GSokol commented Oct 23, 2025

Add one more fix but how could I know, if it fix everything or not... :(

@Felixoid
Copy link
Copy Markdown
Member

Partially works, but the new tests aren't added to the test suit https://github.com/ClickHouse/ClickHouse/actions/runs/18774789304/job/53580623535#step:5:583

@GSokol
Copy link
Copy Markdown
Contributor Author

GSokol commented Oct 24, 2025

@alexey-milovidov
Copy link
Copy Markdown
Member

Screenshot_20251026_155252

@GSokol
Copy link
Copy Markdown
Contributor Author

GSokol commented Oct 27, 2025

@alexey-milovidov, @Felixoid, sure, no rush. And I apologize for tons of errors and fixed in my PR. If there is some documentation how to run praktika jobs locally, I would rather test my code first.

@maxknv
Copy link
Copy Markdown
Member

maxknv commented Oct 27, 2025

@alexey-milovidov, @Felixoid, sure, no rush. And I apologize for tons of errors and fixed in my PR. If there is some documentation how to run praktika jobs locally, I would rather test my code first.

I'll create a PR to make it locally runnable

@Felixoid Felixoid added this pull request to the merge queue Oct 27, 2025
@Felixoid
Copy link
Copy Markdown
Member

Thanks a lot. Let's go, it's so long-awaited tests. Now, we'll add more and more tests to cover our docker images' functions.

Merged via the queue into ClickHouse:master with commit d4ed157 Oct 27, 2025
123 checks passed
@GSokol GSokol deleted the proper_use_of_docker-library branch October 27, 2025 09:45
@robot-ch-test-poll3 robot-ch-test-poll3 added the pr-synced-to-cloud The PR is synced to the cloud repo label Oct 27, 2025
@Algunenano
Copy link
Copy Markdown
Member

@GSokol
Copy link
Copy Markdown
Contributor Author

GSokol commented Oct 27, 2025

@Felixoid, oh no: looks like docker-build.sh doesn't working in the CI. Either need to rewrite it, or (/and) get rid of docker-library at all.

@maxknv
Copy link
Copy Markdown
Member

maxknv commented Oct 27, 2025

I cannot come up with a good fix for running this job locally. As a temporary solution, I can suggest this:

mkdir -p ./ci/tmp
wget https://clickhouse-builds.s3.amazonaws.com/REFs/master/dcdf7c72e818f47f77949c4deca3ce0b6cbfbe10/build_arm_release/artifact_report_build_arm_release.json -P ./ci/tmp
wget https://clickhouse-builds.s3.amazonaws.com/REFs/master/dcdf7c72e818f47f77949c4deca3ce0b6cbfbe10/build_amd_release/artifact_report_build_amd_release.json -P ./ci/tmp
python -m ci.praktika run 'Docker server image'

for image in check_images:
cmd = f"{run_sh} {image}"
cmd = f"{run_sh} {image} -c {repo_path / 'test/config.sh'} -c {config_override}"
test_results.append(Result.from_commands_run(name=test_name, command=cmd))
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need test_name = f"docker library {image} test" or something similar in the loop to avoid duplicates

@maxknv
Copy link
Copy Markdown
Member

maxknv commented Oct 27, 2025

Now we run this library test only in MasterCI, see here

I'm not sure about the motivation for if push there. Perhaps we can remove the if, so that this code gets tested in PRs as well. cc @Felixoid

@GSokol
Copy link
Copy Markdown
Contributor Author

GSokol commented Oct 28, 2025

@Felixoid, @maxknv, FYI, to fix the cross-platform docker-library image overrides, that caused the problem, I would try to push build command override initiative to docker-library. If the maintainers resist against it, I could try to implement our own docker-image testing system instead. What do you think?

@GSokol
Copy link
Copy Markdown
Contributor Author

GSokol commented Oct 28, 2025

Looks like docker-library team is not so enthusiastic about supporting buildx (even on my offer to implement it myself). Does it make any sense to implement our own harness around docker images testing? Here we could use python instead of bash, which makes the stuff slightly more transparent.

@Felixoid
Copy link
Copy Markdown
Member

Felixoid commented Oct 29, 2025

Sorry, I was on vacation for a few days, and I'm now getting through the backlog.

We can avoid rebuilding another image here and mount a file into a container. What was the original reason to use this method?

I'm not sure about the motivation for if push there. Perhaps we can remove the if, so that this code gets tested in PRs as well. cc @Felixoid

Answering this, it's the other way. if not push, there's a comment why See --output=type=docker. We run this test only in PR workflows.

Without this type=docker, there aren't any images in docker locally, so we have nothing to test.

https://github.com/GSokol/ClickHouse/blob/d9125475683827d9ed9b234b7ec74d0928e723a9/ci/jobs/docker_server.py#L217

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

can be tested Allows running workflows for external contributors manual approve Manual approve required to run CI pr-ci pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants