Skip to content

test: add integration tests for CLI client#8689

Merged
reedham-aws merged 6 commits intoaws:feat-buildkitfrom
reedham-aws:buildkit-tests-integ
Mar 5, 2026
Merged

test: add integration tests for CLI client#8689
reedham-aws merged 6 commits intoaws:feat-buildkitfrom
reedham-aws:buildkit-tests-integ

Conversation

@reedham-aws
Copy link
Copy Markdown
Contributor

Integration test PR for buildkit/container CLI support.

Added a few parameterized tests for some image based tests, as well as just general tests for using the --use-buildkit flag. Worth noting is that when using the CLI, there were a lot of resource warnings that the sockets were being left open. I couldn't figure out a fix to this, but I think they're harmless warnings so I ignored them.

Implementation PR: #8661.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@github-actions github-actions bot added area/local/start-api sam local start-api command area/build sam build command area/local/invoke sam local invoke command area/local/start-invoke area/schema JSON schema file pr/internal labels Feb 24, 2026
@reedham-aws reedham-aws marked this pull request as ready for review February 24, 2026 01:31
@reedham-aws reedham-aws requested a review from a team as a code owner February 24, 2026 01:31
@roger-zhangg
Copy link
Copy Markdown
Member

prob need to merge from develop for the chardet thing

Copy link
Copy Markdown
Member

@roger-zhangg roger-zhangg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would suggest to add a ARM64 test, and then mark one test -> x86/arm64 as https://github.com/aws/aws-sam-cli/blob/develop/CONTRIBUTING.md#tier-1-cross-platform-smoke-tests for running that test on finch / windows (to be added)

@reedham-aws
Copy link
Copy Markdown
Contributor Author

would suggest to add a ARM64 test, and then mark one test -> x86/arm64 as https://github.com/aws/aws-sam-cli/blob/develop/CONTRIBUTING.md#tier-1-cross-platform-smoke-tests for running that test on finch / windows (to be added)

Done, I added as tier1, although not sure if this would have fallen more into the tier1_extra category. I think the testing of the CLI is important enough, not sure about the parameters.

@reedham-aws
Copy link
Copy Markdown
Contributor Author

reedham-aws commented Feb 27, 2026

command_list += ["--debug"]

if use_buildkit:
command_list += ["--use-buildkit"]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will there be change in VS Code extension for this feature? If so, should we have a negate version of this as well: --no-use-buildkit. This should be similar to

"--use-container/--no-use-container",

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--no-use-buildkit exists in the implementation, is it required to be in the test options as well?

@reedham-aws
Copy link
Copy Markdown
Contributor Author

reedham-aws commented Mar 2, 2026

Latest integration test run: https://github.com/aws/aws-sam-cli/actions/runs/22595806682. I removed the ARM64 test because it wasn't working well in x86 environment. While BuildKit should technically support this, that's more testing a BuildKit feature than our implementation. I think QEMU needs to be set up in some way in the environment for it to fully work.

Edit: All related and non-flaky tests are passing

@reedham-aws reedham-aws force-pushed the buildkit-tests-integ branch from e4c0cab to bacfe0c Compare March 4, 2026 00:35
@reedham-aws
Copy link
Copy Markdown
Contributor Author

Latest test run: https://github.com/aws/aws-sam-cli/actions/runs/22649580848

Before, the tier1-finch tests were failing. This is because the test was incorrectly finding that the finch CLI was unavailable. The problem was that in the CI runner, finch required sudo to run, but the code did not supply it. The solution was to make it so that sudo was not required for finch or nerdctl.

sudo chmod +s "$NERDCTL_PATH"
sudo chmod +s /usr/bin/finch

@reedham-aws reedham-aws merged commit b9aee06 into aws:feat-buildkit Mar 5, 2026
58 of 60 checks passed
@reedham-aws reedham-aws deleted the buildkit-tests-integ branch March 5, 2026 00:23
github-merge-queue bot pushed a commit that referenced this pull request Mar 10, 2026
* feat: add support for using CLI container clients (#8661)

* feat: add build_client

* feat: add --use-buildkit option

* feat: use build client when building with images

* lint and format

* feat: add command to cli

* fix: use absolute path for dockerfile

* test: add unit test for build_client

* test: add unit test for build_context

* fix: additional test fixes

* fix: schema update

* fix: remove hardcoded unix pathseps

* fix: remove provenance and sbom from docker buildx

* fix: refactor code to use SDKBuildClient

This change also refactors a lot of docker specific things to be "container_client" rather than "docker_client".
In addition, moving from instantiating the image build client in build context to lazily doing it in the app builder.

* test: add additional unit tests

* fix: fix error handling in CLIBuildClient

* fix: reorder error handling in CLIImageBuilder

* test: add integration tests for CLI client (#8689)

* test: add basic integration test

* test: more comprehensive integ tests

* test: add arm64 test and mark as tier1

* fix: update dockerfile error messages

* test: remove arm64 test

* fix: allow finch without sudo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/build sam build command area/local/invoke sam local invoke command area/local/start-api sam local start-api command area/local/start-invoke area/schema JSON schema file pr/internal

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants