Skip to content

kubetest2-kops: use ephemeral discovery store for AWS#17809

Merged
k8s-ci-robot merged 6 commits intokubernetes:masterfrom
ameukam:test-aws-discovery-store
Dec 18, 2025
Merged

kubetest2-kops: use ephemeral discovery store for AWS#17809
k8s-ci-robot merged 6 commits intokubernetes:masterfrom
ameukam:test-aws-discovery-store

Conversation

@ameukam
Copy link
Copy Markdown
Member

@ameukam ameukam commented Dec 11, 2025

Retake of #17520

In addition:

  • ensure bucket is empty before deletion
  • use exponential back-off to apply public read policy
  • ensure bucket public access policy is disabled

@k8s-ci-robot
Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Dec 11, 2025
@ameukam
Copy link
Copy Markdown
Member Author

ameukam commented Dec 11, 2025

/test pull-kops-aws-distro-debian13

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Dec 11, 2025
@ameukam
Copy link
Copy Markdown
Member Author

ameukam commented Dec 11, 2025

/test pull-kops-aws-distro-debian13

@ameukam
Copy link
Copy Markdown
Member Author

ameukam commented Dec 11, 2025

/test pull-kops-aws-distro-al2023
/test pull-kops-aws-distro-ubuntu2404
/test pull-kops-e2e-cni-cilium

@ameukam ameukam marked this pull request as ready for review December 11, 2025 15:11
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 11, 2025
@hakman
Copy link
Copy Markdown
Member

hakman commented Dec 11, 2025

@ameukam This needs to be tested with something that enables discovery store. Do we have such job?

@ameukam
Copy link
Copy Markdown
Member Author

ameukam commented Dec 11, 2025

@ameukam This needs to be tested with something that enables discovery store. Do we have such job?

I don't think we have something like that as a presubmit.

@ameukam
Copy link
Copy Markdown
Member Author

ameukam commented Dec 11, 2025

/test pull-kops-e2e-cni-cilium

@hakman
Copy link
Copy Markdown
Member

hakman commented Dec 11, 2025

Please ignore the failing tests, let's get the review going first.

@ameukam
Copy link
Copy Markdown
Member Author

ameukam commented Dec 11, 2025

cc @justinsb @rifelpet

Copy link
Copy Markdown
Member

@hakman hakman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main part looks great, thank you both @ameukam & @xmudrii for this.
There are still some Q, though nothing huge.

Comment thread tests/e2e/kubetest2-kops/aws/s3.go Outdated
Comment on lines +156 to +159
// Empty the bucket first
paginator := s3.NewListObjectsV2Paginator(c.s3Client, &s3.ListObjectsV2Input{
Bucket: aws.String(bucketName),
})
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a good idea, but I would expect that kOps removes all the files from the bucket, otherwise it's a bug.
This approach may hide such issues.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kOps force delete the S3 buckets at shutdown whether the bucket is empty or not. This is just an extra step to guarantee the bucket is properly deleted.

Potential issues will probably not of the emptiness of the bucket.

Happy to drop this if you think it's unnecessary

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove it for now, please. Thanks!

Comment thread tests/e2e/kubetest2-kops/aws/s3.go Outdated
Comment thread tests/e2e/kubetest2-kops/aws/s3.go Outdated
Comment thread tests/e2e/kubetest2-kops/deployer/common.go
xmudrii and others added 2 commits December 14, 2025 11:30
Signed-off-by: Marko Mudrinić <mudrinic.mare@gmail.com>
Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
@ameukam ameukam force-pushed the test-aws-discovery-store branch from aa03eb6 to 43cee3e Compare December 14, 2025 12:05
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Dec 14, 2025
@ameukam
Copy link
Copy Markdown
Member Author

ameukam commented Dec 14, 2025

@hakman @rifelpet PTAL.

Comment thread tests/e2e/kubetest2-kops/aws/s3.go Outdated
Comment on lines +156 to +159
// Empty the bucket first
paginator := s3.NewListObjectsV2Paginator(c.s3Client, &s3.ListObjectsV2Input{
Bucket: aws.String(bucketName),
})
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove it for now, please. Thanks!

Comment thread tests/e2e/kubetest2-kops/deployer/up.go Outdated
d.KopsBinaryPath, "create", "cluster",
"--name", d.ClusterName,
"--cloud", d.CloudProvider,
"--discovery-store", d.discoveryStore(),
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be conditionally added if not empty, I guess.

Ensure public access policies can be disabled when the bucket is public

Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
bucket.

Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
…uster

on AWS.

Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
Ensure the discovery store S3 bucket is set with KOPS_DISCOVERY_STORE
value.

Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
@ameukam ameukam force-pushed the test-aws-discovery-store branch from 6d7230c to 6b15dd3 Compare December 18, 2025 10:29
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Dec 18, 2025
@hakman
Copy link
Copy Markdown
Member

hakman commented Dec 18, 2025

/test pull-kops-e2e-kubernetes-azure

@hakman
Copy link
Copy Markdown
Member

hakman commented Dec 18, 2025

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Dec 18, 2025
@k8s-ci-robot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hakman

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 18, 2025
@hakman
Copy link
Copy Markdown
Member

hakman commented Dec 18, 2025

/override pull-kops-e2e-k8s-gce-ipalias
/override pull-kops-e2e-k8s-aws-calico

@k8s-ci-robot
Copy link
Copy Markdown
Contributor

@hakman: Overrode contexts on behalf of hakman: pull-kops-e2e-k8s-aws-calico, pull-kops-e2e-k8s-gce-ipalias

Details

In response to this:

/override pull-kops-e2e-k8s-gce-ipalias
/override pull-kops-e2e-k8s-aws-calico

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot merged commit fcf37b8 into kubernetes:master Dec 18, 2025
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants