Reconcile CAPC Cluster as CloudStack CKS Cluster (Default: Opt Out) by yadvr · Pull Request #350 · kubernetes-sigs/cluster-api-provider-cloudstack

yadvr · 2024-03-22T10:16:32Z

Issue #, if available:

Description of changes:

The notion of a CAPC cluster represents a k8s cluster created and managed by CAPC. While a similar construct exists in CloudStack in form of CKS (CloudStack K8S Service) cluster. This PR aims to address #310 by introducing logic that can reconcile a CAPC cluster to a CKS cluster, while having this feature disabled by default (i.e. opt out by default). This will be useful for users (and integrations) who want to be aware about a potentially externally managed K8S cluster in CloudStack via the reconciled CKS cluster.

ACS 4.19 and later support such reconciliation with introduction of feature support PRs in upstream CloudStack releases:
apache/cloudstack-go#59
apache/cloudstack#7515
apache/cloudstack-documentation#315

Previously attempted as #250

Testing performed:

To be updated

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

k8s-ci-robot · 2024-03-22T10:16:35Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

netlify · 2024-03-22T10:16:49Z

✅ Deploy Preview for kubernetes-sigs-cluster-api-cloudstack ready!

Name	Link
🔨 Latest commit	`5bbb717`
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-sigs-cluster-api-cloudstack/deploys/667456f754e1920008530918
😎 Deploy Preview	https://deploy-preview-350--kubernetes-sigs-cluster-api-cloudstack.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

yadvr · 2024-03-22T10:17:13Z

Currently - in progress, draft - not fit or ready for review yet.

codecov-commenter · 2024-04-03T10:47:22Z

Codecov Report

Attention: Patch coverage is 30.81761% with 110 lines in your changes missing coverage. Please review.

Project coverage is 25.80%. Comparing base (d597e80) to head (1f37d9b).
Report is 39 commits behind head on main.

Files	Patch %	Lines
pkg/cloud/cks_cluster.go	0.00%	71 Missing ⚠️
controllers/cks_cluster_controller.go	58.97%	14 Missing and 2 partials ⚠️
controllers/cks_machine_controller.go	72.22%	5 Missing and 5 partials ⚠️
api/v1beta2/zz_generated.conversion.go	0.00%	8 Missing ⚠️
api/v1beta2/cloudstackcluster_conversion.go	0.00%	4 Missing ⚠️
api/v1beta1/conversion.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #350      +/-   ##
==========================================
+ Coverage   25.66%   25.80%   +0.13%     
==========================================
  Files          59       62       +3     
  Lines        5563     4906     -657     
==========================================
- Hits         1428     1266     -162     
+ Misses       3996     3508     -488     
+ Partials      139      132       -7

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

vishesh92 · 2024-04-05T06:26:13Z

/run-e2e -c 4.19

blueorangutan · 2024-04-05T06:28:03Z

@vishesh92 Invalid command: /run-e2e -c 4.19

Unsupported CloudStack version: 4.19

The command to run e2e test for CAPC.

Usage: /run-e2e [-k Kubernetes_Version] [-c CloudStack_Version] [-h Hypervisor] [-i Template/Image] [-f Kubernetes_Version_Upgrade_From] [-t Kubernetes_Version_Upgrade_To]

Supported Kubernetes versions are: ['1.27.2', '1.26.5', '1.25.10', '1.24.14', '1.23.3', '1.22.6']. The default value is '1.27.2'.
Supported CloudStack versions are: ['4.18', '4.17', '4.16']. If it is not set, an existing environment will be used.
Supported hypervisors are: ['kvm', 'vmware', 'xen']. The default value is 'kvm'.
Supported templates are: ['ubuntu-2004-kube', 'rockylinux-8-kube']. The default value is 'ubuntu-2004-kube'.
By default it tests Kubernetes upgrade from version '1.26.5' to '1.27.2'.

Examples:

/run-e2e
/run-e2e -k 1.27.2 -h kvm -i ubuntu-2004-kube
/run-e2e -k 1.27.2 -c 4.18 -h kvm -i ubuntu-2004-kube -f 1.26.5 -t 1.27.2

vishesh92 · 2024-04-05T06:43:43Z

/run-e2e -c 4.18

blueorangutan · 2024-04-05T06:44:03Z

@vishesh92 a jenkins job has been kicked to run test with following paramaters:

kubernetes version: 1.27.2
CloudStack version: 4.18
hypervisor: kvm
template: ubuntu-2004-kube
Kubernetes upgrade from: 1.26.5 to 1.27.2

vishesh92 · 2024-04-05T09:05:48Z

/run-e2e -c 4.19

blueorangutan · 2024-04-05T09:06:03Z

@vishesh92 a jenkins job has been kicked to run test with following paramaters:

kubernetes version: 1.27.2
CloudStack version: 4.19
hypervisor: kvm
template: ubuntu-2004-kube
Kubernetes upgrade from: 1.26.5 to 1.27.2

blueorangutan · 2024-06-12T20:59:46Z

Test Results : (tid-492)
Environment: kvm Rocky8(x3), Advanced Networking with Management Server Rocky8
Kubernetes Version: v1.27.2
Kubernetes Version upgrade from: v1.26.5
Kubernetes Version upgrade to: v1.27.2
CloudStack Version: 4.19
Template: ubuntu-2004-kube
E2E Test Run Logs: https://github.com/blueorangutan/capc-prs/releases/download/capc-pr-ci-cd/capc-e2e-artifacts-pr350-sl-492.zip



Summarizing 3 Failures:
 [FAIL] When testing affinity group [It] Should have host affinity group when affinity is pro
 /jenkins/workspace/capc-e2e-new/test/e2e/common.go:359
 [FAIL] When testing project [AfterEach] Should create a cluster in a project
 /jenkins/workspace/capc-e2e-new/test/e2e/project.go:103
 [TIMEDOUT] When the specified resource does not exist When starting with a healthy cluster [BeforeEach] Should fail to upgrade control plane machine due to insufficient compute resources
 /jenkins/workspace/capc-e2e-new/test/e2e/invalid_resource.go:111

Ran 30 of 31 Specs in 10715.160 seconds
FAIL! - Suite Timeout Elapsed -- 27 Passed | 3 Failed | 0 Pending | 1 Skipped
--- FAIL: TestE2E (10715.16s)
FAIL

blueorangutan · 2024-06-18T08:55:47Z

Test Results : (tid-493)
Environment: kvm Rocky8(x3), Advanced Networking with Management Server Rocky8
Kubernetes Version: v1.27.2
Kubernetes Version upgrade from: v1.26.5
Kubernetes Version upgrade to: v1.27.2
CloudStack Version: 4.19
Template: ubuntu-2004-kube
E2E Test Run Logs: https://github.com/blueorangutan/capc-prs/releases/download/capc-pr-ci-cd/capc-e2e-artifacts-pr350-sl-493.zip



Summarizing 2 Failures:
 [FAIL] When testing affinity group [It] Should have host affinity group when affinity is pro
 /jenkins/workspace/capc-e2e-new/test/e2e/common.go:359
 [FAIL] When testing project [AfterEach] Should create a cluster in a project
 /jenkins/workspace/capc-e2e-new/test/e2e/project.go:103

Ran 30 of 31 Specs in 9744.049 seconds
FAIL! -- 28 Passed | 2 Failed | 0 Pending | 1 Skipped
--- FAIL: TestE2E (9744.05s)
FAIL

blueorangutan · 2024-06-19T09:05:31Z

Test Results : (tid-495)
Environment: kvm Rocky8(x3), Advanced Networking with Management Server Rocky8
Kubernetes Version: v1.27.2
Kubernetes Version upgrade from: v1.26.5
Kubernetes Version upgrade to: v1.27.2
CloudStack Version: 4.19
Template: ubuntu-2004-kube
E2E Test Run Logs: https://github.com/blueorangutan/capc-prs/releases/download/capc-pr-ci-cd/capc-e2e-artifacts-pr350-sl-495.zip



Summarizing 3 Failures:
 [FAIL] When testing affinity group [It] Should have host affinity group when affinity is anti
 /root/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.4.8/framework/controlplane_helpers.go:116
 [FAIL] When testing project [AfterEach] Should create a cluster in a project
 /jenkins/workspace/capc-e2e-new/test/e2e/project.go:103
 [TIMEDOUT] When the specified resource does not exist [It] Should fail due to the specified control plane offering is not found [TC7]
 /jenkins/workspace/capc-e2e-new/test/e2e/invalid_resource.go:76

Ran 20 of 31 Specs in 10817.969 seconds
FAIL! - Suite Timeout Elapsed -- 17 Passed | 3 Failed | 0 Pending | 11 Skipped
--- FAIL: TestE2E (10817.97s)
FAIL

g-gaston

/lgtm

Thanks a lot!

I see you are trying to run tests. Let me know when you are done and ready to merge and I'll approve

blueorangutan · 2024-06-20T11:29:49Z

Test Results : (tid-496)
Environment: kvm Rocky8(x3), Advanced Networking with Management Server Rocky8
Kubernetes Version: v1.27.2
Kubernetes Version upgrade from: v1.26.5
Kubernetes Version upgrade to: v1.27.2
CloudStack Version: 4.19
Template: ubuntu-2004-kube
E2E Test Run Logs: https://github.com/blueorangutan/capc-prs/releases/download/capc-pr-ci-cd/capc-e2e-artifacts-pr350-sl-496.zip



Summarizing 4 Failures:
 [FAIL] When testing resource cleanup [It] Should create a new network when the specified network does not exist
 /jenkins/workspace/capc-e2e-new/test/e2e/resource_cleanup.go:89
 [FAIL] When testing affinity group [It] Should have host affinity group when affinity is pro
 /jenkins/workspace/capc-e2e-new/test/e2e/common.go:359
 [FAIL] When testing Kubernetes version upgrades [It] Should successfully upgrade kubernetes versions when there is a change in relevant fields
 /root/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.4.8/framework/machinedeployment_helpers.go:127
 [TIMEDOUT] When testing with custom disk offering [It] Should successfully create a cluster with a custom disk offering
 /jenkins/workspace/capc-e2e-new/test/e2e/custom_disk_offering.go:61

Ran 27 of 31 Specs in 10808.005 seconds
FAIL! - Suite Timeout Elapsed -- 23 Passed | 4 Failed | 0 Pending | 4 Skipped
--- FAIL: TestE2E (10808.01s)
FAIL

vishesh92 · 2024-06-20T16:20:30Z

/lgtm

Thanks a lot!

I see you are trying to run tests. Let me know when you are done and ready to merge and I'll approve

Thanks @g-gaston for review. @kiranchavala is currently testing this PR. Once he gives a go ahead, we can merge this PR. Meanwhile, I will fix the conflicts in go.mod.

yadvr · 2024-06-21T06:56:06Z

/lgtm

k8s-ci-robot · 2024-06-21T06:56:08Z

@rohityadavcloud: you cannot LGTM your own PR.

Details

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

yadvr · 2024-06-21T06:59:05Z

@vishesh92 while I opened the PR, it’s largely your work. Pl feel free to resubmit a new PR or lgtm this when this is ready after QA. cc @kiranchavala

yadvr · 2024-06-24T07:28:30Z

/run-e2e -c 4.19

blueorangutan · 2024-06-24T07:30:03Z

@rohityadavcloud a jenkins job has been kicked to run test with following paramaters:

kubernetes version: 1.27.2
CloudStack version: 4.19
hypervisor: kvm
template: ubuntu-2004-kube
Kubernetes upgrade from: 1.26.5 to 1.27.2

kiranchavala

LGTM,

Tested manually and

A externally managed kubernetes cluster got created in cloudstack when a capc deployment was initiated

yadvr · 2024-06-24T10:50:58Z

@kiranchavala can you add a /lgtm ?

/approve

blueorangutan · 2024-06-24T11:20:41Z

Test Results : (tid-497)
Environment: kvm Rocky8(x3), Advanced Networking with Management Server Rocky8
Kubernetes Version: v1.27.2
Kubernetes Version upgrade from: v1.26.5
Kubernetes Version upgrade to: v1.27.2
CloudStack Version: 4.19
Template: ubuntu-2004-kube
E2E Test Run Logs: https://github.com/blueorangutan/capc-prs/releases/download/capc-pr-ci-cd/capc-e2e-artifacts-pr350-sl-497.zip



Summarizing 3 Failures:
 [FAIL] When testing project [AfterEach] Should create a cluster in a project
 /jenkins/workspace/capc-e2e-new/test/e2e/project.go:103
 [FAIL] When testing Kubernetes version upgrades [It] Should successfully upgrade kubernetes versions when there is a change in relevant fields
 /root/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.4.8/framework/machinedeployment_helpers.go:127
 [TIMEDOUT] When testing machine remediation [It] Should replace a machine when it is destroyed
 /jenkins/workspace/capc-e2e-new/test/e2e/machine_remediation.go:59

Ran 29 of 31 Specs in 10816.701 seconds
FAIL! - Suite Timeout Elapsed -- 26 Passed | 3 Failed | 0 Pending | 2 Skipped
--- FAIL: TestE2E (10816.70s)
FAIL

kiranchavala

/lgtm

k8s-ci-robot · 2024-06-24T13:00:06Z

@kiranchavala: changing LGTM is restricted to collaborators

Details

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot · 2024-06-24T13:00:08Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kiranchavala, rohityadavcloud, weizhouapache

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [rohityadavcloud,weizhouapache]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

vishesh92 · 2024-06-24T13:07:44Z

/lgtm

k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Mar 22, 2024

k8s-ci-robot requested review from chrisdoherty4 and davidjumani March 22, 2024 10:16

k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Mar 22, 2024

yadvr requested review from vishesh92 and weizhouapache and removed request for chrisdoherty4 and davidjumani March 22, 2024 10:16

yadvr self-assigned this Mar 22, 2024

yadvr added this to the v0.5.0 milestone Mar 22, 2024

yadvr linked an issue Mar 22, 2024 that may be closed by this pull request

Reconcile a CAPC cluster to a CKS cluster #310

Closed

yadvr added the release:must-have label Mar 22, 2024

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Apr 2, 2024

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 5, 2024

Add finalizer for CksMachine Controller

3d1a8c3

Use pointer to a bool instead of bool

69b2fe9

kubernetes-sigs deleted a comment from blueorangutan Jun 19, 2024

g-gaston reviewed Jun 19, 2024

View reviewed changes

k8s-ci-robot assigned g-gaston Jun 19, 2024

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 19, 2024

g-gaston removed the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 19, 2024

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 20, 2024

Merge branch 'main' into reconcile-capc-cluster-to-cks

5bbb717

k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. and removed lgtm "Looks good to me", indicates that a PR is ready to be merged. labels Jun 20, 2024

kiranchavala mentioned this pull request Jun 24, 2024

UI: Issues related to Externally managed kuberntes cluster apache/cloudstack#9291

Closed

kiranchavala approved these changes Jun 24, 2024

View reviewed changes

sp1999 mentioned this pull request Sep 7, 2024

Bump kubernetes-sigs/cluster-api-provider-cloudstack to latest release aws/eks-anywhere-build-tooling#3409

Merged

Conversation

yadvr commented Mar 22, 2024

Uh oh!

k8s-ci-robot commented Mar 22, 2024

Uh oh!

netlify bot commented Mar 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for kubernetes-sigs-cluster-api-cloudstack ready!

Uh oh!

yadvr commented Mar 22, 2024

Uh oh!

codecov-commenter commented Apr 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

vishesh92 commented Apr 5, 2024

Uh oh!

blueorangutan commented Apr 5, 2024

Uh oh!

vishesh92 commented Apr 5, 2024

Uh oh!

blueorangutan commented Apr 5, 2024

Uh oh!

vishesh92 commented Apr 5, 2024

Uh oh!

blueorangutan commented Apr 5, 2024

Uh oh!

blueorangutan commented Jun 12, 2024

Uh oh!

blueorangutan commented Jun 18, 2024

Uh oh!

blueorangutan commented Jun 19, 2024

Uh oh!

g-gaston left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Jun 20, 2024

Uh oh!

vishesh92 commented Jun 20, 2024

Uh oh!

yadvr commented Jun 21, 2024

Uh oh!

k8s-ci-robot commented Jun 21, 2024

Uh oh!

yadvr commented Jun 21, 2024

Uh oh!

yadvr commented Jun 24, 2024

Uh oh!

blueorangutan commented Jun 24, 2024

Uh oh!

kiranchavala left a comment

Choose a reason for hiding this comment

Uh oh!

yadvr commented Jun 24, 2024

Uh oh!

blueorangutan commented Jun 24, 2024

Uh oh!

kiranchavala left a comment

Choose a reason for hiding this comment

Uh oh!

k8s-ci-robot commented Jun 24, 2024

Uh oh!

k8s-ci-robot commented Jun 24, 2024

Uh oh!

vishesh92 commented Jun 24, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

netlify bot commented Mar 22, 2024 •

edited

Loading

codecov-commenter commented Apr 3, 2024 •

edited

Loading

g-gaston left a comment •

edited

Loading