edge-zone,kvm,iso,cks: allow k8s deployment with direct-download iso by shwstppr · Pull Request #8142 · apache/cloudstack

shwstppr · 2023-10-25T07:57:46Z

Description

Allows registering direct-download ISOs
Allows VM deployment using ISOs in edge zones
Allows registering k8s versions as direct-download
Enables k8s cluster deployment in edge-zones

Fixes #7284

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

weizhouapache · 2023-10-25T08:51:11Z

@blueorangutan package

blueorangutan · 2023-10-25T08:52:03Z

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

codecov · 2023-10-25T09:14:02Z

Codecov Report

Merging #8142 (af37049) into 4.18 (6ae3b73) will increase coverage by 0.00%.
Report is 3 commits behind head on 4.18.
The diff coverage is 12.06%.

@@            Coverage Diff            @@
##               4.18    #8142   +/-   ##
=========================================
  Coverage     13.07%   13.07%           
  Complexity     9111     9111           
=========================================
  Files          2720     2720           
  Lines        257568   257625   +57     
  Branches      40154    40167   +13     
=========================================
+ Hits          33666    33677   +11     
- Misses       219670   219712   +42     
- Partials       4232     4236    +4

Files	Coverage Δ
...i/response/KubernetesSupportedVersionResponse.java	`70.45% <100.00%> (+1.40%)`	⬆️
...etes/version/AddKubernetesSupportedVersionCmd.java	`4.16% <0.00%> (-0.19%)`	⬇️
...va/com/cloud/network/router/NetworkHelperImpl.java	`3.67% <0.00%> (-0.01%)`	⬇️
.../hypervisor/kvm/storage/KVMStoragePoolManager.java	`0.47% <0.00%> (-0.01%)`	⬇️
...bernetes/version/KubernetesVersionManagerImpl.java	`53.99% <50.00%> (+0.10%)`	⬆️
...bernetes/cluster/KubernetesClusterManagerImpl.java	`4.37% <0.00%> (-0.03%)`	⬇️
...udstack/storage/image/TemplateDataFactoryImpl.java	`0.00% <0.00%> (ø)`
...ud/hypervisor/kvm/storage/KVMStorageProcessor.java	`4.94% <0.00%> (-0.04%)`	⬇️

... and 5 files with indirect coverage changes

📣 Codecov offers a browser extension for seamless coverage viewing on GitHub. Try it in Chrome or Firefox today!

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

blueorangutan · 2023-10-25T09:46:38Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 7505

DaanHoogland · 2023-10-25T11:19:03Z

@blueorangutan test

blueorangutan · 2023-10-25T11:20:03Z

@DaanHoogland a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

yadvr

LGTM - didn't test it, @shwstppr pl also look at any smoketests/unit tests changes

blueorangutan · 2023-10-25T23:22:05Z

[SF] Trillian test result (tid-8094)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 41765 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8142-t8094-kvm-centos7.zip
Smoke tests completed. 105 look OK, 3 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_08_upgrade_kubernetes_ha_cluster	`Failure`	640.48	test_kubernetes_clusters.py
test_create_pvlan_network	`Error`	0.05	test_pvlan.py
test_hostha_enable_ha_when_host_disabled	`Error`	4.76	test_hostha_kvm.py
test_hostha_enable_ha_when_host_in_maintenance	`Error`	303.91	test_hostha_kvm.py

shwstppr · 2023-10-26T05:07:08Z

@blueorangutan package

blueorangutan · 2023-10-26T05:08:03Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2023-10-26T06:08:35Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 7516

shwstppr · 2023-10-26T06:27:39Z

@blueorangutan test

blueorangutan · 2023-10-26T06:28:03Z

@shwstppr a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

github-actions · 2023-10-26T07:01:33Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

blueorangutan · 2023-10-26T18:56:45Z

[SF] Trillian test result (tid-8100)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43485 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8142-t8100-kvm-centos7.zip
Smoke tests completed. 108 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

DaanHoogland

looks mostly good. one potential UI issue.

DaanHoogland · 2023-11-02T13:29:35Z

...storage/image/src/main/java/org/apache/cloudstack/storage/image/TemplateDataFactoryImpl.java

    private Long getOneMatchingPoolIdFromRefs(List<VMTemplateStoragePoolVO> existingRefs, List<StoragePoolVO> pools) {
        if (pools.isEmpty()) {
-            throw new CloudRuntimeException("No storage pools found");
+            return null;


returning null is not a good practice. Can this if (pools.isEmpty()) be handled in the calling methods and an exception be thrown here?

DaanHoogland · 2023-11-02T13:31:35Z

...storage/image/src/main/java/org/apache/cloudstack/storage/image/TemplateDataFactoryImpl.java

+     * Retrieve storage pools with scope = cluster or zone or local matching clusterId or dataCenterId or hostId depending on their scope
     */
-    private List<StoragePoolVO> getStoragePoolsFromClusterOrZone(Long clusterId, long dataCenterId, Hypervisor.HypervisorType hypervisorType) {
+    private List<StoragePoolVO> getStoragePoolsFromClusterOrZoneOrLocal(Long clusterId, long dataCenterId, Hypervisor.HypervisorType hypervisorType, long hostId) {


naming suggestion

Suggested change

private List<StoragePoolVO> getStoragePoolsFromClusterOrZoneOrLocal(Long clusterId, long dataCenterId, Hypervisor.HypervisorType hypervisorType, long hostId) {

private List<StoragePoolVO> getStoragePoolsForScope(Long clusterId, long dataCenterId, Hypervisor.HypervisorType hypervisorType, long hostId) {

...ns/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStoragePoolManager.java

...storage/image/src/main/java/org/apache/cloudstack/storage/image/TemplateDataFactoryImpl.java

...ernetes-service/src/main/java/com/cloud/kubernetes/version/KubernetesVersionManagerImpl.java

ui/src/views/image/AddKubernetesSupportedVersion.vue

kiranchavala

@shwstppr

Can we show direct download section in Images> Kubernetes iso details page

The detail is shown only under Images> ISO's

The detail is not shown under Images> Kubernetes ISO's

kiranchavala

@shwstppr Cloudstack should not show L2 networks in the, network dropdown create kubernetes cluster form.

Currently its allowing to create k8s cluster in l2 network, its should throw a pop up saying l2 networks are not supported

shwstppr · 2023-11-03T12:58:57Z

@kiranchavala will try to make changes for your suggestions along with Daan's review comments. If you're done with testing can you please comment on the functionality part?

kiranchavala · 2023-11-06T08:20:50Z

@shwstppr Thanks

functionality wise it's working fine , was able to spin up k8s cluster on edge zones and launch vm's from iso

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr · 2023-11-07T12:31:31Z

@DaanHoogland @kiranchavala I've addressed your comments and feedback

@blueorangutan package

blueorangutan · 2023-11-07T12:32:03Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

DaanHoogland

clgtm

blueorangutan · 2023-11-07T13:36:04Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 7659

shwstppr · 2023-11-07T18:55:55Z

@blueorangutan test

blueorangutan · 2023-11-07T18:58:03Z

@shwstppr a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

DaanHoogland · 2023-11-08T13:43:38Z

@shwstppr is this ready for review now?

shwstppr · 2023-11-08T15:13:33Z

@DaanHoogland was waiting for the test result but I guess the test result didn't come due to API limit

DaanHoogland · 2023-11-08T16:07:14Z

from the backend:

Smoke tests completed. 108 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_06_stop_cpvm	`Failure`	35.29	test_ssvm.py

test failure:
FAIL: Test stop CPVM

Traceback (most recent call last):
File "/marvin/tests/smoke/test_ssvm.py", line 824, in test_06_stop_cpvm
self.test_04_cpvm_internals()
File "/marvin/tests/smoke/test_ssvm.py", line 656, in test_04_cpvm_internals
"Check cloud service is running or not"
File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackTestCase.py", line 94, in assertEqual
super(cloudstackTestCase,self).assertEqual(first,second,msg)
AssertionError: 0 != 1 : b'Check cloud service is running or not'

shwstppr · 2023-11-09T04:09:40Z

@kiranchavala Is it okay to merge now? I addressed your comments.

kiranchavala

LGTM, tested manually and it works

…pache#8142) Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr added 4 commits July 8, 2023 13:11

wip

65247f6

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Merge branch '4.18' into cks-version-direct

3b0ea65

fix

9241d00

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

fix for selecting correct pool ref

2ec2220

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

boring-cyborg bot added component:api component:kubernetes component:kvm component:networking component:storage component:UI labels Oct 25, 2023

fix since

a32e1e6

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

yadvr approved these changes Oct 25, 2023

View reviewed changes

shwstppr added this to the 4.18.2.0 milestone Oct 26, 2023

github-actions bot added the status:has-conflicts label Oct 26, 2023

Merge branch '4.18' into cks-version-direct

4dd05a9

github-actions bot removed the status:has-conflicts label Oct 26, 2023

rajujith assigned kiranchavala Nov 2, 2023

DaanHoogland reviewed Nov 2, 2023

View reviewed changes

kiranchavala reviewed Nov 3, 2023

View reviewed changes

shwstppr added 5 commits November 7, 2023 15:56

address testing suggestions

a7da524

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

fix, address comments

76f8c8f

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

fix, refactor direct-download version logic

49e019d

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

ui issue

f7c02b9

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

fix for consecutive zone change

af37049

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

DaanHoogland approved these changes Nov 7, 2023

View reviewed changes

shwstppr marked this pull request as ready for review November 8, 2023 15:11

kiranchavala approved these changes Nov 10, 2023

View reviewed changes

DaanHoogland merged commit d0f3233 into apache:4.18 Nov 10, 2023

DaanHoogland deleted the cks-version-direct branch November 10, 2023 12:56

dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request Nov 16, 2023

edge-zone,kvm,iso,cks: allow k8s deployment with direct-download iso (a…

da70a38

…pache#8142) Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr mentioned this pull request Dec 5, 2023

Edge Zone can not register iso #7284

Closed

kiranchavala mentioned this pull request Apr 23, 2025

UI: Unable to select edge zone during iso registration #10766

Closed

shwstppr mentioned this pull request Aug 29, 2025

Unable to deploy a vm from direct download Iso #11521

Closed

	private List<StoragePoolVO> getStoragePoolsFromClusterOrZoneOrLocal(Long clusterId, long dataCenterId, Hypervisor.HypervisorType hypervisorType, long hostId) {
	private List<StoragePoolVO> getStoragePoolsForScope(Long clusterId, long dataCenterId, Hypervisor.HypervisorType hypervisorType, long hostId) {

Conversation

shwstppr commented Oct 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

weizhouapache commented Oct 25, 2023

Uh oh!

blueorangutan commented Oct 25, 2023

Uh oh!

codecov bot commented Oct 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

blueorangutan commented Oct 25, 2023

Uh oh!

DaanHoogland commented Oct 25, 2023

Uh oh!

blueorangutan commented Oct 25, 2023

Uh oh!

yadvr left a comment

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Oct 25, 2023

Uh oh!

shwstppr commented Oct 26, 2023

Uh oh!

blueorangutan commented Oct 26, 2023

Uh oh!

blueorangutan commented Oct 26, 2023

Uh oh!

shwstppr commented Oct 26, 2023

Uh oh!

blueorangutan commented Oct 26, 2023

Uh oh!

github-actions bot commented Oct 26, 2023

Uh oh!

blueorangutan commented Oct 26, 2023

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland Nov 2, 2023

Choose a reason for hiding this comment

Uh oh!

shwstppr Nov 7, 2023

Choose a reason for hiding this comment

Uh oh!

DaanHoogland Nov 2, 2023

Choose a reason for hiding this comment

Uh oh!

shwstppr Nov 7, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kiranchavala left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kiranchavala left a comment

Choose a reason for hiding this comment

Uh oh!

shwstppr commented Nov 3, 2023

Uh oh!

kiranchavala commented Nov 6, 2023

Uh oh!

shwstppr commented Nov 7, 2023

Uh oh!

blueorangutan commented Nov 7, 2023

Uh oh!

shwstppr commented Oct 25, 2023 •

edited

Loading

codecov bot commented Oct 25, 2023 •

edited

Loading

kiranchavala left a comment •

edited

Loading

DaanHoogland commented Nov 8, 2023 •

edited

Loading

test failure:
FAIL: Test stop CPVM