Add support for shared-file-system `/v2/services` and `/v2/scheduler-stats/pools` by SuperSandro2000 · Pull Request #2350 · gophercloud/gophercloud

SuperSandro2000 · 2022-02-11T14:37:42Z

Links to the line numbers/files in the OpenStack source code that support the
code in this PR:

https://docs.openstack.org/api-ref/shared-file-system/?expanded=#list-back-end-storage-pools
https://docs.openstack.org/api-ref/shared-file-system/?expanded=#list-services

coveralls · 2022-02-11T14:53:28Z

Coverage increased (+0.04%) to 79.932% when pulling 519e501 on sapcc:sharedfilesystems-schedulerstats-services into beef396 on gophercloud:master.

theopenlab-ci · 2022-02-11T16:10:51Z

Build failed.

gophercloud-unittest : SUCCESS in 5m 29s
gophercloud-acceptance-test-compute : SUCCESS in 1h 00m 38s
gophercloud-acceptance-test-storage : FAILURE in 50m 26s

SuperSandro2000 · 2022-02-12T18:40:40Z

I tried to find the log for the failing test but I couldn't. Where can I find out why the test is failing?

majewsky · 2022-02-14T10:38:59Z

I can see FAIL: TestVolumeAttachments (22.46s) because of

blockstorage.go:243: Unable to delete volume 4b387b87-1a13-49be-9e9f-84eaf1fab669: Bad request with: [DELETE http://192.168.0.117/volume/v3/d4f78f9a18b84e6999283039b06b4e9c/volumes/4b387b87-1a13-49be-9e9f-84eaf1fab669], error message: {"badRequest": {"code": 400, "message": "Invalid volume: Volume status must be available or error or error_restoring or error_extending or error_managing and must not be migrating, attached, belong to a group, have snapshots or be disassociated from snapshots after volume transfer."}}

Also FAIL: TestVolumes (13.58s) because of

convenience.go:35: Failure in volumes_test.go, line 61: expected false but got true

Both of these look unrelated to the changes in this branch. @EmilienM Can you trigger a re-run, please?

EmilienM · 2022-02-14T16:24:03Z

@majewsky Github actions look fine to me, those are the actual jobs we care now.

I'll look at the PR asap.

EmilienM · 2022-02-14T22:42:18Z

I just realized we don't have acceptance coverage for this one, @majewsky any plans for doing it?

SuperSandro2000 · 2022-02-15T14:28:31Z

We are currently upstreaming custom code in limes. I have tested the endpoints within limes against our staging openstack installation. For acceptance testing I would probably need to setup my own openstack cluster for testing inside a VM which would probably take a few days. I don't think I could run the tests against our staging installation as it could interfere with other people using it.
So if possible we would like to skip adding new acceptance tests for now.

EmilienM · 2022-02-15T16:28:51Z

We are currently upstreaming custom code in limes. I have tested the endpoints within limes against our staging openstack installation. For acceptance testing I would probably need to setup my own openstack cluster for testing inside a VM which would probably take a few days. I don't think I could run the tests against our staging installation as it could interfere with other people using it. So if possible we would like to skip adding new acceptance tests for now.

Right now sharedfilesystems is not tested by our Github actions, I'll work on that soon. But if you have access to any OpenStack cloud with Manila you should be able to run acceptance test against it, and write new tests for this PR. If it's not possible, I'll have to check with other maintainers, but we usually have a policy of documented & tested code before it merges into master.

SuperSandro2000 · 2022-02-24T16:16:15Z

I have pushed a commit which adds basic tests for the new things I have added in this PR. Locally I had to use an admin account to run them successful. I didn't look into how the tests are run and with what permissions but lets see what the CI run says.

theopenlab-ci · 2022-02-24T18:56:12Z

Build failed.

gophercloud-unittest : SUCCESS in 5m 48s
gophercloud-acceptance-test-compute : SUCCESS in 1h 08m 54s
gophercloud-acceptance-test-storage : FAILURE in 58m 28s

EmilienM · 2022-02-25T15:56:42Z

@SuperSandro2000 sorry for the lag, I finally pushed a PR to enable Manila in our CI: #2361
Once this merge, this PR will need to be rebased, so we can test your code. Thanks a lot for your patience, and yes indeed our new jobs (using Github actions) are way better than what we had before, more coverage, faster, etc.

EmilienM · 2022-02-27T19:32:08Z

please rebase to trigger new ci jobs

theopenlab-ci · 2022-03-07T11:42:04Z

Build failed.

gophercloud-unittest : RETRY_LIMIT in 5m 25s
gophercloud-acceptance-test-compute : RETRY_LIMIT in 45m 19s
gophercloud-acceptance-test-storage : RETRY_LIMIT in 55m 31s

mandre

Hi @SuperSandro2000, thanks for the PR. It's quite a big one, and it would probably be easier in the future to split your work per API calls, so that we can merge them independently when they're ready. This looks mostly OK, I left a few comments, just one thing that surprised me was that you only implemented a subset of the supported fields, is that on purpose? I think you've already made 90% of the work and implementing the missing fields shouldn't be that much effort.

acceptance/openstack/manila/v2/services_test.go

openstack/sharedfilesystems/v2/schedulerstats/requests.go

openstack/sharedfilesystems/v2/services/requests.go

openstack/sharedfilesystems/v2/schedulerstats/results.go

SuperSandro2000 · 2022-03-31T13:21:21Z

is that on purpose? I think you've already made 90% of the work and implementing the missing fields shouldn't be that much effort.

I only added the fields I needed but I can also add the other fields if you want.

theopenlab-ci · 2022-03-31T14:11:44Z

Build failed.

gophercloud-unittest : SUCCESS in 5m 16s
gophercloud-acceptance-test-compute : FAILURE in 7m 35s
gophercloud-acceptance-test-storage : FAILURE in 7m 11s

acceptance/openstack/sharedfilesystems/v2/schedulerstats_test.go

theopenlab-ci · 2022-04-01T10:18:12Z

Build failed.

gophercloud-unittest : SUCCESS in 5m 34s
gophercloud-acceptance-test-compute : FAILURE in 5m 44s
gophercloud-acceptance-test-storage : FAILURE in 6m 15s

SuperSandro2000 · 2022-04-01T10:29:55Z

I am a bit clueless why the CI is failing now. Could someone assist?

mandre

I am a bit clueless why the CI is failing now. Could someone assist?

You had the wrong type for several fields and that caused serialization to fail.

openstack/sharedfilesystems/v2/schedulerstats/results.go

mandre · 2022-04-01T16:15:58Z

openstack/sharedfilesystems/v2/schedulerstats/results.go

+	// The host name for the back end.
+	Host string `json:"host"`
+	// The back end capabilities which include qos, total_capacity_gb, etc.
+	Capabilities Capabilities `json:"capabilities"`


The capabilities are only returned when querying for the pool details. I believe we should have a separate PoolDetails struct.

Other details endpoint don't do it like this and only have one result. I am also not sure how that would even be done because it is always the generic paginator object returned.

…ls}` api endpoints

SuperSandro2000 · 2022-05-09T15:44:28Z

@mandre I finally got around fixing your comments and the tests.

mandre

Thanks @SuperSandro2000 for making the requested changes. This looks good and CI is happy. LGTM.

SuperSandro2000 changed the title ~~Sharedfilesystems schedulerstats services~~ Add support for shared-file-system /v2/services and /v2/scheduler-stats/pools Feb 11, 2022

SuperSandro2000 mentioned this pull request Feb 11, 2022

Plugins capacity manila: convert to gophercloud sapcc/limes#122

Merged

3 tasks

SuperSandro2000 force-pushed the sharedfilesystems-schedulerstats-services branch from 1959017 to 0508ebb Compare February 11, 2022 15:08

SuperSandro2000 force-pushed the sharedfilesystems-schedulerstats-services branch from 6cbe116 to 519e501 Compare March 7, 2022 09:44

mandre reviewed Mar 18, 2022

View reviewed changes

sharedfilesystems: add support for /v2/services api endpoint

54a4862

SuperSandro2000 force-pushed the sharedfilesystems-schedulerstats-services branch from 519e501 to 723ebe5 Compare March 31, 2022 14:02

SuperSandro2000 commented Apr 1, 2022

View reviewed changes

acceptance/openstack/sharedfilesystems/v2/schedulerstats_test.go Outdated Show resolved Hide resolved

SuperSandro2000 force-pushed the sharedfilesystems-schedulerstats-services branch from 723ebe5 to 5721ba0 Compare April 1, 2022 10:08

mandre reviewed Apr 1, 2022

View reviewed changes

sharedfilesystems: Add support for `/v2/scheduler-stats/pools{,/detai…

1edc73e

…ls}` api endpoints

SuperSandro2000 force-pushed the sharedfilesystems-schedulerstats-services branch 2 times, most recently from 7a13241 to 7d0cb7d Compare May 6, 2022 14:10

SuperSandro2000 added 2 commits May 6, 2022 16:50

Add basic acceptance tests for manila/v2/{services,schedulerstats}

420633e

Fix typo

43edb96

SuperSandro2000 force-pushed the sharedfilesystems-schedulerstats-services branch from 7d0cb7d to 43edb96 Compare May 6, 2022 14:50

mandre approved these changes May 10, 2022

View reviewed changes

mandre merged commit 9eb412d into gophercloud:master May 10, 2022

SuperSandro2000 deleted the sharedfilesystems-schedulerstats-services branch May 10, 2022 13:07

Conversation

SuperSandro2000 commented Feb 11, 2022

Uh oh!

coveralls commented Feb 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

theopenlab-ci bot commented Feb 11, 2022

Uh oh!

SuperSandro2000 commented Feb 12, 2022

Uh oh!

majewsky commented Feb 14, 2022

Uh oh!

EmilienM commented Feb 14, 2022

Uh oh!

EmilienM commented Feb 14, 2022

Uh oh!

SuperSandro2000 commented Feb 15, 2022

Uh oh!

EmilienM commented Feb 15, 2022

Uh oh!

SuperSandro2000 commented Feb 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

theopenlab-ci bot commented Feb 24, 2022

Uh oh!

EmilienM commented Feb 25, 2022

Uh oh!

EmilienM commented Feb 27, 2022

Uh oh!

theopenlab-ci bot commented Mar 7, 2022

Uh oh!

mandre left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SuperSandro2000 commented Mar 31, 2022

Uh oh!

theopenlab-ci bot commented Mar 31, 2022

Uh oh!

Uh oh!

theopenlab-ci bot commented Apr 1, 2022

Uh oh!

SuperSandro2000 commented Apr 1, 2022

Uh oh!

mandre left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mandre Apr 1, 2022

Choose a reason for hiding this comment

Uh oh!

SuperSandro2000 May 6, 2022

Choose a reason for hiding this comment

Uh oh!

SuperSandro2000 commented May 9, 2022

Uh oh!

mandre left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

coveralls commented Feb 11, 2022 •

edited

Loading

SuperSandro2000 commented Feb 24, 2022 •

edited

Loading