Skip to content

Add Cinder to PersistentVolumeLabel Admission Controller#73102

Merged
k8s-ci-robot merged 4 commits intokubernetes:masterfrom
andrewsykim:add-openstack-pvl-admission
Jan 24, 2019
Merged

Add Cinder to PersistentVolumeLabel Admission Controller#73102
k8s-ci-robot merged 4 commits intokubernetes:masterfrom
andrewsykim:add-openstack-pvl-admission

Conversation

@andrewsykim
Copy link
Copy Markdown
Member

@andrewsykim andrewsykim commented Jan 19, 2019

What type of PR is this?
/kind feature

What this PR does / why we need it:
Despite the fact that the PersistentVolumeLabel admission controller is deprecated, we need to add support for the Cinder volume plugin here given the out-of-tree equivalent relies on Initializers that are being removed (see #72972). We're going to need a longer-term solution for this that is out-of-tree and does not depend on Initializers but until then, labeling PVs for Cinder will have to be done via the admission controller. Note that there is a good chance that users are not even using this feature because it would have required them to enable the alpha Initializers feature. If users have been using this feature, they will have to use the PersistentVolumeLabel admission controller instead, the tradeoff is that we'll be able to remove usages of Initializers.

Also refactors the admission controller to:

  • call cloudprovider.PVLabeler for consistency. Right now we have the labeling logic split in two places (the admission controller and the cloud providers).
  • use test tables for unit tests (also includes tests for other providers)

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

If you are running the cloud-controller-manager and you have the `pvlabel.kubernetes.io` alpha Initializer enabled, you must now enable PersistentVolume labeling using the `PersistentVolumeLabel` admission controller instead. You can do this by adding `PersistentVolumeLabel` in the `--enable-admission-plugins` kube-apiserver flag. 

/sig openstack
/sig cloud-provider

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Jan 19, 2019
@andrewsykim andrewsykim changed the title Add Cinder PersistentVolumeLabel Admission Controller Add Cinder to PersistentVolumeLabel Admission Controller Jan 19, 2019
@k8s-ci-robot k8s-ci-robot added area/provider/openstack Issues or PRs related to openstack provider sig/cloud-provider Categorizes an issue or PR as relevant to SIG Cloud Provider. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jan 19, 2019
@andrewsykim
Copy link
Copy Markdown
Member Author

/assign @liggitt @dims

@andrewsykim andrewsykim force-pushed the add-openstack-pvl-admission branch from ac73063 to c3dcf86 Compare January 19, 2019 06:52
@dims
Copy link
Copy Markdown
Member

dims commented Jan 19, 2019

I like this trade-off @andrewsykim will look deeper when i get a chance. thanks!

@dims
Copy link
Copy Markdown
Member

dims commented Jan 19, 2019

@andrewsykim

  • Let us please split this into 2 commits, one that switches the impl to PVLabeler and another that adds the cinder support. That will help evaluate the PVLabeler changes independent of the cinder thing
  • Can we please add the cinder stuff to the test case as well.

thanks!

@andrewsykim andrewsykim force-pushed the add-openstack-pvl-admission branch 2 times, most recently from 4457e3b to 3dac8d3 Compare January 22, 2019 00:09
@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jan 22, 2019
@andrewsykim
Copy link
Copy Markdown
Member Author

@dims comments addressed, PTAL! I ended up refactoring the unit tests a bit to account for other cloud providers.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might want to review this file in split mode

@dims
Copy link
Copy Markdown
Member

dims commented Jan 22, 2019

@andrewsykim getAzurePVLabeler / getGCEPVLabeler / getAWSPVLabeler / getOpenStackPVLabeler are all exactly the same except for the cloud type parameter. no?

@andrewsykim
Copy link
Copy Markdown
Member Author

It also caches the pvlabeler (by assigning it to a field).

@andrewsykim
Copy link
Copy Markdown
Member Author

I think it would be cleaner to have 1 field (pvlabler) as opposed to one per cloud provider but that assumes that the admission controller only admits PVs from 1 provider. I think in most clusters this is true but it seems there is an assumption before this patch that the apiserver can admit PVs from any provider.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will panic if the interface is not implemented. seems like we want an error, as before

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do a conditional interface assertion so we can return an error instead of panicking?

@k8s-ci-robot k8s-ci-robot added the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label Jan 23, 2019
@andrewsykim andrewsykim force-pushed the add-openstack-pvl-admission branch from 24c7acc to 47871cb Compare January 23, 2019 17:44
@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Jan 23, 2019
@andrewsykim andrewsykim force-pushed the add-openstack-pvl-admission branch from 47871cb to ee1e9cf Compare January 23, 2019 19:38
@andrewsykim
Copy link
Copy Markdown
Member Author

@dims can I get another review please?

@dims
Copy link
Copy Markdown
Member

dims commented Jan 24, 2019

/lgtm

i am ok to leave the multiple pvlabler fields for now. thanks!

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 24, 2019
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can leave me off of this owners file, and there should probably be a storage reviewer, right?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense! @msau42 @saad-ali can I add one of you here? (Or recommend someone who can) :)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed! @msau42 agreed to be in the OWNERS file :)

@andrewsykim andrewsykim force-pushed the add-openstack-pvl-admission branch from ee1e9cf to 070f4fd Compare January 24, 2019 18:31
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 24, 2019
@andrewsykim andrewsykim force-pushed the add-openstack-pvl-admission branch from 070f4fd to 467a3e5 Compare January 24, 2019 18:32
@liggitt
Copy link
Copy Markdown
Member

liggitt commented Jan 24, 2019

/retest
/approve

@k8s-ci-robot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andrewsykim, liggitt

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 24, 2019
@dims
Copy link
Copy Markdown
Member

dims commented Jan 24, 2019

/lgtm

(re-applying)

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 24, 2019
@k8s-ci-robot k8s-ci-robot merged commit 5fc286f into kubernetes:master Jan 24, 2019
rfranzke added a commit to rfranzke/gardener that referenced this pull request Mar 20, 2019
Due to kubernetes/kubernetes#72972 the alpha `Initializers` feature will be entirely removed with Kubernetes 1.14. We are currently relying on it as the cloud-controller-manager uses it to label persistent volumes before they become visible in the API. This functionality is also removed (see kubernetes/kubernetes#73102) with 1.14. As a consequence, in order to use PV labelling until the community has migrated all CCMs to use `MutatingWebhookConfiguration`s for that, the only way is to enable the (already deprecated) `PersistentVolumeLabel` admission controller in the kube-apiserver. That means that the kube-apiserver needs to talk to the cloud provider to figure out the region/zone of the PV (and later add them as label).
--> We have to provide the cloud credentials + config to the API server in this case.

Shoot clusters with Kubernetes version < 1.14 still have an enabled initializer feature.
richardyuwen pushed a commit to richardyuwen/gardener that referenced this pull request Mar 26, 2019
Due to kubernetes/kubernetes#72972 the alpha `Initializers` feature will be entirely removed with Kubernetes 1.14. We are currently relying on it as the cloud-controller-manager uses it to label persistent volumes before they become visible in the API. This functionality is also removed (see kubernetes/kubernetes#73102) with 1.14. As a consequence, in order to use PV labelling until the community has migrated all CCMs to use `MutatingWebhookConfiguration`s for that, the only way is to enable the (already deprecated) `PersistentVolumeLabel` admission controller in the kube-apiserver. That means that the kube-apiserver needs to talk to the cloud provider to figure out the region/zone of the PV (and later add them as label).
--> We have to provide the cloud credentials + config to the API server in this case.

Shoot clusters with Kubernetes version < 1.14 still have an enabled initializer feature.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. area/provider/openstack Issues or PRs related to openstack provider cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/cloud-provider Categorizes an issue or PR as relevant to SIG Cloud Provider. sig/storage Categorizes an issue or PR as relevant to SIG Storage. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants