Skip to content

Workaround obsolete API versions#7575

Closed
pho-enix wants to merge 2 commits intohelm:masterfrom
pho-enix:k8sAPIVersionMitigation
Closed

Workaround obsolete API versions#7575
pho-enix wants to merge 2 commits intohelm:masterfrom
pho-enix:k8sAPIVersionMitigation

Conversation

@pho-enix
Copy link
Copy Markdown
Contributor

This is a workaround for changes in API versions like
extensions/v1beta1 => apps/v1

From Helm's perspective those are new Resources as the API version
cannot be changed. However there is a compat layer in k8s that will
return the old resources for the new API version. Helm will reject the
resource 'creation' as there is already a resource of the same kind and
version.

Do we want to maintain compat code in this place? Probably not.

@helm-bot helm-bot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Feb 12, 2020
@mattfarina
Copy link
Copy Markdown
Collaborator

@pho-enix thanks for the PR. Helm uses a developers certificate of origin (DCO) to show you are allowed to contribute said material. The checks tab at the top will show you how to correct the commit. In new commits you can use the --signoff flag.

I'm unsure if Helm should transform api versions. For one, they don't always translate from one to another as some subtle changes happened along the way. The error response coming back may not make sense.

I've added this as a discussion topic to the next Helm dev call so we can talk about it. You are welcome to attend if you like. If not, I'm prepared to talk about it.

groupVersion := gvk.GroupVersion().String()

if groupVersion == "apps/v1beta1" || groupVersion == "apps/v1beta2" {
groupVersion = "apps/v1"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't you just check if old_api_version != new_api_version instead?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. Thanks for the hint. However we have to be a bit cautious about the Group. There might be the same Kind in different Groups.

For example:
NetworkPolicy - networking.k8s.io/v1
NetworkPolicy - crd.projectcalico.org/v1

In principle anyone could create a Deplyoment Kind in his CRD.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough, but then you could check instead if only bit after "/" has been changed (for the same api type). That would be a more future proof solution - if there will be "apps/v2" version released in the future you'll have to rewrite this

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this will not be sufficient. As for example Deployment moved
extensions/v1beta1 -> apps/v1

Please check the updated code :)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I see. Updated code looks cool tho +1

@helm-bot helm-bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Feb 13, 2020
This is a workaround for changes in API versions like
extensions/v1beta1 => apps/v1

Problem:
From Helm's perspective those are new Resources as the API version
cannot be changed. However there is a compat layer in k8s that will
return the old resources for the new API version. Helm will reject the
resource 'creation' as there is already a resource of the same kind and
version.

Solution:
In general ignore the Version.
Due to the fact that there might be clashes of Kind we track CRD and
none-CRD resources seperate.

Open issue:
Corner case with same Kind in different CRDs and a pre-existing resource
that would clash with the one created by Helm.

Signed-off-by: Sebastian Poehn <sebastian.poehn@gmail.com>
@pho-enix
Copy link
Copy Markdown
Contributor Author

Issue 1:
Clashes of Kubernetes Resources and CRD with same Kind might produce strange errors (if a Resource with the same name already exists)
Issue 2:
Clashes of CRD and CRD with same Kind might produce strange errors (if a Resource with the same name already exists)

I general we'd have to decide how many corner cases we want to take into account:

Option 1) Simply ignore Version and Group during upgrade.

  • Code change will be super small
  • Issue 1 fixed
  • Issue 2 fixed

Option 2) Current PR (ed739d2)

  • Quite a bit of code
  • Issue 1 fixed
  • Issue 2 fixed

Signed-off-by: Sebastian Poehn <sebastian.poehn@gmail.com>
@jlegrone
Copy link
Copy Markdown
Member

I think there might be a simpler solution to this. Opening a PR for discussion in a little bit and I'll link it here. Thanks for your work on this so far @pho-enix!

return r.Object.GetObjectKind().GroupVersionKind().Kind
}

func isCRD(groupVersion string) bool {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here is how Kubernetes recommends checking whether a resource is a CRD or not:

helm/pkg/kube/client.go

Lines 372 to 373 in 60d8d72

// On newer K8s versions, CRDs aren't unstructured but has this dedicated type
_, isCRD := versionedObject.(*apiextv1beta1.CustomResourceDefinition)

We should probably update that code to check if versionedObject can be casted to a *apiextv1.CustomResourceDefinition as well, now that it's available in 1.17.... Regardless, that would be a more portable option than string parsing.

@pho-enix
Copy link
Copy Markdown
Contributor Author

@jlegrone Agreed that my solution is a bit complex. As this issue is really a blocker for a lot of users we should aim for a fast solution (#7627).

@adnan-kamili
Copy link
Copy Markdown

Maybe this behavior can be enabled through a command-line option so that it doesn't break anything and users who know what they are doing use this command-line option.

@avielb
Copy link
Copy Markdown

avielb commented Mar 8, 2020

Waiting for this as well
could be very useful

@bacongobbler
Copy link
Copy Markdown
Member

closed in favour of #7649.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants