Skip to content

fix(kep-3633): redesign matchLabelSelectors as matchLabelKeys + mismatchLabelKeys#4069

Merged
k8s-ci-robot merged 5 commits intokubernetes:masterfrom
sanposhiho:matchlabelselector-remaining
Jun 14, 2023
Merged

fix(kep-3633): redesign matchLabelSelectors as matchLabelKeys + mismatchLabelKeys#4069
k8s-ci-robot merged 5 commits intokubernetes:masterfrom
sanposhiho:matchlabelselector-remaining

Conversation

@sanposhiho
Copy link
Copy Markdown
Member

@sanposhiho sanposhiho commented Jun 8, 2023

It's continuous work from #4001. (#4001 is mistakenly merged.)

  • Other comments:

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/kep Categorizes KEP tracking issues and PRs modifying the KEP directory sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. labels Jun 8, 2023
@k8s-ci-robot k8s-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Jun 8, 2023
@sanposhiho
Copy link
Copy Markdown
Member Author

/assign @ahg-g @wojtek-t

It's a continuous one from #4001 which was mistakenly merged.

@sanposhiho
Copy link
Copy Markdown
Member Author

sanposhiho commented Jun 8, 2023

@ahg-g @msau42 What do you think about this idea? I'd like to have our decision on this first.
#4001 (comment)

@sanposhiho sanposhiho force-pushed the matchlabelselector-remaining branch from 8ed5cd7 to 33cc9ee Compare June 8, 2023 14:10
@ahg-g
Copy link
Copy Markdown
Member

ahg-g commented Jun 8, 2023

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 8, 2023
@sanposhiho
Copy link
Copy Markdown
Member Author

@ahg-g Could you check this comment? #4069 (comment)

// The default value is empty.
// +optional
MatchLabelSelectors []strinMatchLabelSelectorg
MatchLabelSelectors []MatchLabelSelector
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Continuing the discussion from:
#4001 (comment)

Ah, yes correct. This PR changes MatchLabelSelectors to be handled in kube-apiserver when Pod is created.
So, existing objects will continue to have the labelselector made by MatchLabelSelectors even after the feature gate is turned off.

What is the ideal action here? OK to leave the existing objects when turning off the feature gate (as long as we state in KEP), or we somehow need to ignore the labelselector made by MatchLabelSelectors (gonna be tricky though..)?

Yes - I'm fine with leaving the existing objects as is [in fact we don't really have good mechanisms to do anything smarter].

Just please extend the Downgrade section with explanation about:

  • leaving the existing objects as is
  • scheduler ignoring this field when FG is disabled (even if set)
  • IIUC, also scheduler depends on the FG, so please update the first question in PRR

Once all of the above is applied, this seems good to go for Alpha (IIUC we're only targeting Alpha now)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, thanks for clarifying.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

scheduler ignoring this field when FG is disabled (even if set)
IIUC, also scheduler depends on the FG, so please update the first question in PRR

Regarding them, No.
The kube-apiserver generates LabelSelector from MatchLabelSelectors and kube-scheduler computes PodAffinity/PodAntiAffinity as it does now. So, kube-scheduler doesn't look at MatchLabelSelectors directly and it doesn't depend on the feature gate. (instead, it just looks at LabelSelector generated from kube-apiserver.)

@wojtek-t
Copy link
Copy Markdown
Member

wojtek-t commented Jun 9, 2023

/hold

To address my comment.

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 9, 2023
@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jun 11, 2023
@sanposhiho
Copy link
Copy Markdown
Member Author

/hold

This discussion is on-going in the original thread. If we don't find any good naming, then OK to unhold.
#4069 (comment)

Copy link
Copy Markdown
Member

@wojtek-t wojtek-t left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few more clarifying comments.

kube-apiserver will ignore `MatchLabelSelectors` in PodAffinity/PodAntiAffinity,
and thus, kube-scheduler will also do nothing with it.

But, we leave the existing `MatchLabelSelectors` and `LabelSelector` created from `MatchLabelSelectors` as it is.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I have an additional question: IIUC, PodAffinity/PodAntiAffinity is immutable, right? So once I create pod, I can no longer change that, right?

If the above is true, we should clarify that:

  • if FG is disabled, MatchLabelSelectors will be dropped for newly created pods and for existing pods
  • if FG is disabled, MatchLabelSelectors and changes to LabelSelector propagated from it will remain as they are for already existing pods

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, right. Sure, let me clarify more on the KEP.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c641305

If FG is disabled,

  • for Pod creation, pods with MatchLabelSelectors will be rejected in validation.
  • for Pod update (= for existing Pods), pods with MatchLabelSelectors will not be rejected, it'll remain as it is.

Does it sound good?

###### What happens if we reenable the feature if it was previously rolled back?

Scheduling of new Pods is affected. (Scheduled Pods aren't affected.)
Scheduling of new Pods created with `MatchLabelSelectors` is affected.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Scheduling of newly created pods with MatchLabelSelector set is affected.
All already existing pods are unafected.

// The default value is empty.
// +optional
MatchLabelSelectors []strinMatchLabelSelectorg
MatchLabelSelectors []MatchLabelSelector
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still keen on using two slice to represent the semantics, to avoid another "selector" field to confuse with existing labelSelector. How about:

type PodAffinityTerm struct {
    matchLabelKeys []string
    mismatchLabelKeys []string

where matchLabelKeys follows the same naming pattern and semantics with existing ones like:

https://github.com/kubernetes/kubernetes/blob/07646db483d0ee8c53e8292fb90b547fccb75eae/staging/src/k8s.io/api/core/v1/types.go#L3709

and mismatchLabelKeys is simply an opposite semantics of matchLabelKeys.

cc @ahg-g per #4001 (comment)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@msau42 what do you think?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having matchLabelKeys and mismatchLabelKeys sounds good to me to match what's in podTopologySpread. We can continue discussing if there's a better field name later.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me; @sanposhiho can you please update the KEP? as @msau42 said, we can debate the names in the implementation PR.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I'll update the KEP soon.

@sanposhiho sanposhiho force-pushed the matchlabelselector-remaining branch from e7782e1 to c641305 Compare June 13, 2023 01:03
@wojtek-t
Copy link
Copy Markdown
Member

/lgtm

/hold cancel

@k8s-ci-robot k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. and removed do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. labels Jun 13, 2023
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed lgtm "Looks good to me", indicates that a PR is ready to be merged. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jun 14, 2023
@sanposhiho
Copy link
Copy Markdown
Member Author

/retitle fix(kep-3633): redesign matchLabelSelectors as matchLabelKeys + mismatchLabelKeys

@k8s-ci-robot k8s-ci-robot changed the title fix(kep-3633): redesign matchlabelselectors fix(kep-3633): redesign matchLabelSelectors as matchLabelKeys + mismatchLabelKeys Jun 14, 2023
@ahg-g
Copy link
Copy Markdown
Member

ahg-g commented Jun 14, 2023

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 14, 2023
@k8s-ci-robot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahg-g, sanposhiho

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 14, 2023
@ahg-g
Copy link
Copy Markdown
Member

ahg-g commented Jun 14, 2023

/hold

in case others have any final comments

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 14, 2023
@wojtek-t
Copy link
Copy Markdown
Member

LGTM

title: Introduce MatchLabelKeys and MismatchLabelKeys to PodAffinity and PodAntiAffinity
kep-number: 3633
authors:
- "@sanposhiho"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you update the kep.yaml with status: implementable here?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I'll follow up in another PR

@Huang-Wei
Copy link
Copy Markdown
Member

LGTM still.

@ahg-g
Copy link
Copy Markdown
Member

ahg-g commented Jun 14, 2023

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 14, 2023
@k8s-ci-robot k8s-ci-robot merged commit 9044007 into kubernetes:master Jun 14, 2023
@k8s-ci-robot k8s-ci-robot added this to the v1.28 milestone Jun 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/kep Categorizes KEP tracking issues and PRs modifying the KEP directory lgtm "Looks good to me", indicates that a PR is ready to be merged. sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants