Skip to content

Deploy internal front-proxy#87

Merged
kcp-ci-bot merged 10 commits intokcp-dev:mainfrom
xrstf:shard-indexing
Sep 4, 2025
Merged

Deploy internal front-proxy#87
kcp-ci-bot merged 10 commits intokcp-dev:mainfrom
xrstf:shard-indexing

Conversation

@xrstf
Copy link
Copy Markdown
Contributor

@xrstf xrstf commented Aug 27, 2025

Summary

We want to provision RBAC rules for kubeconfigs (in #49). For that the idea is to connect to the kubeconfig's target (shard, rootshard or front-proxy) and create the Kube objects that way. This allows the operator to profit from the front-proxy's shard resolving capabilities (so that a Kubeconfig that targets a front-proxy can ask to provision RBAC in any workspace).

However the front-proxy will remove the system:masters group from the authInfo, so when we rely on it, we could not connect through a front-proxy to a shard and still be authorized to do anything.

To solve this, this PR introduces an internal front-proxy. This proxy is purposefully not configured to drop any groups and so will allow the operator to be admin everywhere it needs to be.

This new front-proxy is owned by the RootShard. Similar to the regular FrontProxies, it can be configured (e.g. tolerations and affinities), but on the RootShard object.

There is also a new kcp-operator client certificate that gives the operator system:kcp:admin permissions everywhere.

To prevent too much code duplication, I turned the front-proxy code into a reconciler (a term we use waaaay too much). So now the controllers will not reconcile individual resources anymore, but they will instantiate the frontProxy reconciler and tell it to do its thing. There are still a lot of if proxy else root shard blocks in the resulting code (something that would normally make me split the struct into two), but I saw no better way to deal with this.

Originally we tried to solve this via #86, but found that this approach of "oh, u dropping system:masters because of security? well let us just add system:kcp-operator and use that, ha, take that!" was trying to circumvent a security design decision of kcp.

What Type of PR Is This?

/kind feature

Release Notes

The kcp-operator now deploys an internal front-proxy for each RootShard. This proxy must not be exposed to the internet and is only meant to be used by the kcp-operator itself to provision resources inside workspaces.

@kcp-ci-bot kcp-ci-bot added kind/feature Categorizes issue or PR as related to a new feature. release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. dco-signoff: yes Indicates the PR's author has signed the DCO. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Aug 27, 2025
@xrstf
Copy link
Copy Markdown
Contributor Author

xrstf commented Sep 1, 2025

/retest

@xrstf xrstf changed the title WIP - Shard indexing Deploy internal front-proxy Sep 1, 2025
@kcp-ci-bot kcp-ci-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 1, 2025
@xrstf xrstf requested a review from embik September 1, 2025 13:34
@xrstf
Copy link
Copy Markdown
Contributor Author

xrstf commented Sep 1, 2025

/retest

On-behalf-of: @SAP christoph.mewes@sap.com
On-behalf-of: @SAP christoph.mewes@sap.com
On-behalf-of: @SAP christoph.mewes@sap.com
On-behalf-of: @SAP christoph.mewes@sap.com
On-behalf-of: @SAP christoph.mewes@sap.com
On-behalf-of: @SAP christoph.mewes@sap.com
On-behalf-of: @SAP christoph.mewes@sap.com
…ternal proxy

On-behalf-of: @SAP christoph.mewes@sap.com
On-behalf-of: @SAP christoph.mewes@sap.com
Copy link
Copy Markdown
Member

@embik embik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@kcp-ci-bot kcp-ci-bot added the lgtm Indicates that a PR is ready to be merged. label Sep 4, 2025
@kcp-ci-bot
Copy link
Copy Markdown
Contributor

LGTM label has been added.

DetailsGit tree hash: f22d31d66d3455299b48cf3aab2cae27cdbc2011

@kcp-ci-bot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: embik

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kcp-ci-bot kcp-ci-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 4, 2025
@kcp-ci-bot kcp-ci-bot merged commit 93c4948 into kcp-dev:main Sep 4, 2025
10 checks passed
@xrstf xrstf deleted the shard-indexing branch September 4, 2025 13:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates the PR's author has signed the DCO. kind/feature Categorizes issue or PR as related to a new feature. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants