feat: enable controlled CR reconciliation by xiaoweim · Pull Request #6243 · GoogleCloudPlatform/k8s-config-connector

xiaoweim · 2026-01-22T21:11:02Z

BRIEF Change description

Implement Resource Config to allow unmanaging specific resources by the Config Connector controller.

WHY do we need this change?

This feature allows users and platform administrators to selectively disable the reconciliation of specific resources (Group/Kind).

In Cluster Mode, resources can be unmanaged globally via the ConfigConnector object.
In Namespace Mode, resources can be disabled per-namespace via the
ConfigConnectorContext object. This takes precedence over global ConfigConnector settings.

This is useful for:

Disabling creating controllers for unused resources to save memory.

Special notes for your reviewer:

Logic is implemented in pkg/controller/registration/registration_controller.go to check isResourceDisabled before starting a controller.
ScopedNamespace was added to kccmanager configuration to allow namespace-scoped managers to identify and prioritize their relevant ConfigConnectorContext. The controller evaluates both ConfigConnector and ConfigConnectorContext to determine if a resource should be excluded, prioritizing
ConfigConnectorContext.
The feature supports unmanaging resources only. enabled: true entries are ignored and logged as warnings.
A new integration test TestResourceExclusion in pkg/controller/kccmanager/kccmanager_exclusion_test.go verifies end-to-end behavior for both cluster and namespace modes.

Since registration only happens at manager startup, when we update the CCC/CC, we need to restart the pod to pick up the change.

Does this PR add something which needs to be 'release noted'?

Add support for unamanging specific resources via `resourceSettings` in `ConfigConnector` (global) and `ConfigConnectorContext` (per-namespace).

Reviewer reviewed release note.

Additional documentation e.g., references, usage docs, etc.:

Intended Milestone

Please indicate the intended milestone.

Reviewer tagged PR with the actual milestone.

Tests you have done

Testing Performed

Successfully built and deployed KCC to a GKE test cluster (test-partial-crd) in Namespaced mode. Conducted manual verification of the new ResourceSettings conflict and inclusion/exclusion logic:

1. Mixed Modes Conflict Rejection

Configured a global ConfigConnector in Exclusive Mode (enabled: false), against a local ConfigConnectorContext in Inclusive Mode (enabled: true).

kubectl apply -f - <<EOF
apiVersion: core.cnrm.cloud.google.com/v1beta1
kind: ConfigConnector
metadata:
  name: configconnector.core.cnrm.cloud.google.com
spec:
  mode: namespaced
  experiments:
    resourceSettings:
      enabled: false
      resources:
      - group: "pubsub.cnrm.cloud.google.com"
        kind: "PubSubTopic"
---
apiVersion: core.cnrm.cloud.google.com/v1beta1
kind: ConfigConnectorContext
metadata:
  name: configconnectorcontext.core.cnrm.cloud.google.com
  namespace: test1
spec:
  googleServiceAccount: "test-kcc@xiaoweim-gke-dev.iam.gserviceaccount.com"
  experiments:
    resourceSettings:
      enabled: true
      resources:
      - group: "storage.cnrm.cloud.google.com"
        kind: "StorageBucket"
EOF

Result: The Operator correctly flags the conflict as an error and aggressively blocks the StatefulSet creation for that namespace.

$ kubectl get configconnectorcontext -n test1 -o yaml
...
  status:
    errors:
    - 'error during reconciliation: conflict: ConfigConnector and ConfigConnectorContext
      cannot mix inclusive (enabled: true) and exclusive (enabled: false) modes'
    healthy: false

$ kubectl get statefulset -n cnrm-system
NAME                      READY   AGE
cnrm-deletiondefender     1/1     2m43s
cnrm-unmanaged-detector   1/1     2m43s
# (No cnrm-controller-manager deployed)

2. Exclusion List Verification (Deny-list)

Resolved the conflict by setting both to Exclusive Mode (enabled: false).

kubectl apply -f - <<EOF
apiVersion: core.cnrm.cloud.google.com/v1beta1
kind: ConfigConnectorContext
metadata:
  name: configconnectorcontext.core.cnrm.cloud.google.com
  namespace: test1
spec:
  googleServiceAccount: "test-kcc@xiaoweim-gke-dev.iam.gserviceaccount.com"
  experiments:
    resourceSettings:
      enabled: false
      resources:
      - group: "storage.cnrm.cloud.google.com"
        kind: "StorageBucket"
EOF

Result: The Operator clears the error and deploys the manager. Verified the StorageBucket and PubSubTopic controllers are aggressively excluded from memory logs:

$ kubectl get pods -n cnrm-system
NAME                                        READY   STATUS    RESTARTS   AGE
cnrm-controller-manager-44kdpwa-0           2/2     Running   0          9s

$ kubectl logs -l cnrm.cloud.google.com/component=cnrm-controller-manager -n cnrm-system -c manager | grep "Skipping controller registration"
{"severity":"info","timestamp":"2026-03-17T22:21:41.953Z","msg":"Skipping controller registration because resource is disabled in ConfigConnector or ConfigConnectorContext","group":"storage.cnrm.cloud.google.com","version":"v1beta1","kind":"StorageBucket"}
{"severity":"info","timestamp":"2026-03-17T22:21:41.954Z","msg":"Skipping controller registration because resource is disabled in ConfigConnector or ConfigConnectorContext","group":"pubsub.cnrm.cloud.google.com","version":"v1beta1","kind":"PubSubTopic"}

$ kubectl logs -l cnrm.cloud.google.com/component=cnrm-controller-manager -n cnrm-system -c manager | grep "Starting Controller" | wc -l
174

3. Inclusion List Verification (Allow-list)

Inverted configurations to an Inclusive Mode (enabled: true) across both the ConfigConnector and ConfigConnectorContext.

kubectl apply -f - <<EOF
apiVersion: core.cnrm.cloud.google.com/v1beta1
kind: ConfigConnector
metadata:
  name: configconnector.core.cnrm.cloud.google.com
spec:
  mode: namespaced
  experiments:
    resourceSettings:
      enabled: true
      resources:
      - group: "pubsub.cnrm.cloud.google.com"
        kind: "PubSubTopic"
---
apiVersion: core.cnrm.cloud.google.com/v1beta1
kind: ConfigConnectorContext
metadata:
  name: configconnectorcontext.core.cnrm.cloud.google.com
  namespace: test1
spec:
  googleServiceAccount: "test-kcc@xiaoweim-gke-dev.iam.gserviceaccount.com"
  experiments:
    resourceSettings:
      enabled: true
      resources:
      - group: "storage.cnrm.cloud.google.com"
        kind: "StorageBucket"
EOF

Result: Forced a pod restart to evaluate the new environment. The Pod cleanly abandons the other 170+ parent controllers, radically pruning the controller process loop to explicitly map only the whitelist elements into memory.

$ kubectl delete pod cnrm-controller-manager-44kdpwa-0 -n cnrm-system
pod "cnrm-controller-manager-44kdpwa-0" deleted

$ kubectl logs -l cnrm.cloud.google.com/component=cnrm-controller-manager -n cnrm-system -c manager | grep "Starting Controller"
{"severity":"info","timestamp":"2026-03-17T22:28:11.953Z","msg":"Starting Controller","controller":"storagebucket-parent-controller","controllerGroup":"storage.cnrm.cloud.google.com","controllerKind":"StorageBucket"}
{"severity":"info","timestamp":"2026-03-17T22:28:11.954Z","msg":"Starting Controller","controller":"pubsubtopic-parent-controller","controllerGroup":"pubsub.cnrm.cloud.google.com","controllerKind":"PubSubTopic"}

$ kubectl logs -l cnrm.cloud.google.com/component=cnrm-controller-manager -n cnrm-system -c manager | grep "Skipping controller registration" | wc -l
174

Run make ready-pr to ensure this PR is ready for review.
Perform necessary E2E testing for changed resources.

justinsb · 2026-01-22T22:02:22Z

ScopedNamespace was added to kccmanager configuration to ensure namespace-scoped managers only look at their relevant ConfigConnectorContext.

I'm not sure about this. If both ConfigConnector and ConfigConnectorContext support resourceSettings, then I would expect that both would be honored, rather than ignoring one or the other. My assumption would be that we would look first at the CCC (when in namespace mode), and then at the CC.

Am I missing a reason to ignore the CC in namespace mode?

xiaoweim · 2026-01-22T22:12:04Z

ScopedNamespace was added to kccmanager configuration to ensure namespace-scoped managers only look at their relevant ConfigConnectorContext.

I'm not sure about this. If both ConfigConnector and ConfigConnectorContext support resourceSettings, then I would expect that both would be honored, rather than ignoring one or the other. My assumption would be that we would look first at the CCC (when in namespace mode), and then at the CC.

Am I missing a reason to ignore the CC in namespace mode?

Yes, that is the intended behavior I want to implement My assumption would be that we would look first at the CCC (when in namespace mode), and then at the CC. I am have not pushed the code changes for that yet 🤣

xiaoweim · 2026-01-23T01:28:28Z

+}
+
+func getResourceSetting(settings []operatorv1beta1.ResourceSetting, gvk schema.GroupVersionKind) (found bool, enabled bool) {
+	for _, s := range settings {


As per @acpana 's suggestion, we could make resourceSetting a map of [schema.gk]properties, and we could keep adding more resource specific configuration values to the properties struct, just make sure that they are mutually exclusive with enabled:false. This also helps with efficiency as we do not need to iterate through the list here.

@justinsb @cheftako WDYT

xiaoweim · 2026-01-30T03:32:40Z

/assign @cheftako @justinsb

xiaoweim · 2026-03-12T22:01:42Z

ScopedNamespace was added to kccmanager configuration to ensure namespace-scoped managers only look at their relevant ConfigConnectorContext.

I'm not sure about this. If both ConfigConnector and ConfigConnectorContext support resourceSettings, then I would expect that both would be honored, rather than ignoring one or the other. My assumption would be that we would look first at the CCC (when in namespace mode), and then at the CC.

Am I missing a reason to ignore the CC in namespace mode?

No I agree, I have updated the implementation to honor both CC and CCC. with CCC taking precedence in namespace mode.

xiaoweim · 2026-03-18T20:23:43Z

/assign @cheftako

cheftako · 2026-04-01T23:39:18Z

+type ResourceFilter struct {
+	// Group is the API group of the resource.
+	// +required
+	Group string `json:"group"`


Should be a pointer in case we ever want to make it optional. I believe we have been making the use of pointer to native the standard in our types.go files recently.

cheftako · 2026-04-01T23:39:28Z

+
+	// Kind is the Kind of the resource.
+	// +optional
+	Kind string `json:"kind,omitempty"`


…tion

…ts, allow transition states with warnings

…ed bool

cheftako · 2026-04-03T17:49:11Z

+
+type ResourceSettings struct {
+	// Mode controls whether the resources are included or excluded.
+	// Defaults to "exclude" (Exclusion mode).


Should indicate all allowed values as well as the default.

google-oss-prow · 2026-04-03T17:52:12Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cheftako

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [cheftako]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

This PR fixes a bug in the ResourceSettings mode conflict logic introduced in #6243. The previous logic would log a warning and skip the conflict check if both ConfigConnector and ConfigConnectorContext were present but had different modes. This caused TestResourceExclusion/ConflictMode to fail as it expected an error. This PR ensures that the conflict check is performed first if both settings are present. Fixes #7325 Generated by Overseer (powered by the gemini-3-flash-preview model).

google-oss-prow Bot added the do-not-merge/work-in-progress label Jan 22, 2026

xiaoweim added this to the 1.144 milestone Jan 22, 2026

xiaoweim force-pushed the partial-crd branch from 1057cb0 to d8336a9 Compare January 22, 2026 21:55

xiaoweim force-pushed the partial-crd branch 2 times, most recently from a0e8c37 to 122eb05 Compare January 23, 2026 00:02

xiaoweim marked this pull request as ready for review January 23, 2026 00:17

google-oss-prow Bot removed the do-not-merge/work-in-progress label Jan 23, 2026

xiaoweim force-pushed the partial-crd branch from 122eb05 to f0fedc6 Compare January 23, 2026 01:25

xiaoweim commented Jan 23, 2026

View reviewed changes

xiaoweim force-pushed the partial-crd branch from f0fedc6 to a5403b1 Compare January 27, 2026 23:16

xiaoweim modified the milestones: 1.144, 1.145 Jan 28, 2026

xiaoweim force-pushed the partial-crd branch from 2d568b5 to fbdebd4 Compare January 30, 2026 03:32

google-oss-prow Bot assigned cheftako and justinsb Jan 30, 2026

xiaoweim modified the milestones: 1.145, 1.146 Feb 17, 2026

xiaoweim modified the milestones: 1.146, 1.147 Mar 3, 2026

xiaoweim force-pushed the partial-crd branch 2 times, most recently from fef42a4 to 12a9112 Compare March 12, 2026 21:54

xiaoweim force-pushed the partial-crd branch 4 times, most recently from 9d26e5b to 586dd3f Compare March 18, 2026 17:39

xiaoweim force-pushed the partial-crd branch from 087a79f to 4a18b0a Compare April 1, 2026 21:05

cheftako reviewed Apr 1, 2026

View reviewed changes

xiaoweim added 12 commits April 2, 2026 22:12

enable controlled CR reconciliation

89d5c6f

feat: implement mode conflict validation for Controlled CR Reconcilia…

ef9386f

…tion

docs: add Controlled CR Reconciliation feature doc

fbf3382

style: run make fmt

fa8e857

fix: ensure skipped controller logs use manager context logger

9b7cdc0

style: run make fmt

1492f10

Refining ResourceSettings mode validation UX: reject explicit conflic…

70cd038

…ts, allow transition states with warnings

Ran make fmt

39f0ef7

fix: nil pointer dereference when Experiments is nil

33f45a0

Address review comments

fcfa3f2

refactor: address cheftako feedback to use Mode enum instead of Enabl…

ca00f35

…ed bool

refactor: make ResourceFilter Group and Kind pointers

31ca572

xiaoweim force-pushed the partial-crd branch from a4adc2e to 31ca572 Compare April 2, 2026 22:12

cheftako reviewed Apr 3, 2026

View reviewed changes

cheftako approved these changes Apr 3, 2026

View reviewed changes

cheftako added this pull request to the merge queue Apr 3, 2026

google-oss-prow Bot added the lgtm label Apr 3, 2026

google-oss-prow Bot added the approved label Apr 3, 2026

github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Apr 3, 2026

xiaoweim added this pull request to the merge queue Apr 3, 2026

github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Apr 3, 2026

xiaoweim added this pull request to the merge queue Apr 3, 2026

Merged via the queue into GoogleCloudPlatform:master with commit 6b3ff9b Apr 3, 2026
166 checks passed

codebot-robot mentioned this pull request Apr 7, 2026

fix: fix ResourceSettings mode conflict logic #7326

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: enable controlled CR reconciliation#6243

feat: enable controlled CR reconciliation#6243
xiaoweim merged 12 commits into
GoogleCloudPlatform:masterfrom
xiaoweim:partial-crd

xiaoweim commented Jan 22, 2026 •

edited

Loading

Uh oh!

justinsb commented Jan 22, 2026

Uh oh!

xiaoweim commented Jan 22, 2026

Uh oh!

xiaoweim Jan 23, 2026

Uh oh!

xiaoweim Jan 23, 2026

Uh oh!

xiaoweim commented Jan 30, 2026

Uh oh!

xiaoweim commented Mar 12, 2026

Uh oh!

xiaoweim commented Mar 18, 2026

Uh oh!

cheftako Apr 1, 2026

Uh oh!

cheftako Apr 1, 2026

Uh oh!

cheftako Apr 3, 2026

Uh oh!

google-oss-prow Bot commented Apr 3, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

xiaoweim commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

BRIEF Change description

WHY do we need this change?

Special notes for your reviewer:

Does this PR add something which needs to be 'release noted'?

Additional documentation e.g., references, usage docs, etc.:

Intended Milestone

Tests you have done

Testing Performed

1. Mixed Modes Conflict Rejection

2. Exclusion List Verification (Deny-list)

3. Inclusion List Verification (Allow-list)

Uh oh!

justinsb commented Jan 22, 2026

Uh oh!

xiaoweim commented Jan 22, 2026

Uh oh!

xiaoweim Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

xiaoweim Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

xiaoweim commented Jan 30, 2026

Uh oh!

xiaoweim commented Mar 12, 2026

Uh oh!

xiaoweim commented Mar 18, 2026

Uh oh!

cheftako Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

cheftako Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

cheftako Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

google-oss-prow Bot commented Apr 3, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

xiaoweim commented Jan 22, 2026 •

edited

Loading