Skip to content

Filter the types of managed resources that a provider installs #4192

@negz

Description

@negz

What problem are you facing?

Until quite recently some popular Crossplane providers installed support for a lot of managed resources. Providers add support for a new kind of managed resource by:

  • Defining the resource's type and schema using a custom resource definition
  • Starting a controller that knows how to reconcile the new type of managed resource

Providers like upbound/provider-aws installed support for over 900 managed resources. We found this caused a lot of issues for Kubernetes clusters, which were not designed for such a large number of CRDs. We recently addressed this by breaking the largest providers up into families of smaller, more granular providers. You can read about that here:

Initial feedback has been very positive. As best we can tell, everyone who has switched to these new providers is no longer suffering from any CRD-related performance issues.

While working on the CRD-related performance issues we learned that some folks would like even more granular control over what types of managed resource a provider installs support for. Provider families, by convention, scope providers to cloud services. So provider-aws-rds installs support for the 21 managed resources that make up the RDS service. Some folks would like to optimize that further - perhaps to install only the Instance managed resource for example.

The primary reason we've heard for this is for security - for additional defense in depth. In order for someone to successfully use a new kind of Crossplane managed resource, they must:

  1. Install support for the MR using a provider (e.g. install provider-aws-rds for RDS Instance support).
  2. Configure Crossplane with the appropriate cloud credentials to manage the resource (e.g. using a ProviderConfig).
  3. Be granted RBAC access to manipulate the new type of MR, or to a claim or XR that composes it.

It's possible today to avoid granting a Crossplane provider the cloud credentials needed to reconcile types no-one should use, and to avoid granting users RBAC access to ever create an MR. It isn't always possible to avoid installing support for an MR you don't want to use though. For example if you install provider-aws-rds to get Instance support, Crossplane installs support for all the other RDS MRs (e.g. GlobalCluster) whether you intend to use them or not. We've heard that some folks would like to avoid this - to avoid code (and CRDs) that they won't use ever being deployed.

How could Crossplane help solve your problem?

We see two possible ways to do this:

  1. Add support to filter by MR type when installing a provider.
  2. Make it easier to produce custom provider builds that have only the MRs you need.

Which of these options we take depends heavily on how in-demand this feature is. If a lot of people want it it makes sense to make it a configuration knob. If only folks with advanced security needs want it, it may be better to make a custom provider build.

It's worth noting that building filtering is a harder problem than it may seem - this is covered in some detail in the design doc I linked above. It requires support to be added to the package manager and all providers, and that support needs to factor in things like updating the filter over time to add or remove support for particular types of MRs. There have been two POCs of this approach in the past, but neither were complete enough to proceed with:

Metadata

Metadata

Assignees

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions