Skip to content

Suppress "object has been modified" warning events? #2114

@negz

Description

@negz

What problem are you facing?

Crossplane relies heavily on Kubernetes events to help folks debug what it is doing. It's quite common for Crossplane to emit the following warning event:

Warning  ConfigureCompositeResource  19s (x2 over 20s)  offered/compositeresourcedefinition.apiextensions.crossplane.io  cannot patch object: Operation cannot be fulfilled on compositenetworks.gcp.platformref.crossplane.io "test-network-d6xjh": the object has been modified; please apply your changes to the latest version and try again

We emit this event when two controllers race to update a resource. Kubernetes uses a form of optimistic concurrency in which each resource is versioned (via metadata.resourceVersion), and clients must have the latest version of the resource when attempting to update it. This means that if two clients get the same version of a resource then race to update it, the one that loses will encounter this error from the API server.

This is almost never a cause for concern; Crossplane will just try again. The event does make it appear like something is awry though, and is frequently a red herring for Crossplane users trying to debug unrelated issues.

How could Crossplane help solve your problem?

Crossplane could detect this I think it manifests as a 409 Conflict, so we could detect it using a resource.IsConfict(err) helper and either completely suppress it or emit it as a Normal event.

In the meantime, this would be a good thing to add to an FAQ.

Metadata

Metadata

Assignees

Type

No type

Projects

Status

Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions