network driver: manage node config based on cluster configs and label matching#44761
Draft
bersoare wants to merge 53 commits intocilium:feature/dra-driverfrom
Draft
network driver: manage node config based on cluster configs and label matching#44761bersoare wants to merge 53 commits intocilium:feature/dra-driverfrom
bersoare wants to merge 53 commits intocilium:feature/dra-driverfrom
Conversation
Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
network-driver is a new feature introduced that enables exposing unmanaged network interfaces to pods. adding sr-iov interface support and basic netdev configuration inside the pod Signed-off-by: Bernardo Soares <20172413+bersoare@users.noreply.github.com> Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
Add back missing volume mount for /var/run/nri. Without this the plugin is unable to connect to the NRI socket and fails with the following error: ``` msg="NRI plugin failed" module=agent.controlplane.network-driver error="failed to connect to NRI service: dial unix /var/run/nri/nri.sock: connect: no such file or directory" name=dummy.cilium.k8s.io ``` Fixes: e663e3c ("network-driver: add new network-driver feature pkg") Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
If the start of the NRI plugin fails, wait a second before retrying to restart it. This avoid consuming too much CPU in case of a persistent failure. Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
In commit e663e3c the network driver has been reworked to add a modular structure. Add a module to support dummy devices and an example configuration to start the dumme devices support. Related: e663e3c ("network-driver: add new network-driver feature pkg") Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
as part of allowing the Network Driver to be configured by an operator, add the CRD spec so we can have configurations deployed to the cluster adding the CRD definitions, as well as the generated code Signed-off-by: Bernardo Soares <20172413+bersoare@users.noreply.github.com> Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
collect the CRD for the network driver configuration with the sysdump command Signed-off-by: Bernardo Soares <20172413+bersoare@users.noreply.github.com> Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
NRI and DRA frameworks are not available in older kube versions, so perform a version check for mounting the paths. Signed-off-by: Bernardo Soares <20172413+bersoare@users.noreply.github.com> Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
will likely fail for older versions. ensure we're in at least kube v1.34 Signed-off-by: Bernardo Soares <20172413+bersoare@users.noreply.github.com> Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
maps make it hard to change the json tag for the keys, and using an array makes the CRD looks cleaner as we can decouple the field name from the json tag without having to implement a custom marshaler. also adding some default values that were missing Signed-off-by: Bernardo Soares <20172413+bersoare@users.noreply.github.com> Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
no longer use the config types we were using previously, since now we want to get the config from the crd Signed-off-by: Bernardo Soares <20172413+bersoare@users.noreply.github.com> Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
adjusting the current config validation to validate the struct from the CRD removed the config types the code was previously using Signed-off-by: Bernardo Soares <20172413+bersoare@users.noreply.github.com> Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
get and use configuration from the CRD Signed-off-by: Bernardo Soares <20172413+bersoare@users.noreply.github.com> Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
Signed-off-by: Bernardo Soares <20172413+bersoare@users.noreply.github.com> Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
Signed-off-by: Bernardo Soares <20172413+bersoare@users.noreply.github.com> Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
agent now waits for a config at any point in time and starts the network driver if one ever arrives. also encapsulated the containerd and dra initialization logic to make the start sequence more readable. no functional changes were made updates to configuration aren't handled yet. Signed-off-by: Bernardo Soares <20172413+bersoare@users.noreply.github.com> Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
continue reading from the channel even after the first configuration is received, and log any further update attempts Signed-off-by: Bernardo Soares <20172413+bersoare@users.noreply.github.com> Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
check for an empty config before accessing the config definitions (and avoid a nil pointer access) Signed-off-by: Bernardo Soares <20172413+bersoare@users.noreply.github.com> Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
In case of an agent restart without a change to the CiliumNetworkDriverConfig we are going to receive a Sync after an Upsert for the resource. In that case, we should read the last available config version and enable the driver accordingly. Conversely, if a new version of the CiliumNetworkDriverConfig is upserted after the initial sync, we should handle that version. The commit updates the loop reading the events to correctly handle both scenarios (upsert + sync and sync + upsert) while discarding further attempts of updating the config. Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
with this we can write tests more easily without the need of an sr-iov capable nic Signed-off-by: Bernardo Soares <20172413+bersoare@users.noreply.github.com> Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
using netlink attributes to find interface name and pf name. some error handling improvements Signed-off-by: Bernardo Soares <20172413+bersoare@users.noreply.github.com> Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
added some more test coverage, now ensuring that the attributes match the expectation Signed-off-by: Bernardo Soares <20172413+bersoare@users.noreply.github.com> Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
this is so we can access and read/write device sr-iov attributes Signed-off-by: Bernardo Soares <20172413+bersoare@users.noreply.github.com> Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
now using a single sysfs path to simplify the mount structure small refactor to encapsulate the file reading/writing logic added some additional unit tests to cover sriov vf setup Signed-off-by: Bernardo Soares <20172413+bersoare@users.noreply.github.com> Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
sysfs mock in the sr-iov package has circular links, and grep complains about it. this commit excludes the testdata directory from the grep command Signed-off-by: Bernardo Soares <20172413+bersoare@users.noreply.github.com> Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
adding more script test cases Signed-off-by: Bernardo Soares <bersoare@isovalent.com>
c0f4311 to
c624836
Compare
Contributor
Author
|
/test |
2 similar comments
Contributor
Author
|
/test |
Contributor
Author
|
/test |
Contributor
Author
|
failing tests are all failing on |
Contributor
Author
christarazi
approved these changes
Mar 19, 2026
Member
|
Isn't this a duplicate of #44501? |
Contributor
Author
|
wasn't aware of that one 😅 |
Member
Np, let me take a look at this one so we can decide which approach is better. I'll add myself as a reviewer if that's ok with you. |
Contributor
Author
|
moved back to draft, as @pippolo84 has some other branch also in progress |
a842d82 to
70966b4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
by creating CiliumNetworkDriverClusterConfig objects, a user can apply the same CiliumNetworkDriverNodeConfig to multiple nodes based on label selectors.
the cilium-operator looks at the user provided CiliumNetworkDriverClusterConfig objects, and for each node whose labels match the selector specified, it creates CiliumNetworkDriverNodeConfig objects targeting each node.
since only one config can be used at a time on the agent nodes, additional configs matching a node generate an error log and we also propagate an error through the status field for the CiliumNetworkDriverClusterConfig resource.
lastly, non-operator (that is, CiliumNetworkDriverNodeConfig created by the user directly) configs are preferred and not touched by the cilium operator.
implements a reconciler logic that triggers the selection process on every update and manage updates and cleanups.
The following rules are implemented:
Selection logic prefers:
1- User-created configs (nor managed by the operator).
2- Selector based config
3- Catch-all config
4- No config
If selection finds more than one config matching a node, conflicts
are reported by writing to the Status field of the ClusterConfig object.
What is allowed vs what is blocked (what happens when a new configuration comes in):
Blocked (conflict reported):
Selector-based → Selector-based: New config blocked, reports conflict
reconciler.go:485-503
Catch-all → Selector-based: New config (catch-all) blocked, reports conflict
reconciler.go:504-523
Catch-all → Catch-all: New config blocked, reports conflict
reconciler.go:527-545
Allowed (no conflict):
Selector-based → Catch-all: New config (selector-based) updates NodeConfig successfully
reconciler.go:547-548
Any config → Deleted/no config: New config takes ownership
reconciler.go:552-553
Annotations track source config and selector presence
reconciler.go:41 (AnnotationSourceClusterConfig)
reconciler.go:44 (AnnotationHadSelector)
Conflicts reported via Kubernetes Condition API
reconciler.go:800-852 (updateClusterConfigStatus)
NodeConfig updates only when allowed
reconciler.go:566-584 (update logic)
disclaimer: AI assisted