Cilium Feature Proposal

Summary

This CFP proposes extending cilium to support allocating dedicated (NIC) and shared (SRIOV) network devices to workloads - leveraging the Kubernetes DRA framework.

References:

• Dynamic Resource Allocation
• DRA Plugin Interface definition

Motivation

Abstractions do not come for free. The traditional Kubernetes networking model relies on a set of abstractions with the sole purpose of simplifying connectivity between applications and the rest of the world (including other workloads).
Under the hood, Cilium CNI attaches a pod to a logical fabric over which workloads can benefit from secure network access to other workloads, services and clients. These attachments take the form of virtual devices (veth/netkit) that hand packets to a custom forwarding plane (ebpf) that magically achieves connectivity between the endpoints. Most of this magic happens in software with varying levels of involvement of the kernel.
This is suitable for the vast majority of the use cases, but performance/latency sensitive workloads could benefit even further from having access to some underlying network hardware (a NIC, so to speak) and shorten even further the path a packet takes between the application and the network device itself. In such scenarios, performance (HW access) is more important than sophisticated connectivity and feature set.
Some use cases that come to mind are network function (CNF/VNF) workloads, low latency data ingestion, and dpdk based applications combined with SR-IOV.

Proposal

Extends the agent to register itself with the Kubernetes API as a DRA plugin (for publishing resources), and also with the container runtime (for configuring devices).
The Cilium Network Driver publishes local resources (ex: network devices) that match a given set of user specified filters for each node grouped in resource pools, allowing workloads to claim such resources.
Upon receiving a claim request, Cilium Network Driver performs any preparation needed for the allocated device(s). When the Pod finally starts, Cilium Network Driver performs any final configuration before assigning the device to the Pod sandbox.