CFP-18405: ENI IPAM Multi-Pool Migration#87
CFP-18405: ENI IPAM Multi-Pool Migration#87HadrienPatte wants to merge 2 commits intocilium:mainfrom
Conversation
Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
gandro
left a comment
There was a problem hiding this comment.
I am so happy to see this being tackled! 🥹
The plan here sounds very reasonable and implementable to me. I have left some comments on the key questions, but the major concerns of such a change are already addressed in the document from my perspective, so I am approving of this.
| * `Spec.IPAM.Pool` remains populated on all nodes even when all agents have upgraded, until 1.21. | ||
| * Slightly more data written per CiliumNode object. | ||
|
|
||
| #### Option 2: Temporary Opt-Out Flag to Disable Dual-Write |
There was a problem hiding this comment.
I'm mildly in favor of providing this.
We could even go a step further and and enable this by default for new installations. While it is a potentially dangerous flag to enable, I do feel like with proper documentation this risk is mitigated.
There was a problem hiding this comment.
Do we have an established way of doing "if new installation" type of logic today? I feel like a lot of the great options on this page currently default to being disabled (even on new installations) because they're not safe to be enabled on existing clusters.
There was a problem hiding this comment.
Yes, we do, the upgradeCompatibility flag. Though it is possible that many users are not using it when upgrading, which would of course be pretty bad for them. If we want to be safe, maybe your original proposal to make this strictly opt-in is the way to go.
Update based on feedback Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Thanks, I have updated the CFP based on your comments and changed its status from |
Extend `NewCIDRRange` with a `WithAllowFirstLastIPs` functional option that includes the first and last IPs of a CIDR in the allocatable range. This is needed for AWS ENI prefix delegation where /28 prefixes are exclusively assigned to a node and all 16 IPs are usable as there is no shared network segment requiring base/broadcast reservation. Also refactor `ForEach` to use `r.base` directly instead of assuming a +1 offset from the CIDR base, making it correct for both default and `AllowFirstLastIPs` ranges. Relates to [cilium/design-cfps#87](cilium/design-cfps#87) Followup to #34618 See #28637 (comment) Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Extract ENI device configuration into a standalone CiliumNode observer that runs independently of the IPAM allocator. Previously, `configureENIDevices` was called from `nodeStore.updateLocalNodeResource`, coupling ENI network device setup to the CRD allocator code path. The new observer follows the same `job.Observer` pattern used by `startLocalNodeAllocCIDRsSync` in the multi-pool allocator. It watches CiliumNode updates and configures newly attached ENI devices regardless of which allocator is active. Relates to [cilium/design-cfps#87](cilium/design-cfps#87) Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Extract ENI device configuration into a standalone CiliumNode observer that runs independently of the IPAM allocator. Previously, `configureENIDevices` was called from `nodeStore.updateLocalNodeResource`, coupling ENI network device setup to the CRD allocator code path. The new observer follows the same `job.Observer` pattern used by `startLocalNodeAllocCIDRsSync` in the multi-pool allocator. It watches CiliumNode updates and configures newly attached ENI devices regardless of which allocator is active. Relates to [cilium/design-cfps#87](cilium/design-cfps#87) Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Extract ENI device configuration into a standalone CiliumNode observer that runs independently of the IPAM allocator. Previously, `configureENIDevices` was called from `nodeStore.updateLocalNodeResource`, coupling ENI network device setup to the CRD allocator code path. The new observer follows the same `job.Observer` pattern used by `startLocalNodeAllocCIDRsSync` in the multi-pool allocator. It watches CiliumNode updates and configures newly attached ENI devices regardless of which allocator is active. Relates to [cilium/design-cfps#87](cilium/design-cfps#87) Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Extend `NewCIDRRange` with a `WithAllowFirstLastIPs` functional option that includes the first and last IPs of a CIDR in the allocatable range. This is needed for AWS ENI prefix delegation where /28 prefixes are exclusively assigned to a node and all 16 IPs are usable as there is no shared network segment requiring base/broadcast reservation. Also refactor `ForEach` to use `r.base` directly instead of assuming a +1 offset from the CIDR base, making it correct for both default and `AllowFirstLastIPs` ranges. Relates to [cilium/design-cfps#87](cilium/design-cfps#87) Followup to #34618 See #28637 (comment) Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Extend `NewCIDRRange` with a `WithAllowFirstLastIPs` functional option that includes the first and last IPs of a CIDR in the allocatable range. This is needed for AWS ENI prefix delegation where /28 prefixes are exclusively assigned to a node and all 16 IPs are usable as there is no shared network segment requiring base/broadcast reservation. Also refactor `ForEach` to use `r.base` directly instead of assuming a +1 offset from the CIDR base, making it correct for both default and `AllowFirstLastIPs` ranges. Relates to [cilium/design-cfps#87](cilium/design-cfps#87) Followup to #34618 See #28637 (comment) Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Extract ENI device configuration into a standalone CiliumNode observer that runs independently of the IPAM allocator. Previously, `configureENIDevices` was called from `nodeStore.updateLocalNodeResource`, coupling ENI network device setup to the CRD allocator code path. The new observer follows the same `job.Observer` pattern used by `startLocalNodeAllocCIDRsSync` in the multi-pool allocator. It watches CiliumNode updates and configures newly attached ENI devices regardless of which allocator is active. Relates to [cilium/design-cfps#87](cilium/design-cfps#87) Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Extract ENI device configuration into a standalone CiliumNode observer that runs independently of the IPAM allocator. Previously, `configureENIDevices` was called from `nodeStore.updateLocalNodeResource`, coupling ENI network device setup to the CRD allocator code path. The new observer follows the same `job.Observer` pattern used by `startLocalNodeAllocCIDRsSync` in the multi-pool allocator. It watches CiliumNode updates and configures newly attached ENI devices regardless of which allocator is active. Relates to [cilium/design-cfps#87](cilium/design-cfps#87) Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Extend `NewCIDRRange` with a `WithAllowFirstLastIPs` functional option that includes the first and last IPs of a CIDR in the allocatable range. This is needed for AWS ENI prefix delegation where /28 prefixes are exclusively assigned to a node and all 16 IPs are usable as there is no shared network segment requiring base/broadcast reservation. Also refactor `ForEach` to use `r.base` directly instead of assuming a +1 offset from the CIDR base, making it correct for both default and `AllowFirstLastIPs` ranges. Relates to [cilium/design-cfps#87](cilium/design-cfps#87) Followup to #34618 See #28637 (comment) Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Add an `allowFirstLastIPs` parameter to `cidrPool` that propagates the `WithAllowFirstLastIPs` option to `ipallocator.NewCIDRRange` when creating new CIDR allocators in `updatePool`. This allows pools used for delegated prefixes (e.g. AWS /28 prefix delegation) to allocate all IPs in the range without reserving the first and last addresses. Existing multi-pool callers pass false to preserve current behavior. Relates to [cilium/design-cfps#87](cilium/design-cfps#87) This is a direct followup to #45025 and a small setup step before the larger migration or ENI IPAM from the CRD allocator to the multipool allocator. Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Add `buildENIAllocationResult` which derives ENI-specific metadata by matching an allocated IP against each ENI's Addresses and Prefixes lists. This replaces the CRD allocator's inline ENI branch in `buildAllocationResult`, which looked up the ENI by `AllocationIP.Resource` (ENI ID). The IP-based lookup is needed for the multi-pool allocator, as it allocates IPs from CIDRs without tracking per-IP resource metadata. The `eniContainsIP` helper covers all cases: * IP is a secondary ENI IP: under `eni.Addresses` * IP belongs to a delegated prefix: under `eni.Prefixes` * IP is ENI primary IP (necessary for when `UsePrimaryAddress` is enabled): under `eni.IP` Relates to cilium/design-cfps#87 Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Add an `allowFirstLastIPs` parameter to `cidrPool` that propagates the `WithAllowFirstLastIPs` option to `ipallocator.NewCIDRRange` when creating new CIDR allocators in `updatePool`. This allows pools used for delegated prefixes (e.g. AWS /28 prefix delegation) to allocate all IPs in the range without reserving the first and last addresses. Existing multi-pool callers pass false to preserve current behavior. Relates to [cilium/design-cfps#87](cilium/design-cfps#87) This is a direct followup to #45025 and a small setup step before the larger migration or ENI IPAM from the CRD allocator to the multipool allocator. Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Add `buildENIAllocationResult` which derives ENI-specific metadata by matching an allocated IP against each ENI's Addresses and Prefixes lists. This replaces the CRD allocator's inline ENI branch in `buildAllocationResult`, which looked up the ENI by `AllocationIP.Resource` (ENI ID). The IP-based lookup is needed for the multi-pool allocator, as it allocates IPs from CIDRs without tracking per-IP resource metadata. The `eniContainsIP` helper covers all cases: * IP is a secondary ENI IP: under `eni.Addresses` * IP belongs to a delegated prefix: under `eni.Prefixes` * IP is ENI primary IP (necessary for when `UsePrimaryAddress` is enabled): under `eni.IP` Relates to cilium/design-cfps#87 Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
In `syncToAPIServer`, populate `Spec.IPAM.Pools.Allocated` alongside `Spec.IPAM.Pool` for nodes with ENI status. This enables the ENI multi-pool migration (cilium/design-cfps#87): new agents (1.20) will read CIDRs from `Pools.Allocated` (with the multipool allocator) while old agents (1.19) will continue reading from `Pool` (with the CRD allocator). Note: this PR only includes the double write logic on the operator side, a followup PR will change the read path on the agent side. For secondary IP mode, each IP is written as a /32 CIDR. For prefix delegation mode, each /28 prefix is written directly alongside the secondary IP /32s. All CIDRs are placed under the "default" pool. The dual-write is low-frequency (fires on ENI capacity changes, not on every pod event) and will be removed in 1.21. The dual write is also atomic, both `Pool` and `Pools.Allocated` get written in the same CilimNode update operation so they are garanteed to always be in sync and consistent. Relates to cilium/design-cfps#87 Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
In `syncToAPIServer`, populate `Spec.IPAM.Pools.Allocated` alongside `Spec.IPAM.Pool` for nodes with ENI status. This enables the ENI multi-pool migration (cilium/design-cfps#87): new agents (1.20) will read CIDRs from `Pools.Allocated` (with the multipool allocator) while old agents (1.19) will continue reading from `Pool` (with the CRD allocator). Note: this PR only includes the double write logic on the operator side, a followup PR will change the read path on the agent side. For secondary IP mode, each IP is written as a /32 CIDR. For prefix delegation mode, each /28 prefix is written directly alongside the secondary IP /32s. All CIDRs are placed under the "default" pool. The dual-write is low-frequency (fires on ENI capacity changes, not on every pod event) and will be removed in 1.21. The dual write is also atomic, both `Pool` and `Pools.Allocated` get written in the same CilimNode update operation so they are garanteed to always be in sync and consistent. Relates to cilium/design-cfps#87 Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
In `syncToAPIServer`, populate `Spec.IPAM.Pools.Allocated` alongside `Spec.IPAM.Pool` for nodes with ENI status. This enables the ENI multi-pool migration (cilium/design-cfps#87): new agents (1.20) will read CIDRs from `Pools.Allocated` (with the multipool allocator) while old agents (1.19) will continue reading from `Pool` (with the CRD allocator). Note: this PR only includes the double write logic on the operator side, a followup PR will change the read path on the agent side. For secondary IP mode, each IP is written as a /32 CIDR. For prefix delegation mode, each /28 prefix is written directly alongside the secondary IP /32s. All CIDRs are placed under the "default" pool. The dual-write is low-frequency (fires on ENI capacity changes, not on every pod event) and will be removed in 1.21. The dual write is also atomic, both `Pool` and `Pools.Allocated` get written in the same CilimNode update operation so they are garanteed to always be in sync and consistent. Relates to cilium/design-cfps#87 Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
This PR is the "double read" equivalent to #45110. In the operator's `recalculate()`, detect whether the agent is using the multi-pool or CRD allocator by checking `Spec.IPAM.Pools.Requested` entries and `Status.IPAM.Used`: * 1.20 agents write their total desired IP count to `Pools.Requested` and stop writing `Status.IPAM.Used`. * 1.19 agents only write `Status.IPAM.Used`. The dual-check handles the downgrade case: if a 1.20 agent wrote `Pools.Requested` and was then rolled back to 1.19, causing the `CiliumNode` to keep a stale `Pools.Requested` from its time under a 1.20 agent. The operator will properly handle that case by detecting that the agent is now populating `Status.IPAM.Used` again and will ignore `Pools.Requested`. One observable difference between both modes is that under the CRD allocator, the agents communicate their IP usage, so the operator needs to do additional computations on top of that to take into account pre-allocation buffers and watermarks to get the IP needed value. Under the multipool allocator, all those computations are handled by the agents and they directly communicate the resulting number of IP requested. A side effect of that difference, is that under multipool, the operator does not actually have access to the number of used IPs on a node. We now apply the reverse of the agent's "IP used to IP needed" computation logic to infer an approximation of the IP used value to emit it as a metric (`ipam.used_ips`). Relates to cilium/design-cfps#87 Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
This PR is the "double read" equivalent to #45110. In the operator's `recalculate()`, detect whether the agent is using the multi-pool or CRD allocator by checking `Spec.IPAM.Pools.Requested` entries and `Status.IPAM.Used`: * 1.20 agents write their total desired IP count to `Pools.Requested` and stop writing `Status.IPAM.Used`. * 1.19 agents only write `Status.IPAM.Used`. The dual-check handles the downgrade case: if a 1.20 agent wrote `Pools.Requested` and was then rolled back to 1.19, causing the `CiliumNode` to keep a stale `Pools.Requested` from its time under a 1.20 agent. The operator will properly handle that case by detecting that the agent is now populating `Status.IPAM.Used` again and will ignore `Pools.Requested`. Relates to cilium/design-cfps#87 Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
This PR is the "double read" equivalent to #45110. In the operator's `recalculate()`, detect whether the agent is using the multi-pool or CRD allocator by checking `Spec.IPAM.Pools.Requested` entries and `Status.IPAM.Used`: * 1.20 agents write their total desired IP count to `Pools.Requested` and stop writing `Status.IPAM.Used`. * 1.19 agents only write `Status.IPAM.Used`. The dual-check handles the downgrade case: if a 1.20 agent wrote `Pools.Requested` and was then rolled back to 1.19, causing the `CiliumNode` to keep a stale `Pools.Requested` from its time under a 1.20 agent. The operator will properly handle that case by detecting that the agent is now populating `Status.IPAM.Used` again and will ignore `Pools.Requested`. Relates to cilium/design-cfps#87 Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
This PR is the "double read" equivalent to #45110. In the operator's `recalculate()`, detect whether the agent is using the multi-pool or CRD allocator by checking `Spec.IPAM.Pools.Requested` entries and `Status.IPAM.Used`: * 1.20 agents write their total desired IP count to `Pools.Requested` and stop writing `Status.IPAM.Used`. * 1.19 agents only write `Status.IPAM.Used`. The dual-check handles the downgrade case: if a 1.20 agent wrote `Pools.Requested` and was then rolled back to 1.19, causing the `CiliumNode` to keep a stale `Pools.Requested` from its time under a 1.20 agent. The operator will properly handle that case by detecting that the agent is now populating `Status.IPAM.Used` again and will ignore `Pools.Requested`. Relates to cilium/design-cfps#87 Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
This PR is the "double read" equivalent to #45110. In the operator's `recalculate()`, detect whether the agent is using the multi-pool or CRD allocator by checking `Spec.IPAM.Pools.Requested` entries and `Status.IPAM.Used`: * 1.20 agents write their total desired IP count to `Pools.Requested` and stop writing `Status.IPAM.Used`. * 1.19 agents only write `Status.IPAM.Used`. The dual-check handles the downgrade case: if a 1.20 agent wrote `Pools.Requested` and was then rolled back to 1.19, causing the `CiliumNode` to keep a stale `Pools.Requested` from its time under a 1.20 agent. The operator will properly handle that case by detecting that the agent is now populating `Status.IPAM.Used` again and will ignore `Pools.Requested`. Relates to cilium/design-cfps#87 Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
In `syncToAPIServer`, populate `Spec.IPAM.Pools.Allocated` alongside `Spec.IPAM.Pool` for nodes with ENI status. This enables the ENI multi-pool migration (cilium/design-cfps#87): new agents (1.20) will read CIDRs from `Pools.Allocated` (with the multipool allocator) while old agents (1.19) will continue reading from `Pool` (with the CRD allocator). Note: this PR only includes the double write logic on the operator side, a followup PR will change the read path on the agent side. For secondary IP mode, each IP is written as a /32 CIDR. For prefix delegation mode, each /28 prefix is written directly alongside the secondary IP /32s. All CIDRs are placed under the "default" pool. The dual-write is low-frequency (fires on ENI capacity changes, not on every pod event) and will be removed in 1.21. The dual write is also atomic, both `Pool` and `Pools.Allocated` get written in the same CilimNode update operation so they are garanteed to always be in sync and consistent. Relates to cilium/design-cfps#87 Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
This PR is the "double read" equivalent to #45110. In the operator's `recalculate()`, detect whether the agent is using the multi-pool or CRD allocator by checking `Spec.IPAM.Pools.Requested` entries and `Status.IPAM.Used`: * 1.20 agents write their total desired IP count to `Pools.Requested` and stop writing `Status.IPAM.Used`. * 1.19 agents only write `Status.IPAM.Used`. The dual-check handles the downgrade case: if a 1.20 agent wrote `Pools.Requested` and was then rolled back to 1.19, causing the `CiliumNode` to keep a stale `Pools.Requested` from its time under a 1.20 agent. The operator will properly handle that case by detecting that the agent is now populating `Status.IPAM.Used` again and will ignore `Pools.Requested`. Relates to cilium/design-cfps#87 Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
This PR is the "double read" equivalent to #45110. In the operator's `recalculate()`, detect whether the agent is using the multi-pool or CRD allocator by checking `Spec.IPAM.Pools.Requested` entries and `Status.IPAM.Used`: * 1.20 agents write their total desired IP count to `Pools.Requested` and stop writing `Status.IPAM.Used`. * 1.19 agents only write `Status.IPAM.Used`. The dual-check handles the downgrade case: if a 1.20 agent wrote `Pools.Requested` and was then rolled back to 1.19, causing the `CiliumNode` to keep a stale `Pools.Requested` from its time under a 1.20 agent. The operator will properly handle that case by detecting that the agent is now populating `Status.IPAM.Used` again and will ignore `Pools.Requested`. Relates to cilium/design-cfps#87 Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Replace the CRD allocator with the multi-pool allocator for ENI IPAM mode on the agent side. Previous PRs cilium#45110 and cilium#45124 ensure the operator already supports this new agent setup. The new `eniMultiPoolAllocator` is a light wrapper on the standard `multiPoolAllocator` that enriches `AllocationResult` with ENI-specific required metadata via `buildENIAllocationResult` (see cilium#45089). Key differences from the standard multi-pool allocator: * `AllowFirstLastIPs` is enabled so /28 prefix delegation ranges are fully allocatable (see cilium#45025 and cilium#45082). * `LinearPreAlloc` uses a simple `inUse + preAlloc` formula for demand computation instead of `neededIPCeil` rounding. This matches the CRD allocator's `calculateNeededIPs` semantics and is necessary to ensure the operator can recover the exact IP usage from the demand signal (requested - preAllocate) (see cilium#45124). * No dependency on `CiliumPodIPPool` CRDs, pools are managed by the operator via Spec.IPAM.Pools.Allocated (see cilium#45110). The agents now read CIDRs from `Spec.IPAM.Pools.Allocated`, allocate IPs locally, and write aggregate demand to `Spec.IPAM.Pools.Requested`. They no longer writes per-IP usage to `Status.IPAM.Used`, achieving a reduction of kubernetes API pressure. Relates to cilium/design-cfps#87 Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Replace the CRD allocator with the multi-pool allocator for ENI IPAM mode on the agent side. Previous PRs cilium#45110 and cilium#45124 ensure the operator already supports this new agent setup. The new `eniMultiPoolAllocator` is a light wrapper on the standard `multiPoolAllocator` that enriches `AllocationResult` with ENI-specific required metadata via `buildENIAllocationResult` (see cilium#45089). Key differences from the standard multi-pool allocator: * `AllowFirstLastIPs` is enabled so /28 prefix delegation ranges are fully allocatable (see cilium#45025 and cilium#45082). * `LinearPreAlloc` uses a simple `inUse + preAlloc` formula for demand computation instead of `neededIPCeil` rounding. This matches the CRD allocator's `calculateNeededIPs` semantics and is necessary to ensure the operator can recover the exact IP usage from the demand signal (requested - preAllocate) (see cilium#45124). * No dependency on `CiliumPodIPPool` CRDs, pools are managed by the operator via Spec.IPAM.Pools.Allocated (see cilium#45110). The agents now read CIDRs from `Spec.IPAM.Pools.Allocated`, allocate IPs locally, and write aggregate demand to `Spec.IPAM.Pools.Requested`. They no longer writes per-IP usage to `Status.IPAM.Used`, achieving a reduction of kubernetes API pressure. Relates to cilium/design-cfps#87 Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Replace the CRD allocator with the multi-pool allocator for ENI IPAM mode on the agent side. Previous PRs #45110 and #45124 ensure the operator already supports this new agent setup. The new `eniMultiPoolAllocator` is a light wrapper on the standard `multiPoolAllocator` that enriches `AllocationResult` with ENI-specific required metadata via `buildENIAllocationResult` (see #45089). Key differences from the standard multi-pool allocator: * `AllowFirstLastIPs` is enabled so /28 prefix delegation ranges are fully allocatable (see #45025 and #45082). * `LinearPreAlloc` uses a simple `inUse + preAlloc` formula for demand computation instead of `neededIPCeil` rounding. This matches the CRD allocator's `calculateNeededIPs` semantics and is necessary to ensure the operator can recover the exact IP usage from the demand signal (requested - preAllocate) (see #45124). * No dependency on `CiliumPodIPPool` CRDs, pools are managed by the operator via Spec.IPAM.Pools.Allocated (see #45110). The agents now read CIDRs from `Spec.IPAM.Pools.Allocated`, allocate IPs locally, and write aggregate demand to `Spec.IPAM.Pools.Requested`. They no longer writes per-IP usage to `Status.IPAM.Used`, achieving a reduction of kubernetes API pressure. Relates to cilium/design-cfps#87 Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Replace the CRD allocator with the multi-pool allocator for ENI IPAM mode on the agent side. Previous PRs #45110 and #45124 ensure the operator already supports this new agent setup. The new `eniMultiPoolAllocator` is a light wrapper on the standard `multiPoolAllocator` that enriches `AllocationResult` with ENI-specific required metadata via `buildENIAllocationResult` (see #45089). Key differences from the standard multi-pool allocator: * `AllowFirstLastIPs` is enabled so /28 prefix delegation ranges are fully allocatable (see #45025 and #45082). * `LinearPreAlloc` uses a simple `inUse + preAlloc` formula for demand computation instead of `neededIPCeil` rounding. This matches the CRD allocator's `calculateNeededIPs` semantics and is necessary to ensure the operator can recover the exact IP usage from the demand signal (requested - preAllocate) (see #45124). * No dependency on `CiliumPodIPPool` CRDs, pools are managed by the operator via Spec.IPAM.Pools.Allocated (see #45110). The agents now read CIDRs from `Spec.IPAM.Pools.Allocated`, allocate IPs locally, and write aggregate demand to `Spec.IPAM.Pools.Requested`. They no longer writes per-IP usage to `Status.IPAM.Used`, achieving a reduction of kubernetes API pressure. Relates to cilium/design-cfps#87 Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Replace the CRD allocator with the multi-pool allocator for ENI IPAM mode on the agent side. Previous PR #45124 ensures the operator already supports this new agent setup. The new `eniMultiPoolAllocator` is a light wrapper on the standard `multiPoolAllocator` that enriches `AllocationResult` with ENI-specific required metadata via `buildENIAllocationResult` (see #45089). Key differences from the standard multi-pool allocator: * `AllowFirstLastIPs` is enabled so /28 prefix delegation ranges are fully allocatable (see #45025 and #45082). * `LinearPreAlloc` uses a simple `inUse + preAlloc` formula for demand computation instead of `neededIPCeil` rounding. This matches the CRD allocator's `calculateNeededIPs` semantics and is necessary to ensure the operator can recover the exact IP usage from the demand signal (requested - preAllocate) (see #45124). * No dependency on `CiliumPodIPPool` CRDs, pools are instead populated by the agent from `Status.ENI.ENIs` which is maintained by the operator. The agents now read CIDRs from `Status.ENI.ENIs`, allocate IPs locally, and write aggregate demand to `Spec.IPAM.Pools.Requested`. They no longer writes per-IP usage to `Status.IPAM.Used`, achieving a reduction of kubernetes API pressure. Relates to cilium/design-cfps#87 Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Replace the CRD allocator with the multi-pool allocator for ENI IPAM mode on the agent side. Previous PR #45124 ensures the operator already supports this new agent setup. The new `eniMultiPoolAllocator` is a light wrapper on the standard `multiPoolAllocator` that enriches `AllocationResult` with ENI-specific required metadata via `buildENIAllocationResult` (see #45089). Key differences from the standard multi-pool allocator: * `AllowFirstLastIPs` is enabled so /28 prefix delegation ranges are fully allocatable (see #45025 and #45082). * `LinearPreAlloc` uses a simple `inUse + preAlloc` formula for demand computation instead of `neededIPCeil` rounding. This matches the CRD allocator's `calculateNeededIPs` semantics and is necessary to ensure the operator can recover the exact IP usage from the demand signal (requested - preAllocate) (see #45124). * No dependency on `CiliumPodIPPool` CRDs, pools are instead populated by the agent from `Status.ENI.ENIs` which is maintained by the operator. The agents now read CIDRs from `Status.ENI.ENIs`, allocate IPs locally, and write aggregate demand to `Spec.IPAM.Pools.Requested`. They no longer writes per-IP usage to `Status.IPAM.Used`, achieving a reduction of kubernetes API pressure. Relates to cilium/design-cfps#87 Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Replace the CRD allocator with the multi-pool allocator for ENI IPAM mode on the agent side. Previous PR #45124 ensures the operator already supports this new agent setup. The new `eniMultiPoolAllocator` is a light wrapper on the standard `multiPoolAllocator` that enriches `AllocationResult` with ENI-specific required metadata via `buildENIAllocationResult` (see #45089). Key differences from the standard multi-pool allocator: * `AllowFirstLastIPs` is enabled so /28 prefix delegation ranges are fully allocatable (see #45025 and #45082). * `LinearPreAlloc` uses a simple `inUse + preAlloc` formula for demand computation instead of `neededIPCeil` rounding. This matches the CRD allocator's `calculateNeededIPs` semantics and is necessary to ensure the operator can recover the exact IP usage from the demand signal (requested - preAllocate) (see #45124). * No dependency on `CiliumPodIPPool` CRDs, pools are instead populated by the agent from `Status.ENI.ENIs` which is maintained by the operator. The agents now read CIDRs from `Status.ENI.ENIs`, allocate IPs locally, and write aggregate demand to `Spec.IPAM.Pools.Requested`. They no longer writes per-IP usage to `Status.IPAM.Used`, achieving a reduction of kubernetes API pressure. Relates to cilium/design-cfps#87 Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
This CFP is a more detailed version of the design I mentioned in cilium/cilium#19251 (comment)