Skip to content

CFP-18405: ENI IPAM Multi-Pool Migration#87

Open
HadrienPatte wants to merge 2 commits intocilium:mainfrom
HadrienPatte:pr/HadrienPatte/eni-ipam-multi-pool
Open

CFP-18405: ENI IPAM Multi-Pool Migration#87
HadrienPatte wants to merge 2 commits intocilium:mainfrom
HadrienPatte:pr/HadrienPatte/eni-ipam-multi-pool

Conversation

@HadrienPatte
Copy link
Copy Markdown
Member

This CFP is a more detailed version of the design I mentioned in cilium/cilium#19251 (comment)

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Copy link
Copy Markdown
Member

@gandro gandro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am so happy to see this being tackled! 🥹

The plan here sounds very reasonable and implementable to me. I have left some comments on the key questions, but the major concerns of such a change are already addressed in the document from my perspective, so I am approving of this.

* `Spec.IPAM.Pool` remains populated on all nodes even when all agents have upgraded, until 1.21.
* Slightly more data written per CiliumNode object.

#### Option 2: Temporary Opt-Out Flag to Disable Dual-Write
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm mildly in favor of providing this.

We could even go a step further and and enable this by default for new installations. While it is a potentially dangerous flag to enable, I do feel like with proper documentation this risk is mitigated.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have an established way of doing "if new installation" type of logic today? I feel like a lot of the great options on this page currently default to being disabled (even on new installations) because they're not safe to be enabled on existing clusters.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we do, the upgradeCompatibility flag. Though it is possible that many users are not using it when upgrading, which would of course be pretty bad for them. If we want to be safe, maybe your original proposal to make this strictly opt-in is the way to go.

https://docs.cilium.io/en/stable/operations/upgrade/#step-2-use-helm-to-upgrade-your-cilium-deployment

Update based on feedback

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
@HadrienPatte
Copy link
Copy Markdown
Member Author

The plan here sounds very reasonable and implementable to me. I have left some comments on the key questions, but the major concerns of such a change are already addressed in the document from my perspective, so I am approving of this.

Thanks, I have updated the CFP based on your comments and changed its status from draft to implementable. Will leave it open for more people to comment.

HadrienPatte added a commit to cilium/cilium that referenced this pull request Mar 27, 2026
Extend `NewCIDRRange` with a `WithAllowFirstLastIPs` functional option
that includes the first and last IPs of a CIDR in the allocatable range.
This is needed for AWS ENI prefix delegation where /28 prefixes are
exclusively assigned to a node and all 16 IPs are usable as there is no
shared network segment requiring base/broadcast reservation.

Also refactor `ForEach` to use `r.base` directly instead of assuming a +1
offset from the CIDR base, making it correct for both default and
`AllowFirstLastIPs` ranges.

Relates to [cilium/design-cfps#87](cilium/design-cfps#87)

Followup to #34618

See #28637 (comment)

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to cilium/cilium that referenced this pull request Mar 27, 2026
Extract ENI device configuration into a standalone CiliumNode observer
that runs independently of the IPAM allocator. Previously,
`configureENIDevices` was called from
`nodeStore.updateLocalNodeResource`, coupling ENI network device setup to
the CRD allocator code path.

The new observer follows the same `job.Observer` pattern used by
`startLocalNodeAllocCIDRsSync` in the multi-pool allocator. It watches
CiliumNode updates and configures newly attached ENI devices regardless
of which allocator is active.

Relates to [cilium/design-cfps#87](cilium/design-cfps#87)

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to cilium/cilium that referenced this pull request Mar 27, 2026
Extract ENI device configuration into a standalone CiliumNode observer
that runs independently of the IPAM allocator. Previously,
`configureENIDevices` was called from
`nodeStore.updateLocalNodeResource`, coupling ENI network device setup to
the CRD allocator code path.

The new observer follows the same `job.Observer` pattern used by
`startLocalNodeAllocCIDRsSync` in the multi-pool allocator. It watches
CiliumNode updates and configures newly attached ENI devices regardless
of which allocator is active.

Relates to [cilium/design-cfps#87](cilium/design-cfps#87)

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to cilium/cilium that referenced this pull request Mar 28, 2026
Extract ENI device configuration into a standalone CiliumNode observer
that runs independently of the IPAM allocator. Previously,
`configureENIDevices` was called from
`nodeStore.updateLocalNodeResource`, coupling ENI network device setup to
the CRD allocator code path.

The new observer follows the same `job.Observer` pattern used by
`startLocalNodeAllocCIDRsSync` in the multi-pool allocator. It watches
CiliumNode updates and configures newly attached ENI devices regardless
of which allocator is active.

Relates to [cilium/design-cfps#87](cilium/design-cfps#87)

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to cilium/cilium that referenced this pull request Mar 30, 2026
Extend `NewCIDRRange` with a `WithAllowFirstLastIPs` functional option
that includes the first and last IPs of a CIDR in the allocatable range.
This is needed for AWS ENI prefix delegation where /28 prefixes are
exclusively assigned to a node and all 16 IPs are usable as there is no
shared network segment requiring base/broadcast reservation.

Also refactor `ForEach` to use `r.base` directly instead of assuming a +1
offset from the CIDR base, making it correct for both default and
`AllowFirstLastIPs` ranges.

Relates to [cilium/design-cfps#87](cilium/design-cfps#87)

Followup to #34618

See #28637 (comment)

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to cilium/cilium that referenced this pull request Mar 30, 2026
Extend `NewCIDRRange` with a `WithAllowFirstLastIPs` functional option
that includes the first and last IPs of a CIDR in the allocatable range.
This is needed for AWS ENI prefix delegation where /28 prefixes are
exclusively assigned to a node and all 16 IPs are usable as there is no
shared network segment requiring base/broadcast reservation.

Also refactor `ForEach` to use `r.base` directly instead of assuming a +1
offset from the CIDR base, making it correct for both default and
`AllowFirstLastIPs` ranges.

Relates to [cilium/design-cfps#87](cilium/design-cfps#87)

Followup to #34618

See #28637 (comment)

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to cilium/cilium that referenced this pull request Mar 30, 2026
Extract ENI device configuration into a standalone CiliumNode observer
that runs independently of the IPAM allocator. Previously,
`configureENIDevices` was called from
`nodeStore.updateLocalNodeResource`, coupling ENI network device setup to
the CRD allocator code path.

The new observer follows the same `job.Observer` pattern used by
`startLocalNodeAllocCIDRsSync` in the multi-pool allocator. It watches
CiliumNode updates and configures newly attached ENI devices regardless
of which allocator is active.

Relates to [cilium/design-cfps#87](cilium/design-cfps#87)

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
github-merge-queue bot pushed a commit to cilium/cilium that referenced this pull request Mar 31, 2026
Extract ENI device configuration into a standalone CiliumNode observer
that runs independently of the IPAM allocator. Previously,
`configureENIDevices` was called from
`nodeStore.updateLocalNodeResource`, coupling ENI network device setup to
the CRD allocator code path.

The new observer follows the same `job.Observer` pattern used by
`startLocalNodeAllocCIDRsSync` in the multi-pool allocator. It watches
CiliumNode updates and configures newly attached ENI devices regardless
of which allocator is active.

Relates to [cilium/design-cfps#87](cilium/design-cfps#87)

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
github-merge-queue bot pushed a commit to cilium/cilium that referenced this pull request Mar 31, 2026
Extend `NewCIDRRange` with a `WithAllowFirstLastIPs` functional option
that includes the first and last IPs of a CIDR in the allocatable range.
This is needed for AWS ENI prefix delegation where /28 prefixes are
exclusively assigned to a node and all 16 IPs are usable as there is no
shared network segment requiring base/broadcast reservation.

Also refactor `ForEach` to use `r.base` directly instead of assuming a +1
offset from the CIDR base, making it correct for both default and
`AllowFirstLastIPs` ranges.

Relates to [cilium/design-cfps#87](cilium/design-cfps#87)

Followup to #34618

See #28637 (comment)

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to cilium/cilium that referenced this pull request Mar 31, 2026
Add an `allowFirstLastIPs` parameter to `cidrPool` that propagates the
`WithAllowFirstLastIPs` option to `ipallocator.NewCIDRRange` when
creating new CIDR allocators in `updatePool`. This allows pools used for
delegated prefixes (e.g. AWS /28 prefix delegation) to allocate all IPs
in the range without reserving the first and last addresses.

Existing multi-pool callers pass false to preserve current behavior.

Relates to [cilium/design-cfps#87](cilium/design-cfps#87)

This is a direct followup to #45025 and a small setup step before the
larger migration or ENI IPAM from the CRD allocator to the multipool
allocator.

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to cilium/cilium that referenced this pull request Mar 31, 2026
Add `buildENIAllocationResult` which derives ENI-specific metadata by
matching an allocated IP against each ENI's Addresses and Prefixes lists.
This replaces the CRD allocator's inline ENI branch in
`buildAllocationResult`, which looked up the ENI by
`AllocationIP.Resource` (ENI ID).

The IP-based lookup is needed for the multi-pool allocator, as it
allocates IPs from CIDRs without tracking per-IP resource metadata. The
`eniContainsIP` helper covers all cases:
* IP is a secondary ENI IP: under `eni.Addresses`
* IP belongs to a delegated prefix: under `eni.Prefixes`
* IP is ENI primary IP (necessary for when `UsePrimaryAddress` is
  enabled): under `eni.IP`

Relates to cilium/design-cfps#87

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
github-merge-queue bot pushed a commit to cilium/cilium that referenced this pull request Apr 1, 2026
Add an `allowFirstLastIPs` parameter to `cidrPool` that propagates the
`WithAllowFirstLastIPs` option to `ipallocator.NewCIDRRange` when
creating new CIDR allocators in `updatePool`. This allows pools used for
delegated prefixes (e.g. AWS /28 prefix delegation) to allocate all IPs
in the range without reserving the first and last addresses.

Existing multi-pool callers pass false to preserve current behavior.

Relates to [cilium/design-cfps#87](cilium/design-cfps#87)

This is a direct followup to #45025 and a small setup step before the
larger migration or ENI IPAM from the CRD allocator to the multipool
allocator.

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
github-merge-queue bot pushed a commit to cilium/cilium that referenced this pull request Apr 1, 2026
Add `buildENIAllocationResult` which derives ENI-specific metadata by
matching an allocated IP against each ENI's Addresses and Prefixes lists.
This replaces the CRD allocator's inline ENI branch in
`buildAllocationResult`, which looked up the ENI by
`AllocationIP.Resource` (ENI ID).

The IP-based lookup is needed for the multi-pool allocator, as it
allocates IPs from CIDRs without tracking per-IP resource metadata. The
`eniContainsIP` helper covers all cases:
* IP is a secondary ENI IP: under `eni.Addresses`
* IP belongs to a delegated prefix: under `eni.Prefixes`
* IP is ENI primary IP (necessary for when `UsePrimaryAddress` is
  enabled): under `eni.IP`

Relates to cilium/design-cfps#87

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to cilium/cilium that referenced this pull request Apr 1, 2026
In `syncToAPIServer`, populate `Spec.IPAM.Pools.Allocated` alongside
`Spec.IPAM.Pool` for nodes with ENI status. This enables the ENI
multi-pool migration (cilium/design-cfps#87): new agents (1.20) will read
CIDRs from `Pools.Allocated` (with the multipool allocator) while old
agents (1.19) will continue reading from `Pool` (with the CRD allocator).

Note: this PR only includes the double write logic on the operator side,
a followup PR will change the read path on the agent side.

For secondary IP mode, each IP is written as a /32 CIDR. For prefix
delegation mode, each /28 prefix is written directly alongside the
secondary IP /32s. All CIDRs are placed under the "default" pool.

The dual-write is low-frequency (fires on ENI capacity changes, not on
every pod event) and will be removed in 1.21. The dual write is also
atomic, both `Pool` and `Pools.Allocated` get written in the same
CilimNode update operation so they are garanteed to always be in sync
and consistent.

Relates to cilium/design-cfps#87

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to DataDog/cilium that referenced this pull request Apr 1, 2026
In `syncToAPIServer`, populate `Spec.IPAM.Pools.Allocated` alongside
`Spec.IPAM.Pool` for nodes with ENI status. This enables the ENI
multi-pool migration (cilium/design-cfps#87): new agents (1.20) will read
CIDRs from `Pools.Allocated` (with the multipool allocator) while old
agents (1.19) will continue reading from `Pool` (with the CRD allocator).

Note: this PR only includes the double write logic on the operator side,
a followup PR will change the read path on the agent side.

For secondary IP mode, each IP is written as a /32 CIDR. For prefix
delegation mode, each /28 prefix is written directly alongside the
secondary IP /32s. All CIDRs are placed under the "default" pool.

The dual-write is low-frequency (fires on ENI capacity changes, not on
every pod event) and will be removed in 1.21. The dual write is also
atomic, both `Pool` and `Pools.Allocated` get written in the same
CilimNode update operation so they are garanteed to always be in sync
and consistent.

Relates to cilium/design-cfps#87

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to DataDog/cilium that referenced this pull request Apr 1, 2026
In `syncToAPIServer`, populate `Spec.IPAM.Pools.Allocated` alongside
`Spec.IPAM.Pool` for nodes with ENI status. This enables the ENI
multi-pool migration (cilium/design-cfps#87): new agents (1.20) will read
CIDRs from `Pools.Allocated` (with the multipool allocator) while old
agents (1.19) will continue reading from `Pool` (with the CRD allocator).

Note: this PR only includes the double write logic on the operator side,
a followup PR will change the read path on the agent side.

For secondary IP mode, each IP is written as a /32 CIDR. For prefix
delegation mode, each /28 prefix is written directly alongside the
secondary IP /32s. All CIDRs are placed under the "default" pool.

The dual-write is low-frequency (fires on ENI capacity changes, not on
every pod event) and will be removed in 1.21. The dual write is also
atomic, both `Pool` and `Pools.Allocated` get written in the same
CilimNode update operation so they are garanteed to always be in sync
and consistent.

Relates to cilium/design-cfps#87

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to cilium/cilium that referenced this pull request Apr 1, 2026
This PR is the "double read" equivalent to #45110.

In the operator's `recalculate()`, detect whether the agent is using the
multi-pool or CRD allocator by checking `Spec.IPAM.Pools.Requested`
entries and `Status.IPAM.Used`:
* 1.20 agents write their total desired IP count to `Pools.Requested` and
  stop writing `Status.IPAM.Used`.
* 1.19 agents only write `Status.IPAM.Used`.

The dual-check handles the downgrade case: if a 1.20 agent wrote
`Pools.Requested` and was then rolled back to 1.19, causing the
`CiliumNode` to keep a stale `Pools.Requested` from its time under a
1.20 agent. The operator will properly handle that case by detecting
that the agent is now populating `Status.IPAM.Used` again and will
ignore `Pools.Requested`.

One observable difference between both modes is that under the CRD
allocator, the agents communicate their IP usage, so the operator needs
to do additional computations on top of that to take into account
pre-allocation buffers and watermarks to get the IP needed value. Under
the multipool allocator, all those computations are handled by the
agents and they directly communicate the resulting number of IP
requested.

A side effect of that difference, is that under multipool, the operator
does not actually have access to the number of used IPs on a node. We
now apply the reverse of the agent's "IP used to IP needed" computation
logic to infer an approximation of the IP used value to emit it as a
metric (`ipam.used_ips`).

Relates to cilium/design-cfps#87

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to cilium/cilium that referenced this pull request Apr 1, 2026
This PR is the "double read" equivalent to #45110.

In the operator's `recalculate()`, detect whether the agent is using the
multi-pool or CRD allocator by checking `Spec.IPAM.Pools.Requested`
entries and `Status.IPAM.Used`:
* 1.20 agents write their total desired IP count to `Pools.Requested` and
  stop writing `Status.IPAM.Used`.
* 1.19 agents only write `Status.IPAM.Used`.

The dual-check handles the downgrade case: if a 1.20 agent wrote
`Pools.Requested` and was then rolled back to 1.19, causing the
`CiliumNode` to keep a stale `Pools.Requested` from its time under a
1.20 agent. The operator will properly handle that case by detecting
that the agent is now populating `Status.IPAM.Used` again and will
ignore `Pools.Requested`.

Relates to cilium/design-cfps#87

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to cilium/cilium that referenced this pull request Apr 1, 2026
This PR is the "double read" equivalent to #45110.

In the operator's `recalculate()`, detect whether the agent is using the
multi-pool or CRD allocator by checking `Spec.IPAM.Pools.Requested`
entries and `Status.IPAM.Used`:
* 1.20 agents write their total desired IP count to `Pools.Requested` and
  stop writing `Status.IPAM.Used`.
* 1.19 agents only write `Status.IPAM.Used`.

The dual-check handles the downgrade case: if a 1.20 agent wrote
`Pools.Requested` and was then rolled back to 1.19, causing the
`CiliumNode` to keep a stale `Pools.Requested` from its time under a
1.20 agent. The operator will properly handle that case by detecting
that the agent is now populating `Status.IPAM.Used` again and will
ignore `Pools.Requested`.

Relates to cilium/design-cfps#87

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to cilium/cilium that referenced this pull request Apr 1, 2026
This PR is the "double read" equivalent to #45110.

In the operator's `recalculate()`, detect whether the agent is using the
multi-pool or CRD allocator by checking `Spec.IPAM.Pools.Requested`
entries and `Status.IPAM.Used`:
* 1.20 agents write their total desired IP count to `Pools.Requested` and
  stop writing `Status.IPAM.Used`.
* 1.19 agents only write `Status.IPAM.Used`.

The dual-check handles the downgrade case: if a 1.20 agent wrote
`Pools.Requested` and was then rolled back to 1.19, causing the
`CiliumNode` to keep a stale `Pools.Requested` from its time under a
1.20 agent. The operator will properly handle that case by detecting
that the agent is now populating `Status.IPAM.Used` again and will
ignore `Pools.Requested`.

Relates to cilium/design-cfps#87

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to cilium/cilium that referenced this pull request Apr 1, 2026
This PR is the "double read" equivalent to #45110.

In the operator's `recalculate()`, detect whether the agent is using the
multi-pool or CRD allocator by checking `Spec.IPAM.Pools.Requested`
entries and `Status.IPAM.Used`:
* 1.20 agents write their total desired IP count to `Pools.Requested` and
  stop writing `Status.IPAM.Used`.
* 1.19 agents only write `Status.IPAM.Used`.

The dual-check handles the downgrade case: if a 1.20 agent wrote
`Pools.Requested` and was then rolled back to 1.19, causing the
`CiliumNode` to keep a stale `Pools.Requested` from its time under a
1.20 agent. The operator will properly handle that case by detecting
that the agent is now populating `Status.IPAM.Used` again and will
ignore `Pools.Requested`.

Relates to cilium/design-cfps#87

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
github-merge-queue bot pushed a commit to cilium/cilium that referenced this pull request Apr 3, 2026
In `syncToAPIServer`, populate `Spec.IPAM.Pools.Allocated` alongside
`Spec.IPAM.Pool` for nodes with ENI status. This enables the ENI
multi-pool migration (cilium/design-cfps#87): new agents (1.20) will read
CIDRs from `Pools.Allocated` (with the multipool allocator) while old
agents (1.19) will continue reading from `Pool` (with the CRD allocator).

Note: this PR only includes the double write logic on the operator side,
a followup PR will change the read path on the agent side.

For secondary IP mode, each IP is written as a /32 CIDR. For prefix
delegation mode, each /28 prefix is written directly alongside the
secondary IP /32s. All CIDRs are placed under the "default" pool.

The dual-write is low-frequency (fires on ENI capacity changes, not on
every pod event) and will be removed in 1.21. The dual write is also
atomic, both `Pool` and `Pools.Allocated` get written in the same
CilimNode update operation so they are garanteed to always be in sync
and consistent.

Relates to cilium/design-cfps#87

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to cilium/cilium that referenced this pull request Apr 3, 2026
This PR is the "double read" equivalent to #45110.

In the operator's `recalculate()`, detect whether the agent is using the
multi-pool or CRD allocator by checking `Spec.IPAM.Pools.Requested`
entries and `Status.IPAM.Used`:
* 1.20 agents write their total desired IP count to `Pools.Requested` and
  stop writing `Status.IPAM.Used`.
* 1.19 agents only write `Status.IPAM.Used`.

The dual-check handles the downgrade case: if a 1.20 agent wrote
`Pools.Requested` and was then rolled back to 1.19, causing the
`CiliumNode` to keep a stale `Pools.Requested` from its time under a
1.20 agent. The operator will properly handle that case by detecting
that the agent is now populating `Status.IPAM.Used` again and will
ignore `Pools.Requested`.

Relates to cilium/design-cfps#87

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
github-merge-queue bot pushed a commit to cilium/cilium that referenced this pull request Apr 3, 2026
This PR is the "double read" equivalent to #45110.

In the operator's `recalculate()`, detect whether the agent is using the
multi-pool or CRD allocator by checking `Spec.IPAM.Pools.Requested`
entries and `Status.IPAM.Used`:
* 1.20 agents write their total desired IP count to `Pools.Requested` and
  stop writing `Status.IPAM.Used`.
* 1.19 agents only write `Status.IPAM.Used`.

The dual-check handles the downgrade case: if a 1.20 agent wrote
`Pools.Requested` and was then rolled back to 1.19, causing the
`CiliumNode` to keep a stale `Pools.Requested` from its time under a
1.20 agent. The operator will properly handle that case by detecting
that the agent is now populating `Status.IPAM.Used` again and will
ignore `Pools.Requested`.

Relates to cilium/design-cfps#87

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to DataDog/cilium that referenced this pull request Apr 3, 2026
Replace the CRD allocator with the multi-pool allocator for ENI IPAM mode
on the agent side. Previous PRs cilium#45110 and cilium#45124 ensure the operator
already supports this new agent setup.

The new `eniMultiPoolAllocator` is a light wrapper on the standard
`multiPoolAllocator` that enriches `AllocationResult` with ENI-specific
required metadata via `buildENIAllocationResult` (see cilium#45089).

Key differences from the standard multi-pool allocator:
* `AllowFirstLastIPs` is enabled so /28 prefix delegation ranges are
  fully allocatable (see cilium#45025 and cilium#45082).
* `LinearPreAlloc` uses a simple `inUse + preAlloc` formula for demand
  computation instead of `neededIPCeil` rounding. This matches the CRD
  allocator's `calculateNeededIPs` semantics and is necessary to ensure
  the operator can recover the exact IP usage from the demand signal
  (requested - preAllocate) (see cilium#45124).
* No dependency on `CiliumPodIPPool` CRDs, pools are managed by the
  operator via Spec.IPAM.Pools.Allocated (see cilium#45110).

The agents now read CIDRs from `Spec.IPAM.Pools.Allocated`, allocate IPs
locally, and write aggregate demand to `Spec.IPAM.Pools.Requested`. They
no longer writes per-IP usage to `Status.IPAM.Used`, achieving a
reduction of kubernetes API pressure.

Relates to cilium/design-cfps#87

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to DataDog/cilium that referenced this pull request Apr 3, 2026
Replace the CRD allocator with the multi-pool allocator for ENI IPAM mode
on the agent side. Previous PRs cilium#45110 and cilium#45124 ensure the operator
already supports this new agent setup.

The new `eniMultiPoolAllocator` is a light wrapper on the standard
`multiPoolAllocator` that enriches `AllocationResult` with ENI-specific
required metadata via `buildENIAllocationResult` (see cilium#45089).

Key differences from the standard multi-pool allocator:
* `AllowFirstLastIPs` is enabled so /28 prefix delegation ranges are
  fully allocatable (see cilium#45025 and cilium#45082).
* `LinearPreAlloc` uses a simple `inUse + preAlloc` formula for demand
  computation instead of `neededIPCeil` rounding. This matches the CRD
  allocator's `calculateNeededIPs` semantics and is necessary to ensure
  the operator can recover the exact IP usage from the demand signal
  (requested - preAllocate) (see cilium#45124).
* No dependency on `CiliumPodIPPool` CRDs, pools are managed by the
  operator via Spec.IPAM.Pools.Allocated (see cilium#45110).

The agents now read CIDRs from `Spec.IPAM.Pools.Allocated`, allocate IPs
locally, and write aggregate demand to `Spec.IPAM.Pools.Requested`. They
no longer writes per-IP usage to `Status.IPAM.Used`, achieving a
reduction of kubernetes API pressure.

Relates to cilium/design-cfps#87

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to cilium/cilium that referenced this pull request Apr 3, 2026
Replace the CRD allocator with the multi-pool allocator for ENI IPAM mode
on the agent side. Previous PRs #45110 and #45124 ensure the operator
already supports this new agent setup.

The new `eniMultiPoolAllocator` is a light wrapper on the standard
`multiPoolAllocator` that enriches `AllocationResult` with ENI-specific
required metadata via `buildENIAllocationResult` (see #45089).

Key differences from the standard multi-pool allocator:
* `AllowFirstLastIPs` is enabled so /28 prefix delegation ranges are
  fully allocatable (see #45025 and #45082).
* `LinearPreAlloc` uses a simple `inUse + preAlloc` formula for demand
  computation instead of `neededIPCeil` rounding. This matches the CRD
  allocator's `calculateNeededIPs` semantics and is necessary to ensure
  the operator can recover the exact IP usage from the demand signal
  (requested - preAllocate) (see #45124).
* No dependency on `CiliumPodIPPool` CRDs, pools are managed by the
  operator via Spec.IPAM.Pools.Allocated (see #45110).

The agents now read CIDRs from `Spec.IPAM.Pools.Allocated`, allocate IPs
locally, and write aggregate demand to `Spec.IPAM.Pools.Requested`. They
no longer writes per-IP usage to `Status.IPAM.Used`, achieving a
reduction of kubernetes API pressure.

Relates to cilium/design-cfps#87

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to cilium/cilium that referenced this pull request Apr 4, 2026
Replace the CRD allocator with the multi-pool allocator for ENI IPAM mode
on the agent side. Previous PRs #45110 and #45124 ensure the operator
already supports this new agent setup.

The new `eniMultiPoolAllocator` is a light wrapper on the standard
`multiPoolAllocator` that enriches `AllocationResult` with ENI-specific
required metadata via `buildENIAllocationResult` (see #45089).

Key differences from the standard multi-pool allocator:
* `AllowFirstLastIPs` is enabled so /28 prefix delegation ranges are
  fully allocatable (see #45025 and #45082).
* `LinearPreAlloc` uses a simple `inUse + preAlloc` formula for demand
  computation instead of `neededIPCeil` rounding. This matches the CRD
  allocator's `calculateNeededIPs` semantics and is necessary to ensure
  the operator can recover the exact IP usage from the demand signal
  (requested - preAllocate) (see #45124).
* No dependency on `CiliumPodIPPool` CRDs, pools are managed by the
  operator via Spec.IPAM.Pools.Allocated (see #45110).

The agents now read CIDRs from `Spec.IPAM.Pools.Allocated`, allocate IPs
locally, and write aggregate demand to `Spec.IPAM.Pools.Requested`. They
no longer writes per-IP usage to `Status.IPAM.Used`, achieving a
reduction of kubernetes API pressure.

Relates to cilium/design-cfps#87

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to cilium/cilium that referenced this pull request Apr 4, 2026
Replace the CRD allocator with the multi-pool allocator for ENI IPAM mode
on the agent side. Previous PR #45124 ensures the operator already
supports this new agent setup.

The new `eniMultiPoolAllocator` is a light wrapper on the standard
`multiPoolAllocator` that enriches `AllocationResult` with ENI-specific
required metadata via `buildENIAllocationResult` (see #45089).

Key differences from the standard multi-pool allocator:
* `AllowFirstLastIPs` is enabled so /28 prefix delegation ranges are
  fully allocatable (see #45025 and #45082).
* `LinearPreAlloc` uses a simple `inUse + preAlloc` formula for demand
  computation instead of `neededIPCeil` rounding. This matches the CRD
  allocator's `calculateNeededIPs` semantics and is necessary to ensure
  the operator can recover the exact IP usage from the demand signal
  (requested - preAllocate) (see #45124).
* No dependency on `CiliumPodIPPool` CRDs, pools are instead populated
  by the agent from `Status.ENI.ENIs` which is maintained by the
  operator.

The agents now read CIDRs from `Status.ENI.ENIs`, allocate IPs locally,
and write aggregate demand to `Spec.IPAM.Pools.Requested`. They
no longer writes per-IP usage to `Status.IPAM.Used`, achieving a
reduction of kubernetes API pressure.

Relates to cilium/design-cfps#87

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to cilium/cilium that referenced this pull request Apr 4, 2026
Replace the CRD allocator with the multi-pool allocator for ENI IPAM mode
on the agent side. Previous PR #45124 ensures the operator already
supports this new agent setup.

The new `eniMultiPoolAllocator` is a light wrapper on the standard
`multiPoolAllocator` that enriches `AllocationResult` with ENI-specific
required metadata via `buildENIAllocationResult` (see #45089).

Key differences from the standard multi-pool allocator:
* `AllowFirstLastIPs` is enabled so /28 prefix delegation ranges are
  fully allocatable (see #45025 and #45082).
* `LinearPreAlloc` uses a simple `inUse + preAlloc` formula for demand
  computation instead of `neededIPCeil` rounding. This matches the CRD
  allocator's `calculateNeededIPs` semantics and is necessary to ensure
  the operator can recover the exact IP usage from the demand signal
  (requested - preAllocate) (see #45124).
* No dependency on `CiliumPodIPPool` CRDs, pools are instead populated
  by the agent from `Status.ENI.ENIs` which is maintained by the
  operator.

The agents now read CIDRs from `Status.ENI.ENIs`, allocate IPs locally,
and write aggregate demand to `Spec.IPAM.Pools.Requested`. They
no longer writes per-IP usage to `Status.IPAM.Used`, achieving a
reduction of kubernetes API pressure.

Relates to cilium/design-cfps#87

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
HadrienPatte added a commit to cilium/cilium that referenced this pull request Apr 5, 2026
Replace the CRD allocator with the multi-pool allocator for ENI IPAM mode
on the agent side. Previous PR #45124 ensures the operator already
supports this new agent setup.

The new `eniMultiPoolAllocator` is a light wrapper on the standard
`multiPoolAllocator` that enriches `AllocationResult` with ENI-specific
required metadata via `buildENIAllocationResult` (see #45089).

Key differences from the standard multi-pool allocator:
* `AllowFirstLastIPs` is enabled so /28 prefix delegation ranges are
  fully allocatable (see #45025 and #45082).
* `LinearPreAlloc` uses a simple `inUse + preAlloc` formula for demand
  computation instead of `neededIPCeil` rounding. This matches the CRD
  allocator's `calculateNeededIPs` semantics and is necessary to ensure
  the operator can recover the exact IP usage from the demand signal
  (requested - preAllocate) (see #45124).
* No dependency on `CiliumPodIPPool` CRDs, pools are instead populated
  by the agent from `Status.ENI.ENIs` which is maintained by the
  operator.

The agents now read CIDRs from `Status.ENI.ENIs`, allocate IPs locally,
and write aggregate demand to `Spec.IPAM.Pools.Requested`. They
no longer writes per-IP usage to `Status.IPAM.Used`, achieving a
reduction of kubernetes API pressure.

Relates to cilium/design-cfps#87

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants