Description

As documented in cilium/cilium#18706 there are two problems that lead to Cilium's routing policy rules getting lost. One is that the networkd default ManageForeignRoutingPolicyRules=yes discards Cilium's changes and there is no easy way for Cilium to annotate them (e.q., requires a dummy network interface), and the second is that the systemd 249 version we have in Stable has a bug that discards routing policies while it should only discard routes for the default ManageForeignRoutes=yes.

Impact

No IPv4 network connectivity, e.g., after a networkd reconfiguration due to the underlying interface of a bond having a state change.

Environment and steps to reproduce

See linked issue

Expected behavior

All works

Additional information

A workaround is to add the following to /etc/systemd/networkd.conf (under [Network]):

ManageForeignRoutes=no
ManageForeignRoutingPolicyRules=no

For Alpha only ManageForeignRoutingPolicyRules=no is needed. For Stable this only works when we ship a newer systemd 249 point release.

Since we can't expect Cilium to be able to tell this to networkd at runtime, we have to use ManageForeignRoutingPolicyRules=no as new default for Flatcar.

Maybe we should also use ManageForeignRoutes=no as new default but currently we don't have a use case profiting from it.