zebra, bgpd: EVPN VXLAN Multihome support with extern mode#19438
zebra, bgpd: EVPN VXLAN Multihome support with extern mode#19438mrin-g wants to merge 8 commits intoFRRouting:masterfrom
Conversation
|
This commit needs to be broken up into small logical units of work such that a reviewer can look at it and understand the logical steps that get you to this new feature. I have given some suggestions on how to do this already. I am sure that there are other places it can be broken up some. Additionally I see no documentation at all. This needs to be added too. I can review further once this is broken up |
dc5a881 to
a451fcd
Compare
e2285f0 to
559ccc8
Compare
f2b42df to
bf18b37
Compare
Its broken down now, with the zebra.rst file updated for the documentation on the new option. |
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
louis-6wind
left a comment
There was a problem hiding this comment.
a few comments, on the form for the moment
tests/topotests/bgp_evpn_mh_l2l3vni_ext_learn/test_evpn_mh_l2l3vni_ext_learn.py
Outdated
Show resolved
Hide resolved
tests/topotests/bgp_evpn_mh_l2l3vni_ext_learn/test_evpn_mh_l2l3vni_ext_learn.py
Show resolved
Hide resolved
tests/topotests/bgp_evpn_mh_l2l3vni_ext_learn/test_evpn_mh_l2l3vni_ext_learn.py
Outdated
Show resolved
Hide resolved
tests/topotests/bgp_evpn_mh_l2l3vni_ext_learn/test_evpn_mh_l2l3vni_ext_learn.py
Outdated
Show resolved
Hide resolved
tests/topotests/bgp_evpn_mh_l2l3vni_ext_learn/test_evpn_mh_l2l3vni_ext_learn.py
Outdated
Show resolved
Hide resolved
tests/topotests/bgp_evpn_mh_l2l3vni_ext_learn/test_evpn_mh_l2l3vni_ext_learn.py
Outdated
Show resolved
Hide resolved
tests/topotests/bgp_evpn_mh_l2l3vni_ext_learn/test_evpn_mh_l2l3vni_ext_learn.py
Outdated
Show resolved
Hide resolved
tests/topotests/bgp_evpn_mh_l2l3vni_ext_learn/test_evpn_mh_l2l3vni_ext_learn.py
Outdated
Show resolved
Hide resolved
tests/topotests/bgp_evpn_mh_l2l3vni_ext_learn/test_evpn_mh_l2l3vni_ext_learn.py
Outdated
Show resolved
Hide resolved
tests/topotests/bgp_evpn_mh_l2l3vni_ext_learn/test_evpn_mh_l2l3vni_ext_learn.py
Outdated
Show resolved
Hide resolved
bf18b37 to
9e33ccb
Compare
| case 'R': | ||
| rt_table_main_id = atoi(optarg); | ||
| break; | ||
| case OPTION_KERNEL_MAC_EXT_LEARN: |
There was a problem hiding this comment.
Recommendation: is it possible to detect and warn if the kernel does not support this flag, or fallback to standard learning.
There was a problem hiding this comment.
Yes, thats a good point. Presently if we see there is a mismatch between the mac_external mode and the kernel MAC update e.g https://github.com/FRRouting/frr/pull/19438/files#diff-c5ead220153b4b424c817d54e28ca166882af2a78e0958c106b585bf1ec652ecR4176 , we drop the entry. So no local MAC learning will happen without external flag in mac external mode.
But presently its all config based and no operational mode is maintained, however the only tricky part to this dynamic mode, is if we do the evaluation per MAC basis; any bug in the underlying mac learning infra can cause flip flop between the modes. Would love to discuss your idea in a huddle.
e60b6ee to
a3f51d1
Compare
|
@mrin-g it seems you have not replied to Ido Schimmel's concerns in https://lore.kernel.org/netdev/20250818175258.275997-1-mrghosh@cisco.com/T/#m9c3757ae08af5aac2ccf890f4e17ecfcfa888e37 , have you ? The feature has to be first accepted by the kernel maintainers. And assuming it is accepted how do you manage the retro compatibility with old kernels ? |
tests/topotests/bgp_evpn_mh_l2l3vni_ext_learn/test_evpn_mh_l2l3vni_ext_learn.py
Outdated
Show resolved
Hide resolved
tests/topotests/bgp_evpn_mh_l2l3vni_ext_learn/test_evpn_mh_l2l3vni_ext_learn.py
Outdated
Show resolved
Hide resolved
tests/topotests/bgp_evpn_mh_l2l3vni_ext_learn/test_evpn_mh_l2l3vni_ext_learn.py
Show resolved
Hide resolved
1bdd568 to
d6f9074
Compare
|
This change depends on a kernel change. Has it been accepted yet? |
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
Introduction of "--kernel-mac-ext-learn" mode of operation in zebra.
This is part of the change is to enable the
option '--kernel-mac-ext-learn' in zebra.
Rationale:
EVPN Multihoming External Mode Support is to enable platforms doing
Hardware Based MAC Learning and Aging to support EVPN VXLAN Multihome.
MAC's in this mode for both data and control plane will be
marked and programmed as 'extern_only' in kernel, so that kernel aging
is disabled for these MACs. Zebra along with Hardware(HW) will control
these MACs for control plane and data plane respectively.
Per File change summary:
zebra/main.c:
Add option '--kernel-mac-ext-learn' in zebra startup.
Alignment changes for the other options to match the new option width
and keep description alignment consistent
zebra/zebra_router.c:
- zebra_router_init definition updation to pass
kernel_mac_ext_learn mode
- store kernel_mac_ext_learn in zrouter zav structure
zebra/zebra_router.h:
- new bool field 'kernel_mac_ext_learn' in zebra_router, to
store extern only mode
zebra/zebra_vxlan.h:
- Accessor for Extern Only mode i.e zebra_mac_ext_learn_mode
Signed-off-by: Mrinmoy Ghosh <mrinmoy_g@hotmail.com>
Signed-off-by: Mrinmoy Ghosh <mrghosh@cisco.com>
Signed-off-by: Patrice Brissette <pbrisset@cisco.com>
Protocol field is added in bridge FDB, to distinguish between MAC addresses learned via the control plane and those learned via the data plane with hardware aging. Protocol 'hw' (i.e RTPROT_HW aka hardware) for MAC learnt by hardware will be used for data plane(hardware) learnt MAC while existing protocol 'zebra' to be used for control plane learnt ones. Kernel Patch in review: https://lore.kernel.org/netdev/20250818175258.275997-1-mrghosh@cisco.com/ iproute2 patch in review: https://lore.kernel.org/netdev/20250818193756.277327-1-mrghosh@cisco.com/ Signed-off-by: Mrinmoy Ghosh <mrinmoy_g@hotmail.com> Signed-off-by: Mrinmoy Ghosh <mrghosh@cisco.com> Signed-off-by: Patrice Brissette <pbrisset@cisco.com>
Doc update for new zebra option '--kernel-mac-ext-learn' Signed-off-by: Mrinmoy Ghosh <mrinmoy_g@hotmail.com> Signed-off-by: Mrinmoy Ghosh <mrghosh@cisco.com>
This is to update netlink debug print, to print 'protocol' i.e NDA_PROTOCOL field for netlink messages. Signed-off-by: Mrinmoy Ghosh <mrinmoy_g@hotmail.com> Signed-off-by: Mrinmoy Ghosh <mrghosh@cisco.com>
For MAC External mode, presently ARP/ND Suppression is not supported, thereby disabled. Remote Neighbor programming is also not required, based on zebra_mac_ext_learn_mode. Signed-off-by: Mrinmoy Ghosh <mrinmoy_g@hotmail.com> Signed-off-by: Mrinmoy Ghosh <mrghosh@cisco.com> Signed-off-by: Patrice Brissette <pbrisset@cisco.com>
Receive handling: - Debounce of AF_BRIDGE netlink message based on 'protocol' field ZEBRA - Update zlog to include protocol - Ignore feedback netlink from zebra based on protocol zebra - Ignore if "NTF_EXT_LEARNED" flag is not marked in the netlink message for extern only mode of operation Send Operation: - Always mark MAC as NTF_EXT_LEARNED in extern only mode - Mark Sync MAC as NTF_E_MH_PEER_SYNC, only in extern only mode - Peer-Sync flag (i.e NTF_E_MH_PEER_SYNC) state print in Tx zlog Signed-off-by: Mrinmoy Ghosh <mrinmoy_g@hotmail.com> Signed-off-by: Mrinmoy Ghosh <mrghosh@cisco.com> Signed-off-by: Patrice Brissette <pbrisset@cisco.com>
Topotests added under bgp_evpn_mh_l2l3vni_ext_learn to: - Validate correct ES discovery and advertisement for both local and remote PEs. - Check VTEP peer lists for accuracy, including handling of downed VTEPs and ES state transitions. - Ensure L2VNI and L3VNI are correctly instantiated and associated with the appropriate VRFs and VXLAN interfaces. - Test orphaned hosts, dual-attached hosts, and single-attached hosts in various failure and recovery scenarios. - MAC 'protocol' state transitions i.e data plane learnt to control plane learnt and vice versa, delete, relearn in both peers sequeunces Utility and Parser Functions: Utility functions in lib/bgp_evpn.py(new) are added. These changes adds the test coverage and reliability for EVPN VXLAN multihoming in external mode, making it easier to detect regressions and validate new features. Signed-off-by: Mrinmoy Ghosh <mrinmoy_g@hotmail.com> Signed-off-by: Mrinmoy Ghosh <mrghosh@cisco.com> Signed-off-by: Patrice Brissette <pbrisset@cisco.com>
flog_err added for evpn install to zebra failure for ip and mac. Signed-off-by: Mrinmoy Ghosh <mrinmoy_g@hotmail.com> Signed-off-by: Mrinmoy Ghosh <mrghosh@cisco.com> Signed-off-by: Mike Mallin <mmallin@cisco.com>
d6f9074 to
b135fdf
Compare
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
Introduction of "--kernel-mac-ext-learn" mode of operation in zebra.
This delivers a comprehensive set of enhancements and fixes to support and validate EVPN VXLAN multihoming in external learning mode.
EVPN Multihoming External Mode Support:
Test Enhancements and New Scenarios:
The test framework is updated to cover EVPN VXLAN multihoming with both L2VNI and L3VNI, specifically targeting external learning mode. This includes new logic to handle and verify the correct operation of Ethernet Segments (ES), Designated Forwarder (DF) election, and VTEP peer relationships.
Added and updated topotests under bgp_evpn_mh_l2l3vni_ext_learn to: Validate correct ES discovery and advertisement for both local and remote PEs. Check VTEP peer lists for accuracy, including handling of downed VTEPs and ES state transitions. Ensure L2VNI and L3VNI are correctly instantiated and associated with the appropriate VRFs and VXLAN interfaces. Test orphaned hosts, dual-attached hosts, and single-attached hosts in various failure and recovery scenarios. Utility and Parser Functions:
Utility functions in lib/bgp_evpn.py(new) is added, such as a parser for the show evpn vni json command. This parser extracts VNI-to-type mappings, simplifying validation of VNI configuration and type (L2/L3) in automated tests. These changes adds the test coverage and reliability for EVPN VXLAN multihoming in external mode, making it easier to detect regressions and validate new features.
Per File change summary:
bgpd/bgp_evpn.c:
Addition of flog_err for error in installation of VNI MAC and IP to zebra. Non functional.
include/linux/rtnetlink.h:
RTPROT_HW definition as 'protocol' in bridge fdb
zebra/debug_nl.c:
Updated Netlink print infra to include 'protocol' field in netlink msg dump
zebra/main.c:
Add option '--kernel-ext-learn' in zebra startup.
zebra/rt_netlink.c:
Debounce/Accept MAC update from kernel based on 'protocol' field (i.e NDA_PROTOCOL)
Add ext_flags e.g NTF_E_MH_PEER_SYNC as Peer Sync for 'extern_only' mode.
Add NTF_EXT_LEARNED flag for MAC in dplane programming, also drop for netlink messages from kernel without the flag
zebra/zebra_evpn.c:
ARP ND Suppression 'set' and 'get' function definition
zebra/zebra_evpn.h:
Corresponding, ARP ND Suppression 'set' and 'get' function declaration
zebra/zebra_evpn_mac.c:
zebra_evpn_sync_mac_dp_install: For Sync MAC DPLANE installation, make sure the MAC is inactive for extern mode
In zebra_evpn_mac_hold_exp_cb: In extern only mode, explicit flush of the mac is required
In zebra_evpn_sync_mac_del: In extern mode, only reprogram MACif it does not have any PEER flags or neighbor and LOCAL INACTIVE. This helps to clear up the "static" for PEER_PROXY on proxy withdrawal
zebra/zebra_evpn_mh.c:
ES Info Set: In extern only mode, use the API zebra_evpn_es_bypass_update_macs to clean MAC from interfaces and ES
zebra/zebra_evpn_mh.h:
Declare API zebra_evpn_flush_local_mac as its used in zebra_evpn_mac.c for hold timer expiry cleanup and sync delete of MAC
zebra/zebra_evpn_neigh.c:
Prevent neighbor entry installation n DPLANE for remote neighbor if ARP/ND suppression is not enabled. For Extern only mode, ARP/ND Supression is not enabled
zebra/zebra_router.c:
For extern only mode, disable ARP suppression
zebra/zebra_router.h:
new bool field 'kernel_mac_ext_learn' in zebra_router, to store extern only mode
zebra/zebra_vxlan.c:
Add print for EVPN ARP/ND Suppression in "show evpn"
zebra/zebra_vxlan.h:
Accessor for Extern Only mode i.e zebra_mac_ext_learn_mode
Review for Kernel Change for Protocol field Addition: https://lore.kernel.org/netdev/20250818175258.275997-1-mrghosh@cisco.com/T/#u
Review for iproute2 change for protocol field Addition: https://lore.kernel.org/netdev/20250818193756.277327-1-mrghosh@cisco.com/T/#u