Skip to content

*: PBR - netlink interaction and basic definitions#1

Closed
vivek-cumulus wants to merge 402 commits intodonaldsharp:masterfrom
vivek-cumulus:PBRD
Closed

*: PBR - netlink interaction and basic definitions#1
vivek-cumulus wants to merge 402 commits intodonaldsharp:masterfrom
vivek-cumulus:PBRD

Conversation

@vivek-cumulus
Copy link
Copy Markdown

No description provided.

rzalamena and others added 30 commits December 14, 2017 14:04
Handle the return value of argv_find_and_parse_afi() to avoid passing
along bad values.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
…ian-file

debianpkg: Add missing frr-dbg.lintian-overrides to Makefile
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya<mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.comy>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
Signed-off-by: Mitesh Kanjariya <mitesh@cumulusnetworks.com>
@vivek-cumulus vivek-cumulus deleted the PBRD branch May 25, 2020 19:52
donaldsharp pushed a commit that referenced this pull request Jun 26, 2020
=================================================================
==24764==ERROR: AddressSanitizer: heap-use-after-free on address 0x60d0000115c8 at pc 0x55cb9cfad312 bp 0x7fffa0552140 sp 0x7fffa0552138
READ of size 8 at 0x60d0000115c8 thread T0
    #0 0x55cb9cfad311 in zebra_evpn_remote_es_flush zebra/zebra_evpn_mh.c:2041
    #1 0x55cb9cfad311 in zebra_evpn_es_cleanup zebra/zebra_evpn_mh.c:2234
    #2 0x55cb9cf6ae78 in zebra_vrf_disable zebra/zebra_vrf.c:205
    #3 0x7fc8d478f114 in vrf_delete lib/vrf.c:229
    #4 0x7fc8d478f99a in vrf_terminate lib/vrf.c:541
    #5 0x55cb9ceba0af in sigint zebra/main.c:176
    #6 0x55cb9ceba0af in sigint zebra/main.c:130
    #7 0x7fc8d4765d20 in quagga_sigevent_process lib/sigevent.c:103
    #8 0x7fc8d4787e8c in thread_fetch lib/thread.c:1396
    #9 0x7fc8d4708782 in frr_run lib/libfrr.c:1092
    #10 0x55cb9ce931d8 in main zebra/main.c:488
    #11 0x7fc8d43ee09a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409a)
    #12 0x55cb9ce94c09 in _start (/usr/lib/frr/zebra+0x8ac09)
=================================================================

Signed-off-by: Anuradha Karuppiah <anuradhak@cumulusnetworks.com>
donaldsharp pushed a commit that referenced this pull request Sep 10, 2020
There are two changes in this commit -

1. Maintain a list of global MAC-IP routes per-ES. This list is maintained
for quick processing on the following events -
a. When the first VTEP/PE becomes active in the ES-VRF, the L3 NHG is
activated and the route can be sent to zebra.
b. When there are no active PEs in the ES-VRF the L3 NHG is
de-activated and -
- If the ES is present in the VRF -
The route is not installed in zebra as there are no active PEs for
the ES-VRF
- If the ES is not present in the VRF -
The route is installed with a flat multi-path list i.e. without L3NHG.
This is to handle the case where there are no locally attached L2VNIs
on the ES (for that tenant VRF).

2. Reinstall VRF route when an ES is installed or uninstalled in a
tenant VRF (the global MAC-IP list in #1 is used for this purpose also).
If an ES is present in the VRF we use L3NHG to enable fast-failover of
routed traffic.

Signed-off-by: Anuradha Karuppiah <anuradhak@cumulusnetworks.com>
donaldsharp added a commit that referenced this pull request Sep 16, 2020
Current code in bgp bestpath selection would accept the newest
locally originated path as the best path.  Making the selection
non-deterministic.  Modify the code to always come to the
same bestpath conclusion when you have multiple locally originated
paths in bestpath selection.

Before:

eva# conf
eva(config)# router bgp 323
eva(config-router)# address-family ipv4 uni
eva(config-router-af)# redistribute connected
eva(config-router-af)# network 192.168.161.0/24
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:02:52 2020
eva(config-router-af)# no redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (First path received)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#  redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #2, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:03:32 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#

Notice the route choosen depends on order received

Fixed behavior:

eva# conf
eva(config)# router bgp 323
eva(config-router)# address-family ipv4 uni
eva(config-router-af)# redistribute connected
eva(config-router-af)# network 192.168.161.0/24
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:02:52 2020
eva(config-router-af)# no redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (First path received)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#  redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #2, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:03:32 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#

Ticket: CM-31490
Found-by: Trey Aspelund <taspelund@nvidia.com>
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp pushed a commit that referenced this pull request Sep 25, 2020
This problem was reported by the sanitizer -
=================================================================
==24764==ERROR: AddressSanitizer: heap-use-after-free on address 0x60d0000115c8 at pc 0x55cb9cfad312 bp 0x7fffa0552140 sp 0x7fffa0552138
READ of size 8 at 0x60d0000115c8 thread T0
    #0 0x55cb9cfad311 in zebra_evpn_remote_es_flush zebra/zebra_evpn_mh.c:2041
    #1 0x55cb9cfad311 in zebra_evpn_es_cleanup zebra/zebra_evpn_mh.c:2234
    #2 0x55cb9cf6ae78 in zebra_vrf_disable zebra/zebra_vrf.c:205
    #3 0x7fc8d478f114 in vrf_delete lib/vrf.c:229
    #4 0x7fc8d478f99a in vrf_terminate lib/vrf.c:541
    #5 0x55cb9ceba0af in sigint zebra/main.c:176
    #6 0x55cb9ceba0af in sigint zebra/main.c:130
    #7 0x7fc8d4765d20 in quagga_sigevent_process lib/sigevent.c:103
    #8 0x7fc8d4787e8c in thread_fetch lib/thread.c:1396
    #9 0x7fc8d4708782 in frr_run lib/libfrr.c:1092
    #10 0x55cb9ce931d8 in main zebra/main.c:488
    #11 0x7fc8d43ee09a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409a)
    #12 0x55cb9ce94c09 in _start (/usr/lib/frr/zebra+0x8ac09)
=================================================================

Signed-off-by: Anuradha Karuppiah <anuradhak@cumulusnetworks.com>
donaldsharp added a commit that referenced this pull request Sep 30, 2020
Current code in bgp bestpath selection would accept the newest
locally originated path as the best path.  Making the selection
non-deterministic.  Modify the code to always come to the
same bestpath conclusion when you have multiple locally originated
paths in bestpath selection.

Before:

eva# conf
eva(config)# router bgp 323
eva(config-router)# address-family ipv4 uni
eva(config-router-af)# redistribute connected
eva(config-router-af)# network 192.168.161.0/24
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:02:52 2020
eva(config-router-af)# no redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (First path received)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#  redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #2, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:03:32 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#

Notice the route choosen depends on order received

Fixed behavior:

eva# conf
eva(config)# router bgp 323
eva(config-router)# address-family ipv4 uni
eva(config-router-af)# redistribute connected
eva(config-router-af)# network 192.168.161.0/24
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:02:52 2020
eva(config-router-af)# no redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (First path received)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#  redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #2, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:03:32 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#

Ticket: CM-31490
Found-by: Trey Aspelund <taspelund@nvidia.com>
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp added a commit that referenced this pull request Oct 1, 2020
Current code in bgp bestpath selection would accept the newest
locally originated path as the best path.  Making the selection
non-deterministic.  Modify the code to always come to the
same bestpath conclusion when you have multiple locally originated
paths in bestpath selection.

Before:

eva# conf
eva(config)# router bgp 323
eva(config-router)# address-family ipv4 uni
eva(config-router-af)# redistribute connected
eva(config-router-af)# network 192.168.161.0/24
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:02:52 2020
eva(config-router-af)# no redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (First path received)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#  redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #2, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:03:32 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#

Notice the route choosen depends on order received

Fixed behavior:

eva# conf
eva(config)# router bgp 323
eva(config-router)# address-family ipv4 uni
eva(config-router-af)# redistribute connected
eva(config-router-af)# network 192.168.161.0/24
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:02:52 2020
eva(config-router-af)# no redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (First path received)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#  redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #2, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:03:32 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#

Ticket: CM-31490
Found-by: Trey Aspelund <taspelund@nvidia.com>
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp added a commit that referenced this pull request Oct 2, 2020
Current code in bgp bestpath selection would accept the newest
locally originated path as the best path.  Making the selection
non-deterministic.  Modify the code to always come to the
same bestpath conclusion when you have multiple locally originated
paths in bestpath selection.

Before:

eva# conf
eva(config)# router bgp 323
eva(config-router)# address-family ipv4 uni
eva(config-router-af)# redistribute connected
eva(config-router-af)# network 192.168.161.0/24
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:02:52 2020
eva(config-router-af)# no redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (First path received)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#  redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #2, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:03:32 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#

Notice the route choosen depends on order received

Fixed behavior:

eva# conf
eva(config)# router bgp 323
eva(config-router)# address-family ipv4 uni
eva(config-router-af)# redistribute connected
eva(config-router-af)# network 192.168.161.0/24
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:02:52 2020
eva(config-router-af)# no redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (First path received)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#  redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #2, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:03:32 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#

Ticket: CM-31490
Found-by: Trey Aspelund <taspelund@nvidia.com>
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp added a commit that referenced this pull request Oct 6, 2020
Current code in bgp bestpath selection would accept the newest
locally originated path as the best path.  Making the selection
non-deterministic.  Modify the code to always come to the
same bestpath conclusion when you have multiple locally originated
paths in bestpath selection.

Before:

eva# conf
eva(config)# router bgp 323
eva(config-router)# address-family ipv4 uni
eva(config-router-af)# redistribute connected
eva(config-router-af)# network 192.168.161.0/24
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:02:52 2020
eva(config-router-af)# no redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (First path received)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#  redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #2, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:03:32 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#

Notice the route choosen depends on order received

Fixed behavior:

eva# conf
eva(config)# router bgp 323
eva(config-router)# address-family ipv4 uni
eva(config-router-af)# redistribute connected
eva(config-router-af)# network 192.168.161.0/24
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:02:52 2020
eva(config-router-af)# no redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (First path received)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#  redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #2, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:03:32 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#

Ticket: CM-31490
Found-by: Trey Aspelund <taspelund@nvidia.com>
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp added a commit that referenced this pull request Oct 7, 2020
The bgp_l3vpn_to_bgp_vrf test is looking for a prefix
on multiple routers that the ordered received is non-deterministic.
As such the regex's are failing occassionaly when the
route is received in an unexpected order.

One possible order:
(FRRouting#89) scripts/check_routes.py:120 COMMAND:ce3:vtysh -c "show bgp ipv4 uni 6.0.1.0":2 available, best .*192.168.1.1.* Local.* 99.0.0.3 from 0.0.0.0 .99.0.0.3.* Origin IGP, metric 200, localpref 50, weight 32768, valid, sourced, local, best .Weight.* Community: 0:67.* Extended Community: RT:89:123.* Large Community: 12:34:56.* Local.* 192.168.1.1 from 192.168.1.1 .192.168.1.1.* Origin IGP, metric 98, localpref 123, valid, internal.* Community: 0:67.* Extended Community: RT:52:100 RT:89:123.* Large Community: 12:34:56:pass:Redundant route 1 details c:
COMMAND OUTPUT:BGP routing table entry for 6.0.1.0/24^M
Paths: (2 available, best #1, table default)^M
  Advertised to non peer-group peers:^M
  192.168.1.1^M
  Local^M
    99.0.0.3 from 0.0.0.0 (99.0.0.3)^M
      Origin IGP, metric 200, localpref 50, weight 32768, valid, sourced, local, best (Weight)^M
      Community: 0:67^M
      Extended Community: RT:89:123^M
      Large Community: 12:34:56^M
      Last update: Wed Oct  7 11:12:22 2020^M
  Local^M
    192.168.1.1 from 192.168.1.1 (192.168.1.1)^M
      Origin IGP, metric 98, localpref 123, valid, internal^M
      Community: 0:67^M
      Extended Community: RT:52:100 RT:89:123^M
      Large Community: 12:34:56^M
      Last update: Wed Oct  7 11:12:41 2020:
R:89   ce3    Redundant route 1 details c                              1    0

Second possible order:
(FRRouting#89) scripts/check_routes.py:120 COMMAND:ce3:vtysh -c "show bgp ipv4 uni 6.0.1.0":2 available, best .*192.168.1.1.* Local.* 99.0.0.3 from 0.0.0.0 .99.0.0.3.* Origin IGP, metric 200, localpref 50, weight 32768, valid, sourced, local, best .Weight.* Community: 0:67.* Extended Community: RT:89:123.* Large Community: 12:34:56.* Local.* 192.168.1.1 from 192.168.1.1 .192.168.1.1.* Origin IGP, metric 98, localpref 123, valid, internal.* Community: 0:67.* Extended Community: RT:52:100 RT:89:123.* Large Community: 12:34:56:pass:Redundant route 1 details c:
COMMAND OUTPUT:BGP routing table entry for 6.0.1.0/24^M
Paths: (2 available, best #2, table default)^M
  Advertised to non peer-group peers:^M
  192.168.1.1^M
  Local^M
    192.168.1.1 from 192.168.1.1 (192.168.1.1)^M
      Origin IGP, metric 98, localpref 123, valid, internal^M
      Community: 0:67^M
      Extended Community: RT:52:100 RT:89:123^M
      Large Community: 12:34:56^M
      Last update: Wed Oct  7 11:14:45 2020^M
  Local^M
    99.0.0.3 from 0.0.0.0 (99.0.0.3)^M
      Origin IGP, metric 200, localpref 50, weight 32768, valid, sourced, local, best (Weight)^M
      Community: 0:67^M
      Extended Community: RT:89:123^M
      Large Community: 12:34:56^M
      Last update: Wed Oct  7 11:14:27 2020:
R:89   ce3    Redundant route 1 details c                              0    1

BGP displays the paths in the order received since it's just a linked list.
For this test modify/add the luCommands to track that we may
receive the paths in a non-deterministic order.

Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp added a commit that referenced this pull request Oct 8, 2020
The bgp_l3vpn_to_bgp_vrf test is looking for a prefix
on multiple routers that the ordered received is non-deterministic.
As such the regex's are failing occassionaly when the
route is received in an unexpected order.

One possible order:
(FRRouting#89) scripts/check_routes.py:120 COMMAND:ce3:vtysh -c "show bgp ipv4 uni 6.0.1.0":2 available, best .*192.168.1.1.* Local.* 99.0.0.3 from 0.0.0.0 .99.0.0.3.* Origin IGP, metric 200, localpref 50, weight 32768, valid, sourced, local, best .Weight.* Community: 0:67.* Extended Community: RT:89:123.* Large Community: 12:34:56.* Local.* 192.168.1.1 from 192.168.1.1 .192.168.1.1.* Origin IGP, metric 98, localpref 123, valid, internal.* Community: 0:67.* Extended Community: RT:52:100 RT:89:123.* Large Community: 12:34:56:pass:Redundant route 1 details c:
COMMAND OUTPUT:BGP routing table entry for 6.0.1.0/24^M
Paths: (2 available, best #1, table default)^M
  Advertised to non peer-group peers:^M
  192.168.1.1^M
  Local^M
    99.0.0.3 from 0.0.0.0 (99.0.0.3)^M
      Origin IGP, metric 200, localpref 50, weight 32768, valid, sourced, local, best (Weight)^M
      Community: 0:67^M
      Extended Community: RT:89:123^M
      Large Community: 12:34:56^M
      Last update: Wed Oct  7 11:12:22 2020^M
  Local^M
    192.168.1.1 from 192.168.1.1 (192.168.1.1)^M
      Origin IGP, metric 98, localpref 123, valid, internal^M
      Community: 0:67^M
      Extended Community: RT:52:100 RT:89:123^M
      Large Community: 12:34:56^M
      Last update: Wed Oct  7 11:12:41 2020:
R:89   ce3    Redundant route 1 details c                              1    0

Second possible order:
(FRRouting#89) scripts/check_routes.py:120 COMMAND:ce3:vtysh -c "show bgp ipv4 uni 6.0.1.0":2 available, best .*192.168.1.1.* Local.* 99.0.0.3 from 0.0.0.0 .99.0.0.3.* Origin IGP, metric 200, localpref 50, weight 32768, valid, sourced, local, best .Weight.* Community: 0:67.* Extended Community: RT:89:123.* Large Community: 12:34:56.* Local.* 192.168.1.1 from 192.168.1.1 .192.168.1.1.* Origin IGP, metric 98, localpref 123, valid, internal.* Community: 0:67.* Extended Community: RT:52:100 RT:89:123.* Large Community: 12:34:56:pass:Redundant route 1 details c:
COMMAND OUTPUT:BGP routing table entry for 6.0.1.0/24^M
Paths: (2 available, best #2, table default)^M
  Advertised to non peer-group peers:^M
  192.168.1.1^M
  Local^M
    192.168.1.1 from 192.168.1.1 (192.168.1.1)^M
      Origin IGP, metric 98, localpref 123, valid, internal^M
      Community: 0:67^M
      Extended Community: RT:52:100 RT:89:123^M
      Large Community: 12:34:56^M
      Last update: Wed Oct  7 11:14:45 2020^M
  Local^M
    99.0.0.3 from 0.0.0.0 (99.0.0.3)^M
      Origin IGP, metric 200, localpref 50, weight 32768, valid, sourced, local, best (Weight)^M
      Community: 0:67^M
      Extended Community: RT:89:123^M
      Large Community: 12:34:56^M
      Last update: Wed Oct  7 11:14:27 2020:
R:89   ce3    Redundant route 1 details c                              0    1

BGP displays the paths in the order received since it's just a linked list.
For this test modify/add the luCommands to track that we may
receive the paths in a non-deterministic order.

Signed-off-by: Donald Sharp <sharpd@nvidia.com>
qlyoung added a commit that referenced this pull request Oct 8, 2020
The bgp_l3vpn_to_bgp_vrf test is looking for a prefix
on multiple routers that the ordered received is non-deterministic.
As such the regex's are failing occassionaly when the
route is received in an unexpected order.

One possible order:
(FRRouting#89) scripts/check_routes.py:120 COMMAND:ce3:vtysh -c "show bgp ipv4 uni 6.0.1.0":2 available, best .*192.168.1.1.* Local.* 99.0.0.3 from 0.0.0.0 .99.0.0.3.* Origin IGP, metric 200, localpref 50, weight 32768, valid, sourced, local, best .Weight.* Community: 0:67.* Extended Community: RT:89:123.* Large Community: 12:34:56.* Local.* 192.168.1.1 from 192.168.1.1 .192.168.1.1.* Origin IGP, metric 98, localpref 123, valid, internal.* Community: 0:67.* Extended Community: RT:52:100 RT:89:123.* Large Community: 12:34:56:pass:Redundant route 1 details c:
COMMAND OUTPUT:BGP routing table entry for 6.0.1.0/24^M
Paths: (2 available, best #1, table default)^M
  Advertised to non peer-group peers:^M
  192.168.1.1^M
  Local^M
    99.0.0.3 from 0.0.0.0 (99.0.0.3)^M
      Origin IGP, metric 200, localpref 50, weight 32768, valid, sourced, local, best (Weight)^M
      Community: 0:67^M
      Extended Community: RT:89:123^M
      Large Community: 12:34:56^M
      Last update: Wed Oct  7 11:12:22 2020^M
  Local^M
    192.168.1.1 from 192.168.1.1 (192.168.1.1)^M
      Origin IGP, metric 98, localpref 123, valid, internal^M
      Community: 0:67^M
      Extended Community: RT:52:100 RT:89:123^M
      Large Community: 12:34:56^M
      Last update: Wed Oct  7 11:12:41 2020:
R:89   ce3    Redundant route 1 details c                              1    0

Second possible order:
(FRRouting#89) scripts/check_routes.py:120 COMMAND:ce3:vtysh -c "show bgp ipv4 uni 6.0.1.0":2 available, best .*192.168.1.1.* Local.* 99.0.0.3 from 0.0.0.0 .99.0.0.3.* Origin IGP, metric 200, localpref 50, weight 32768, valid, sourced, local, best .Weight.* Community: 0:67.* Extended Community: RT:89:123.* Large Community: 12:34:56.* Local.* 192.168.1.1 from 192.168.1.1 .192.168.1.1.* Origin IGP, metric 98, localpref 123, valid, internal.* Community: 0:67.* Extended Community: RT:52:100 RT:89:123.* Large Community: 12:34:56:pass:Redundant route 1 details c:
COMMAND OUTPUT:BGP routing table entry for 6.0.1.0/24^M
Paths: (2 available, best #2, table default)^M
  Advertised to non peer-group peers:^M
  192.168.1.1^M
  Local^M
    192.168.1.1 from 192.168.1.1 (192.168.1.1)^M
      Origin IGP, metric 98, localpref 123, valid, internal^M
      Community: 0:67^M
      Extended Community: RT:52:100 RT:89:123^M
      Large Community: 12:34:56^M
      Last update: Wed Oct  7 11:14:45 2020^M
  Local^M
    99.0.0.3 from 0.0.0.0 (99.0.0.3)^M
      Origin IGP, metric 200, localpref 50, weight 32768, valid, sourced, local, best (Weight)^M
      Community: 0:67^M
      Extended Community: RT:89:123^M
      Large Community: 12:34:56^M
      Last update: Wed Oct  7 11:14:27 2020:
R:89   ce3    Redundant route 1 details c                              0    1

BGP displays the paths in the order received since it's just a linked list.
For this test modify/add the luCommands to track that we may
receive the paths in a non-deterministic order.

Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Signed-off-by: Quentin Young <qlyoung@nvidia.com>
donaldsharp added a commit that referenced this pull request Oct 9, 2020
Current code in bgp bestpath selection would accept the newest
locally originated path as the best path.  Making the selection
non-deterministic.  Modify the code to always come to the
same bestpath conclusion when you have multiple locally originated
paths in bestpath selection.

Before:

eva# conf
eva(config)# router bgp 323
eva(config-router)# address-family ipv4 uni
eva(config-router-af)# redistribute connected
eva(config-router-af)# network 192.168.161.0/24
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:02:52 2020
eva(config-router-af)# no redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (First path received)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#  redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #2, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:03:32 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#

Notice the route choosen depends on order received

Fixed behavior:

eva# conf
eva(config)# router bgp 323
eva(config-router)# address-family ipv4 uni
eva(config-router-af)# redistribute connected
eva(config-router-af)# network 192.168.161.0/24
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:02:52 2020
eva(config-router-af)# no redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (First path received)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#  redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #2, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:03:32 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#

Ticket: CM-31490
Found-by: Trey Aspelund <taspelund@nvidia.com>
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp added a commit that referenced this pull request Oct 16, 2020
When zebra is running with debugs turned on there
is a use after free reported by the address sanitizer:

2020/10/16 12:58:02 ZEBRA: rib_delnode: (0:254):4.5.6.16/32: rn 0x60b000026f20, re 0x6080000131a0, removing
2020/10/16 12:58:02 ZEBRA: rib_meta_queue_add: (0:254):4.5.6.16/32: queued rn 0x60b000026f20 into sub-queue 3
=================================================================
==3101430==ERROR: AddressSanitizer: heap-use-after-free on address 0x608000011d28 at pc 0x555555705ab6 bp 0x7fffffffdab0 sp 0x7fffffffdaa8
READ of size 8 at 0x608000011d28 thread T0
    #0 0x555555705ab5 in re_list_const_first zebra/rib.h:222
    #1 0x555555705b54 in re_list_first zebra/rib.h:222
    #2 0x555555711a4f in process_subq_route zebra/zebra_rib.c:2248
    #3 0x555555711d2e in process_subq zebra/zebra_rib.c:2286
    #4 0x555555711ec7 in meta_queue_process zebra/zebra_rib.c:2320
    #5 0x7ffff74701f7 in work_queue_run lib/workqueue.c:291
    #6 0x7ffff7450e9c in thread_call lib/thread.c:1581
    #7 0x7ffff738eaf7 in frr_run lib/libfrr.c:1099
    #8 0x55555561a578 in main zebra/main.c:455
    #9 0x7ffff7079cc9 in __libc_start_main ../csu/libc-start.c:308
    #10 0x5555555e3429 in _start (/usr/lib/frr/zebra+0x8f429)
0x608000011d28 is located 8 bytes inside of 88-byte region [0x608000011d20,0x608000011d78)
freed by thread T0 here:
    #0 0x7ffff768bb6f in __interceptor_free (/lib/x86_64-linux-gnu/libasan.so.6+0xa9b6f)
    #1 0x7ffff739ccad in qfree lib/memory.c:129
    #2 0x555555709ee4 in rib_gc_dest zebra/zebra_rib.c:746
    #3 0x55555570ca76 in rib_process zebra/zebra_rib.c:1240
    #4 0x555555711a05 in process_subq_route zebra/zebra_rib.c:2245
    #5 0x555555711d2e in process_subq zebra/zebra_rib.c:2286
    #6 0x555555711ec7 in meta_queue_process zebra/zebra_rib.c:2320
    #7 0x7ffff74701f7 in work_queue_run lib/workqueue.c:291
    #8 0x7ffff7450e9c in thread_call lib/thread.c:1581
    #9 0x7ffff738eaf7 in frr_run lib/libfrr.c:1099
    #10 0x55555561a578 in main zebra/main.c:455
    #11 0x7ffff7079cc9 in __libc_start_main ../csu/libc-start.c:308
previously allocated by thread T0 here:
    #0 0x7ffff768c037 in calloc (/lib/x86_64-linux-gnu/libasan.so.6+0xaa037)
    #1 0x7ffff739cb98 in qcalloc lib/memory.c:110
    #2 0x555555712ace in zebra_rib_create_dest zebra/zebra_rib.c:2515
    #3 0x555555712c6c in rib_link zebra/zebra_rib.c:2576
    #4 0x555555712faa in rib_addnode zebra/zebra_rib.c:2607
    #5 0x555555715bf0 in rib_add_multipath_nhe zebra/zebra_rib.c:3012
    #6 0x555555715f56 in rib_add_multipath zebra/zebra_rib.c:3049
    #7 0x55555571788b in rib_add zebra/zebra_rib.c:3327
    #8 0x5555555e584a in connected_up zebra/connected.c:254
    #9 0x5555555e42ff in connected_announce zebra/connected.c:94
    #10 0x5555555e4fd3 in connected_update zebra/connected.c:195
    #11 0x5555555e61ad in connected_add_ipv4 zebra/connected.c:340
    #12 0x5555555f26f5 in netlink_interface_addr zebra/if_netlink.c:1213
    #13 0x55555560f756 in netlink_information_fetch zebra/kernel_netlink.c:350
    #14 0x555555612e49 in netlink_parse_info zebra/kernel_netlink.c:941
    #15 0x55555560f9f1 in kernel_read zebra/kernel_netlink.c:402
    #16 0x7ffff7450e9c in thread_call lib/thread.c:1581
    #17 0x7ffff738eaf7 in frr_run lib/libfrr.c:1099
    #18 0x55555561a578 in main zebra/main.c:455
    #19 0x7ffff7079cc9 in __libc_start_main ../csu/libc-start.c:308
SUMMARY: AddressSanitizer: heap-use-after-free zebra/rib.h:222 in re_list_const_first

This is happening because we are using the dest pointer after a call into
rib_gc_dest.  In process_subq_route, we call rib_process() and if the
dest is deleted dest pointer is now garbage.  We must reload the
dest pointer in this case.

Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp pushed a commit that referenced this pull request Oct 16, 2020
This problem was reported by the sanitizer -
=================================================================
==24764==ERROR: AddressSanitizer: heap-use-after-free on address 0x60d0000115c8 at pc 0x55cb9cfad312 bp 0x7fffa0552140 sp 0x7fffa0552138
READ of size 8 at 0x60d0000115c8 thread T0
    #0 0x55cb9cfad311 in zebra_evpn_remote_es_flush zebra/zebra_evpn_mh.c:2041
    #1 0x55cb9cfad311 in zebra_evpn_es_cleanup zebra/zebra_evpn_mh.c:2234
    #2 0x55cb9cf6ae78 in zebra_vrf_disable zebra/zebra_vrf.c:205
    #3 0x7fc8d478f114 in vrf_delete lib/vrf.c:229
    #4 0x7fc8d478f99a in vrf_terminate lib/vrf.c:541
    #5 0x55cb9ceba0af in sigint zebra/main.c:176
    #6 0x55cb9ceba0af in sigint zebra/main.c:130
    #7 0x7fc8d4765d20 in quagga_sigevent_process lib/sigevent.c:103
    #8 0x7fc8d4787e8c in thread_fetch lib/thread.c:1396
    #9 0x7fc8d4708782 in frr_run lib/libfrr.c:1092
    #10 0x55cb9ce931d8 in main zebra/main.c:488
    #11 0x7fc8d43ee09a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409a)
    #12 0x55cb9ce94c09 in _start (/usr/lib/frr/zebra+0x8ac09)
=================================================================

Signed-off-by: Anuradha Karuppiah <anuradhak@cumulusnetworks.com>
donaldsharp added a commit that referenced this pull request Oct 16, 2020
When zebra is running with debugs turned on there
is a use after free reported by the address sanitizer:

2020/10/16 12:58:02 ZEBRA: rib_delnode: (0:254):4.5.6.16/32: rn 0x60b000026f20, re 0x6080000131a0, removing
2020/10/16 12:58:02 ZEBRA: rib_meta_queue_add: (0:254):4.5.6.16/32: queued rn 0x60b000026f20 into sub-queue 3
=================================================================
==3101430==ERROR: AddressSanitizer: heap-use-after-free on address 0x608000011d28 at pc 0x555555705ab6 bp 0x7fffffffdab0 sp 0x7fffffffdaa8
READ of size 8 at 0x608000011d28 thread T0
    #0 0x555555705ab5 in re_list_const_first zebra/rib.h:222
    #1 0x555555705b54 in re_list_first zebra/rib.h:222
    #2 0x555555711a4f in process_subq_route zebra/zebra_rib.c:2248
    #3 0x555555711d2e in process_subq zebra/zebra_rib.c:2286
    #4 0x555555711ec7 in meta_queue_process zebra/zebra_rib.c:2320
    #5 0x7ffff74701f7 in work_queue_run lib/workqueue.c:291
    #6 0x7ffff7450e9c in thread_call lib/thread.c:1581
    #7 0x7ffff738eaf7 in frr_run lib/libfrr.c:1099
    #8 0x55555561a578 in main zebra/main.c:455
    #9 0x7ffff7079cc9 in __libc_start_main ../csu/libc-start.c:308
    #10 0x5555555e3429 in _start (/usr/lib/frr/zebra+0x8f429)
0x608000011d28 is located 8 bytes inside of 88-byte region [0x608000011d20,0x608000011d78)
freed by thread T0 here:
    #0 0x7ffff768bb6f in __interceptor_free (/lib/x86_64-linux-gnu/libasan.so.6+0xa9b6f)
    #1 0x7ffff739ccad in qfree lib/memory.c:129
    #2 0x555555709ee4 in rib_gc_dest zebra/zebra_rib.c:746
    #3 0x55555570ca76 in rib_process zebra/zebra_rib.c:1240
    #4 0x555555711a05 in process_subq_route zebra/zebra_rib.c:2245
    #5 0x555555711d2e in process_subq zebra/zebra_rib.c:2286
    #6 0x555555711ec7 in meta_queue_process zebra/zebra_rib.c:2320
    #7 0x7ffff74701f7 in work_queue_run lib/workqueue.c:291
    #8 0x7ffff7450e9c in thread_call lib/thread.c:1581
    #9 0x7ffff738eaf7 in frr_run lib/libfrr.c:1099
    #10 0x55555561a578 in main zebra/main.c:455
    #11 0x7ffff7079cc9 in __libc_start_main ../csu/libc-start.c:308
previously allocated by thread T0 here:
    #0 0x7ffff768c037 in calloc (/lib/x86_64-linux-gnu/libasan.so.6+0xaa037)
    #1 0x7ffff739cb98 in qcalloc lib/memory.c:110
    #2 0x555555712ace in zebra_rib_create_dest zebra/zebra_rib.c:2515
    #3 0x555555712c6c in rib_link zebra/zebra_rib.c:2576
    #4 0x555555712faa in rib_addnode zebra/zebra_rib.c:2607
    #5 0x555555715bf0 in rib_add_multipath_nhe zebra/zebra_rib.c:3012
    #6 0x555555715f56 in rib_add_multipath zebra/zebra_rib.c:3049
    #7 0x55555571788b in rib_add zebra/zebra_rib.c:3327
    #8 0x5555555e584a in connected_up zebra/connected.c:254
    #9 0x5555555e42ff in connected_announce zebra/connected.c:94
    #10 0x5555555e4fd3 in connected_update zebra/connected.c:195
    #11 0x5555555e61ad in connected_add_ipv4 zebra/connected.c:340
    #12 0x5555555f26f5 in netlink_interface_addr zebra/if_netlink.c:1213
    #13 0x55555560f756 in netlink_information_fetch zebra/kernel_netlink.c:350
    #14 0x555555612e49 in netlink_parse_info zebra/kernel_netlink.c:941
    #15 0x55555560f9f1 in kernel_read zebra/kernel_netlink.c:402
    #16 0x7ffff7450e9c in thread_call lib/thread.c:1581
    #17 0x7ffff738eaf7 in frr_run lib/libfrr.c:1099
    #18 0x55555561a578 in main zebra/main.c:455
    #19 0x7ffff7079cc9 in __libc_start_main ../csu/libc-start.c:308
SUMMARY: AddressSanitizer: heap-use-after-free zebra/rib.h:222 in re_list_const_first

This is happening because we are using the dest pointer after a call into
rib_gc_dest.  In process_subq_route, we call rib_process() and if the
dest is deleted dest pointer is now garbage.  We must reload the
dest pointer in this case.

Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp added a commit that referenced this pull request Oct 17, 2020
When zebra is running with debugs turned on there
is a use after free reported by the address sanitizer:

2020/10/16 12:58:02 ZEBRA: rib_delnode: (0:254):4.5.6.16/32: rn 0x60b000026f20, re 0x6080000131a0, removing
2020/10/16 12:58:02 ZEBRA: rib_meta_queue_add: (0:254):4.5.6.16/32: queued rn 0x60b000026f20 into sub-queue 3
=================================================================
==3101430==ERROR: AddressSanitizer: heap-use-after-free on address 0x608000011d28 at pc 0x555555705ab6 bp 0x7fffffffdab0 sp 0x7fffffffdaa8
READ of size 8 at 0x608000011d28 thread T0
    #0 0x555555705ab5 in re_list_const_first zebra/rib.h:222
    #1 0x555555705b54 in re_list_first zebra/rib.h:222
    #2 0x555555711a4f in process_subq_route zebra/zebra_rib.c:2248
    #3 0x555555711d2e in process_subq zebra/zebra_rib.c:2286
    #4 0x555555711ec7 in meta_queue_process zebra/zebra_rib.c:2320
    #5 0x7ffff74701f7 in work_queue_run lib/workqueue.c:291
    #6 0x7ffff7450e9c in thread_call lib/thread.c:1581
    #7 0x7ffff738eaf7 in frr_run lib/libfrr.c:1099
    #8 0x55555561a578 in main zebra/main.c:455
    #9 0x7ffff7079cc9 in __libc_start_main ../csu/libc-start.c:308
    #10 0x5555555e3429 in _start (/usr/lib/frr/zebra+0x8f429)
0x608000011d28 is located 8 bytes inside of 88-byte region [0x608000011d20,0x608000011d78)
freed by thread T0 here:
    #0 0x7ffff768bb6f in __interceptor_free (/lib/x86_64-linux-gnu/libasan.so.6+0xa9b6f)
    #1 0x7ffff739ccad in qfree lib/memory.c:129
    #2 0x555555709ee4 in rib_gc_dest zebra/zebra_rib.c:746
    #3 0x55555570ca76 in rib_process zebra/zebra_rib.c:1240
    #4 0x555555711a05 in process_subq_route zebra/zebra_rib.c:2245
    #5 0x555555711d2e in process_subq zebra/zebra_rib.c:2286
    #6 0x555555711ec7 in meta_queue_process zebra/zebra_rib.c:2320
    #7 0x7ffff74701f7 in work_queue_run lib/workqueue.c:291
    #8 0x7ffff7450e9c in thread_call lib/thread.c:1581
    #9 0x7ffff738eaf7 in frr_run lib/libfrr.c:1099
    #10 0x55555561a578 in main zebra/main.c:455
    #11 0x7ffff7079cc9 in __libc_start_main ../csu/libc-start.c:308
previously allocated by thread T0 here:
    #0 0x7ffff768c037 in calloc (/lib/x86_64-linux-gnu/libasan.so.6+0xaa037)
    #1 0x7ffff739cb98 in qcalloc lib/memory.c:110
    #2 0x555555712ace in zebra_rib_create_dest zebra/zebra_rib.c:2515
    #3 0x555555712c6c in rib_link zebra/zebra_rib.c:2576
    #4 0x555555712faa in rib_addnode zebra/zebra_rib.c:2607
    #5 0x555555715bf0 in rib_add_multipath_nhe zebra/zebra_rib.c:3012
    #6 0x555555715f56 in rib_add_multipath zebra/zebra_rib.c:3049
    #7 0x55555571788b in rib_add zebra/zebra_rib.c:3327
    #8 0x5555555e584a in connected_up zebra/connected.c:254
    #9 0x5555555e42ff in connected_announce zebra/connected.c:94
    #10 0x5555555e4fd3 in connected_update zebra/connected.c:195
    #11 0x5555555e61ad in connected_add_ipv4 zebra/connected.c:340
    #12 0x5555555f26f5 in netlink_interface_addr zebra/if_netlink.c:1213
    #13 0x55555560f756 in netlink_information_fetch zebra/kernel_netlink.c:350
    #14 0x555555612e49 in netlink_parse_info zebra/kernel_netlink.c:941
    #15 0x55555560f9f1 in kernel_read zebra/kernel_netlink.c:402
    #16 0x7ffff7450e9c in thread_call lib/thread.c:1581
    #17 0x7ffff738eaf7 in frr_run lib/libfrr.c:1099
    #18 0x55555561a578 in main zebra/main.c:455
    #19 0x7ffff7079cc9 in __libc_start_main ../csu/libc-start.c:308
SUMMARY: AddressSanitizer: heap-use-after-free zebra/rib.h:222 in re_list_const_first

This is happening because we are using the dest pointer after a call into
rib_gc_dest.  In process_subq_route, we call rib_process() and if the
dest is deleted dest pointer is now garbage.  We must reload the
dest pointer in this case.

Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp pushed a commit that referenced this pull request Oct 27, 2020
Sample Configuration with prefix-list and community match rules
---------------------------------------------------------------

R1 ------- R2(DUT) ------- R3

Router2# show running-config
Building configuration...

Current configuration:
!
frr version 7.6-dev-MyOwnFRRVersion
frr defaults traditional
hostname router
log file /var/log/frr/bgpd.log
log syslog informational
hostname Router2
service integrated-vtysh-config
!
debug bgp updates in
debug bgp updates out
!
debug route-map
!
ip route 20.20.0.0/16 blackhole
ipv6 route 2001:db8::200/128 blackhole
!
interface enp0s9
 ip address 10.10.10.2/24
!
interface enp0s10
 ip address 10.10.20.2/24
!
interface lo
 ip address 2.2.2.2/32
!
router bgp 2
 bgp log-neighbor-changes
 no bgp ebgp-requires-policy
 neighbor 10.10.10.1 remote-as 1
 neighbor 10.10.20.3 remote-as 3
 !
 address-family ipv4 unicast
  neighbor 10.10.10.1 soft-reconfiguration inbound
  neighbor 10.10.20.3 soft-reconfiguration inbound
  neighbor 10.10.20.3 advertise-map ADV-MAP non-exist-map EXIST-MAP
 exit-address-family
!
ip prefix-list DEFAULT seq 5 permit 1.1.1.5/32
ip prefix-list DEFAULT seq 10 permit 1.1.1.1/32
ip prefix-list EXIST seq 5 permit 10.10.10.10/32
ip prefix-list DEFAULT-ROUTE seq 5 permit 0.0.0.0/0
ip prefix-list IP1 seq 5 permit 10.139.224.0/20
ip prefix-list T2 seq 5 permit 1.1.1.5/32
!
bgp community-list standard DC-ROUTES seq 5 permit 64952:3008
bgp community-list standard DC-ROUTES seq 10 permit 64671:501
bgp community-list standard DC-ROUTES seq 15 permit 64950:3009
bgp community-list standard DEFAULT-ROUTE seq 5 permit 65013:200
!
route-map ADV-MAP permit 10
 match ip address prefix-list IP1
!
route-map ADV-MAP permit 20
 match community DC-ROUTES
!
route-map EXIST-MAP permit 10
 match community DEFAULT-ROUTE
 match ip address prefix-list DEFAULT-ROUTE
!
line vty
!
end
Router2#

Router2# show ip bgp 0.0.0.0
BGP routing table entry for 0.0.0.0/0
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  10.10.10.1 10.10.20.3
  1
    10.10.10.1 from 10.10.10.1 (10.139.224.1)
      Origin IGP, metric 0, valid, external, best (First path received)
      Community: 64848:3011 65011:200 65013:200
      Last update: Tue Oct  6 02:39:42 2020
Router2#

Sample output with non-exist-map when default route present in table
--------------------------------------------------------------------
Router2# show ip bgp
BGP table version is 4, local router ID is 2.2.2.2, vrf id 0
Default local pref 100, local AS 2
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 0.0.0.0/0        10.10.10.1               0             0 1 i
*> 1.1.1.1/32       10.10.10.1               0             0 1 i
*> 1.1.1.5/32       10.10.10.1               0             0 1 i
*> 10.139.224.0/20  10.10.10.1               0             0 1 ?

Displayed  4 routes and 4 total paths

Router2# show ip bgp neighbors 10.10.20.3 advertised-routes
BGP table version is 4, local router ID is 2.2.2.2, vrf id 0
Default local pref 100, local AS 2
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 0.0.0.0/0        0.0.0.0                                0 1 i
*> 1.1.1.5/32       0.0.0.0                                0 1 i   		<<<<<<<<<  non-exist-map : 0.0.0.0/0 is present so, 10.139.224.0/20 not advertised

Total number of prefixes 2

Sample output with non-exist-map when default route not present in table
------------------------------------------------------------------------
Router2# show ip bgp
BGP table version is 5, local router ID is 2.2.2.2, vrf id 0
Default local pref 100, local AS 2
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 1.1.1.1/32       10.10.10.1               0             0 1 i
*> 1.1.1.5/32       10.10.10.1               0             0 1 i
*> 10.139.224.0/20  10.10.10.1               0             0 1 ?

Displayed  3 routes and 3 total paths
Router2#
Router2#
Router2# show ip bgp neighbors 10.10.20.3 advertised-routes
BGP table version is 5, local router ID is 2.2.2.2, vrf id 0
Default local pref 100, local AS 2
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 1.1.1.1/32       0.0.0.0                                0 1 i
*> 1.1.1.5/32       0.0.0.0                                0 1 i
*> 10.139.224.0/20  0.0.0.0                                0 1 ?                <<<<<<<<<  non-exist-map : 0.0.0.0/0 is not present so, 10.139.224.0/20 advertised

Total number of prefixes 3
Router2#

Sample output with exist-map when default route present in table
--------------------------------------------------------------------
Router2# show ip bgp
BGP table version is 8, local router ID is 2.2.2.2, vrf id 0
Default local pref 100, local AS 2
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 0.0.0.0/0        10.10.10.1               0             0 1 i
*> 1.1.1.1/32       10.10.10.1               0             0 1 i
*> 1.1.1.5/32       10.10.10.1               0             0 1 i
*> 10.139.224.0/20  10.10.10.1               0             0 1 ?

Displayed  4 routes and 4 total paths
Router2#
Router2#
Router2#
Router2#
Router2# show ip bgp neighbors 10.10.20.3 advertised-routes
BGP table version is 8, local router ID is 2.2.2.2, vrf id 0
Default local pref 100, local AS 2
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 0.0.0.0/0        0.0.0.0                                0 1 i
*> 1.1.1.1/32       0.0.0.0                                0 1 i
*> 1.1.1.5/32       0.0.0.0                                0 1 i
*> 10.139.224.0/20  0.0.0.0                                0 1 ?		<<<<<<<<<  exist-map : 0.0.0.0/0 is present so, 10.139.224.0/20 advertised

Total number of prefixes 4
Router2#

Sample output with exist-map when default route not present in table
--------------------------------------------------------------------
Router2# show ip bgp
BGP table version is 9, local router ID is 2.2.2.2, vrf id 0
Default local pref 100, local AS 2
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 1.1.1.1/32       10.10.10.1               0             0 1 i
*> 1.1.1.5/32       10.10.10.1               0             0 1 i
*> 10.139.224.0/20  10.10.10.1               0             0 1 ?

Displayed  3 routes and 3 total paths
Router2#
Router2#
Router2#
Router2# show ip bgp neighbors 10.10.20.3 advertised-routes
BGP table version is 9, local router ID is 2.2.2.2, vrf id 0
Default local pref 100, local AS 2
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 1.1.1.5/32       0.0.0.0                                0 1 i		<<<<<<<<<  exist-map : 0.0.0.0/0 is not present so, 10.139.224.0/20 not advertised

Total number of prefixes 1
Router2#

Signed-off-by: Madhuri Kuruganti <k.madhuri@samsung.com>
donaldsharp pushed a commit that referenced this pull request Oct 29, 2020
This problem was reported by the sanitizer -
=================================================================
==24764==ERROR: AddressSanitizer: heap-use-after-free on address 0x60d0000115c8 at pc 0x55cb9cfad312 bp 0x7fffa0552140 sp 0x7fffa0552138
READ of size 8 at 0x60d0000115c8 thread T0
    #0 0x55cb9cfad311 in zebra_evpn_remote_es_flush zebra/zebra_evpn_mh.c:2041
    #1 0x55cb9cfad311 in zebra_evpn_es_cleanup zebra/zebra_evpn_mh.c:2234
    #2 0x55cb9cf6ae78 in zebra_vrf_disable zebra/zebra_vrf.c:205
    #3 0x7fc8d478f114 in vrf_delete lib/vrf.c:229
    #4 0x7fc8d478f99a in vrf_terminate lib/vrf.c:541
    #5 0x55cb9ceba0af in sigint zebra/main.c:176
    #6 0x55cb9ceba0af in sigint zebra/main.c:130
    #7 0x7fc8d4765d20 in quagga_sigevent_process lib/sigevent.c:103
    #8 0x7fc8d4787e8c in thread_fetch lib/thread.c:1396
    #9 0x7fc8d4708782 in frr_run lib/libfrr.c:1092
    #10 0x55cb9ce931d8 in main zebra/main.c:488
    #11 0x7fc8d43ee09a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409a)
    #12 0x55cb9ce94c09 in _start (/usr/lib/frr/zebra+0x8ac09)
=================================================================

Signed-off-by: Anuradha Karuppiah <anuradhak@cumulusnetworks.com>
donaldsharp added a commit that referenced this pull request Oct 29, 2020
When zebra is running with debugs turned on there
is a use after free reported by the address sanitizer:

2020/10/16 12:58:02 ZEBRA: rib_delnode: (0:254):4.5.6.16/32: rn 0x60b000026f20, re 0x6080000131a0, removing
2020/10/16 12:58:02 ZEBRA: rib_meta_queue_add: (0:254):4.5.6.16/32: queued rn 0x60b000026f20 into sub-queue 3
=================================================================
==3101430==ERROR: AddressSanitizer: heap-use-after-free on address 0x608000011d28 at pc 0x555555705ab6 bp 0x7fffffffdab0 sp 0x7fffffffdaa8
READ of size 8 at 0x608000011d28 thread T0
    #0 0x555555705ab5 in re_list_const_first zebra/rib.h:222
    #1 0x555555705b54 in re_list_first zebra/rib.h:222
    #2 0x555555711a4f in process_subq_route zebra/zebra_rib.c:2248
    #3 0x555555711d2e in process_subq zebra/zebra_rib.c:2286
    #4 0x555555711ec7 in meta_queue_process zebra/zebra_rib.c:2320
    #5 0x7ffff74701f7 in work_queue_run lib/workqueue.c:291
    #6 0x7ffff7450e9c in thread_call lib/thread.c:1581
    #7 0x7ffff738eaf7 in frr_run lib/libfrr.c:1099
    #8 0x55555561a578 in main zebra/main.c:455
    #9 0x7ffff7079cc9 in __libc_start_main ../csu/libc-start.c:308
    #10 0x5555555e3429 in _start (/usr/lib/frr/zebra+0x8f429)
0x608000011d28 is located 8 bytes inside of 88-byte region [0x608000011d20,0x608000011d78)
freed by thread T0 here:
    #0 0x7ffff768bb6f in __interceptor_free (/lib/x86_64-linux-gnu/libasan.so.6+0xa9b6f)
    #1 0x7ffff739ccad in qfree lib/memory.c:129
    #2 0x555555709ee4 in rib_gc_dest zebra/zebra_rib.c:746
    #3 0x55555570ca76 in rib_process zebra/zebra_rib.c:1240
    #4 0x555555711a05 in process_subq_route zebra/zebra_rib.c:2245
    #5 0x555555711d2e in process_subq zebra/zebra_rib.c:2286
    #6 0x555555711ec7 in meta_queue_process zebra/zebra_rib.c:2320
    #7 0x7ffff74701f7 in work_queue_run lib/workqueue.c:291
    #8 0x7ffff7450e9c in thread_call lib/thread.c:1581
    #9 0x7ffff738eaf7 in frr_run lib/libfrr.c:1099
    #10 0x55555561a578 in main zebra/main.c:455
    #11 0x7ffff7079cc9 in __libc_start_main ../csu/libc-start.c:308
previously allocated by thread T0 here:
    #0 0x7ffff768c037 in calloc (/lib/x86_64-linux-gnu/libasan.so.6+0xaa037)
    #1 0x7ffff739cb98 in qcalloc lib/memory.c:110
    #2 0x555555712ace in zebra_rib_create_dest zebra/zebra_rib.c:2515
    #3 0x555555712c6c in rib_link zebra/zebra_rib.c:2576
    #4 0x555555712faa in rib_addnode zebra/zebra_rib.c:2607
    #5 0x555555715bf0 in rib_add_multipath_nhe zebra/zebra_rib.c:3012
    #6 0x555555715f56 in rib_add_multipath zebra/zebra_rib.c:3049
    #7 0x55555571788b in rib_add zebra/zebra_rib.c:3327
    #8 0x5555555e584a in connected_up zebra/connected.c:254
    #9 0x5555555e42ff in connected_announce zebra/connected.c:94
    #10 0x5555555e4fd3 in connected_update zebra/connected.c:195
    #11 0x5555555e61ad in connected_add_ipv4 zebra/connected.c:340
    #12 0x5555555f26f5 in netlink_interface_addr zebra/if_netlink.c:1213
    #13 0x55555560f756 in netlink_information_fetch zebra/kernel_netlink.c:350
    #14 0x555555612e49 in netlink_parse_info zebra/kernel_netlink.c:941
    #15 0x55555560f9f1 in kernel_read zebra/kernel_netlink.c:402
    #16 0x7ffff7450e9c in thread_call lib/thread.c:1581
    #17 0x7ffff738eaf7 in frr_run lib/libfrr.c:1099
    #18 0x55555561a578 in main zebra/main.c:455
    #19 0x7ffff7079cc9 in __libc_start_main ../csu/libc-start.c:308
SUMMARY: AddressSanitizer: heap-use-after-free zebra/rib.h:222 in re_list_const_first

This is happening because we are using the dest pointer after a call into
rib_gc_dest.  In process_subq_route, we call rib_process() and if the
dest is deleted dest pointer is now garbage.  We must reload the
dest pointer in this case.

Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp pushed a commit that referenced this pull request Oct 30, 2020
The fields in the broadcast/p2p union struct in an isis circuit are
initialized when the circuit goes up, but currently this step is
skipped if the interface is passive. This can create problems if the
circuit type (referred to as network type in the config) changes from
broadcast to point-to-point. We can end up with the p2p neighbor
pointer pointing at some garbage left by the broadcast struct in the
union, which would then cause a segfault the first time we would
dereference it - for example when building the lsp, or computing the
SPF tree.

compressed backtrace of a possible crash:
 #0  0x0000555555579a9c in lsp_build at frr/isisd/isis_lsp.c:1114
 #1  0x000055555557a516 in lsp_regenerate at frr/isisd/isis_lsp.c:1301
 #2  0x000055555557aa25 in lsp_refresh at frr/isisd/isis_lsp.c:1381
 #3  0x00007ffff7b2622c in thread_call at frr/lib/thread.c:1549
 #4  0x00007ffff7ad6df4 in frr_run at frr/lib/libfrr.c:1098
 #5  0x000055555556b67f in main at frr/isisd/isis_main.c:272

isis_lsp.c:
1112	case CIRCUIT_T_P2P: {
1113		struct isis_adjacency *nei = circuit->u.p2p.neighbor;
1114		if (nei && nei->adj_state == ISIS_ADJ_UP

Signed-off-by: Emanuele Di Pascale <emanuele@voltanet.io>
donaldsharp added a commit that referenced this pull request Nov 25, 2020
We are using data after it has been freed and handed back to the
OS.
Address Sanitizer output:

error	23-Nov-2020 18:53:57	ERROR: AddressSanitizer: heap-use-after-free on address 0x631000024838 at pc 0x55f825998f58 bp 0x7fffa5b0f5b0 sp 0x7fffa5b0f5a0
error	23-Nov-2020 18:53:57	READ of size 4 at 0x631000024838 thread T0
error	23-Nov-2020 18:53:57	    #0 0x55f825998f57 in lde_imsg_compose_parent_sync ldpd/lde.c:226
error	23-Nov-2020 18:53:57	    #1 0x55f8259ca9ed in vlog ldpd/log.c:48
error	23-Nov-2020 18:53:57	    #2 0x55f8259cb1c8 in log_info ldpd/log.c:102
error	23-Nov-2020 18:53:57	    #3 0x55f82599e841 in lde_shutdown ldpd/lde.c:208
error	23-Nov-2020 18:53:57	    #4 0x55f8259a2703 in lde_dispatch_parent ldpd/lde.c:666
error	23-Nov-2020 18:53:57	    #5 0x55f825ac3815 in thread_call lib/thread.c:1681
error	23-Nov-2020 18:53:57	    #6 0x55f825998d5e in lde ldpd/lde.c:160
error	23-Nov-2020 18:53:57	    #7 0x55f82598a289 in main ldpd/ldpd.c:320
error	23-Nov-2020 18:53:57	    #8 0x7fe3f749db96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
error	23-Nov-2020 18:53:57	    #9 0x55f825982579 in _start (/usr/lib/frr/ldpd+0xb3579)
error	23-Nov-2020 18:53:57
error	23-Nov-2020 18:53:57	0x631000024838 is located 65592 bytes inside of 65632-byte region [0x631000014800,0x631000024860)
error	23-Nov-2020 18:53:57	freed by thread T0 here:
error	23-Nov-2020 18:53:57	    #0 0x7fe3f8a4d7a8 in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xde7a8)
error	23-Nov-2020 18:53:57	    #1 0x55f82599e830 in lde_shutdown ldpd/lde.c:206
error	23-Nov-2020 18:53:57	    #2 0x55f8259a2703 in lde_dispatch_parent ldpd/lde.c:666
error	23-Nov-2020 18:53:57	    #3 0x55f825ac3815 in thread_call lib/thread.c:1681
error	23-Nov-2020 18:53:57	    #4 0x55f825998d5e in lde ldpd/lde.c:160
error	23-Nov-2020 18:53:57	    #5 0x55f82598a289 in main ldpd/ldpd.c:320
error	23-Nov-2020 18:53:57	    #6 0x7fe3f749db96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
error	23-Nov-2020 18:53:57
error	23-Nov-2020 18:53:57	previously allocated by thread T0 here:
error	23-Nov-2020 18:53:57	    #0 0x7fe3f8a4dd28 in __interceptor_calloc (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xded28)
error	23-Nov-2020 18:53:57	    #1 0x55f825998cb7 in lde ldpd/lde.c:151
error	23-Nov-2020 18:53:57	    #2 0x55f82598a289 in main ldpd/ldpd.c:320
error	23-Nov-2020 18:53:57	    #3 0x7fe3f749db96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
error	23-Nov-2020 18:53:57

The fix is to put this in global space.

Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp pushed a commit that referenced this pull request Nov 25, 2020
The fields in the broadcast/p2p union struct in an isis circuit are
initialized when the circuit goes up, but currently this step is
skipped if the interface is passive. This can create problems if the
circuit type (referred to as network type in the config) changes from
broadcast to point-to-point. We can end up with the p2p neighbor
pointer pointing at some garbage left by the broadcast struct in the
union, which would then cause a segfault the first time we would
dereference it - for example when building the lsp, or computing the
SPF tree.

compressed backtrace of a possible crash:
 #0  0x0000555555579a9c in lsp_build at frr/isisd/isis_lsp.c:1114
 #1  0x000055555557a516 in lsp_regenerate at frr/isisd/isis_lsp.c:1301
 #2  0x000055555557aa25 in lsp_refresh at frr/isisd/isis_lsp.c:1381
 #3  0x00007ffff7b2622c in thread_call at frr/lib/thread.c:1549
 #4  0x00007ffff7ad6df4 in frr_run at frr/lib/libfrr.c:1098
 #5  0x000055555556b67f in main at frr/isisd/isis_main.c:272

isis_lsp.c:
1112	case CIRCUIT_T_P2P: {
1113		struct isis_adjacency *nei = circuit->u.p2p.neighbor;
1114		if (nei && nei->adj_state == ISIS_ADJ_UP

Signed-off-by: Emanuele Di Pascale <emanuele@voltanet.io>
donaldsharp added a commit that referenced this pull request Nov 26, 2020
error	26-Nov-2020 14:35:02	ERROR: AddressSanitizer: heap-use-after-free on address 0x631000024838 at pc 0x55cefae977e9 bp 0x7ffdd3546860 sp 0x7ffdd3546850
error	26-Nov-2020 14:35:02	READ of size 4 at 0x631000024838 thread T0
error	26-Nov-2020 14:35:02	    #0 0x55cefae977e8 in ldpe_imsg_compose_parent_sync ldpd/ldpe.c:256
error	26-Nov-2020 14:35:02	    #1 0x55cefae9ab13 in vlog ldpd/log.c:53
error	26-Nov-2020 14:35:02	    #2 0x55cefae9b21f in log_info ldpd/log.c:102
error	26-Nov-2020 14:35:02	    #3 0x55cefae96eae in ldpe_shutdown ldpd/ldpe.c:237
error	26-Nov-2020 14:35:02	    #4 0x55cefae99254 in ldpe_dispatch_main ldpd/ldpe.c:585
error	26-Nov-2020 14:35:02	    #5 0x55cefaf93875 in thread_call lib/thread.c:1681
error	26-Nov-2020 14:35:02	    #6 0x55cefae97304 in ldpe ldpd/ldpe.c:136
error	26-Nov-2020 14:35:02	    #7 0x55cefae5a2e2 in main ldpd/ldpd.c:322
error	26-Nov-2020 14:35:02	    #8 0x7f4ef0c33b96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
error	26-Nov-2020 14:35:02	    #9 0x55cefae525e9 in _start (/usr/lib/frr/ldpd+0xb35e9)
error	26-Nov-2020 14:35:02
error	26-Nov-2020 14:35:02	0x631000024838 is located 65592 bytes inside of 65632-byte region [0x631000014800,0x631000024860)
error	26-Nov-2020 14:35:02	freed by thread T0 here:
error	26-Nov-2020 14:35:02	    #0 0x7f4ef21e37a8 in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xde7a8)
error	26-Nov-2020 14:35:02	    #1 0x55cefae96e91 in ldpe_shutdown ldpd/ldpe.c:234
error	26-Nov-2020 14:35:02	    #2 0x55cefae99254 in ldpe_dispatch_main ldpd/ldpe.c:585
error	26-Nov-2020 14:35:02	    #3 0x55cefaf93875 in thread_call lib/thread.c:1681
error	26-Nov-2020 14:35:02	    #4 0x55cefae97304 in ldpe ldpd/ldpe.c:136
error	26-Nov-2020 14:35:02	    #5 0x55cefae5a2e2 in main ldpd/ldpd.c:322
error	26-Nov-2020 14:35:02	    #6 0x7f4ef0c33b96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
error	26-Nov-2020 14:35:02
error	26-Nov-2020 14:35:02	previously allocated by thread T0 here:
error	26-Nov-2020 14:35:02	    #0 0x7f4ef21e3d28 in __interceptor_calloc (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xded28)
error	26-Nov-2020 14:35:02	    #1 0x55cefae9725d in ldpe ldpd/ldpe.c:127
error	26-Nov-2020 14:35:02	    #2 0x55cefae5a2e2 in main ldpd/ldpd.c:322
error	26-Nov-2020 14:35:02	    #3 0x7f4ef0c33b96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)

Clean this problem up in the same way as the previous commit

Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp added a commit that referenced this pull request Dec 19, 2020
We are using data after it has been freed and handed back to the
OS.
Address Sanitizer output:

error	23-Nov-2020 18:53:57	ERROR: AddressSanitizer: heap-use-after-free on address 0x631000024838 at pc 0x55f825998f58 bp 0x7fffa5b0f5b0 sp 0x7fffa5b0f5a0
error	23-Nov-2020 18:53:57	READ of size 4 at 0x631000024838 thread T0
error	23-Nov-2020 18:53:57	    #0 0x55f825998f57 in lde_imsg_compose_parent_sync ldpd/lde.c:226
error	23-Nov-2020 18:53:57	    #1 0x55f8259ca9ed in vlog ldpd/log.c:48
error	23-Nov-2020 18:53:57	    #2 0x55f8259cb1c8 in log_info ldpd/log.c:102
error	23-Nov-2020 18:53:57	    #3 0x55f82599e841 in lde_shutdown ldpd/lde.c:208
error	23-Nov-2020 18:53:57	    #4 0x55f8259a2703 in lde_dispatch_parent ldpd/lde.c:666
error	23-Nov-2020 18:53:57	    #5 0x55f825ac3815 in thread_call lib/thread.c:1681
error	23-Nov-2020 18:53:57	    #6 0x55f825998d5e in lde ldpd/lde.c:160
error	23-Nov-2020 18:53:57	    #7 0x55f82598a289 in main ldpd/ldpd.c:320
error	23-Nov-2020 18:53:57	    #8 0x7fe3f749db96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
error	23-Nov-2020 18:53:57	    #9 0x55f825982579 in _start (/usr/lib/frr/ldpd+0xb3579)
error	23-Nov-2020 18:53:57
error	23-Nov-2020 18:53:57	0x631000024838 is located 65592 bytes inside of 65632-byte region [0x631000014800,0x631000024860)
error	23-Nov-2020 18:53:57	freed by thread T0 here:
error	23-Nov-2020 18:53:57	    #0 0x7fe3f8a4d7a8 in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xde7a8)
error	23-Nov-2020 18:53:57	    #1 0x55f82599e830 in lde_shutdown ldpd/lde.c:206
error	23-Nov-2020 18:53:57	    #2 0x55f8259a2703 in lde_dispatch_parent ldpd/lde.c:666
error	23-Nov-2020 18:53:57	    #3 0x55f825ac3815 in thread_call lib/thread.c:1681
error	23-Nov-2020 18:53:57	    #4 0x55f825998d5e in lde ldpd/lde.c:160
error	23-Nov-2020 18:53:57	    #5 0x55f82598a289 in main ldpd/ldpd.c:320
error	23-Nov-2020 18:53:57	    #6 0x7fe3f749db96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
error	23-Nov-2020 18:53:57
error	23-Nov-2020 18:53:57	previously allocated by thread T0 here:
error	23-Nov-2020 18:53:57	    #0 0x7fe3f8a4dd28 in __interceptor_calloc (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xded28)
error	23-Nov-2020 18:53:57	    #1 0x55f825998cb7 in lde ldpd/lde.c:151
error	23-Nov-2020 18:53:57	    #2 0x55f82598a289 in main ldpd/ldpd.c:320
error	23-Nov-2020 18:53:57	    #3 0x7fe3f749db96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
error	23-Nov-2020 18:53:57

The fix is to put this in global space.

Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp added a commit that referenced this pull request Dec 19, 2020
error	26-Nov-2020 14:35:02	ERROR: AddressSanitizer: heap-use-after-free on address 0x631000024838 at pc 0x55cefae977e9 bp 0x7ffdd3546860 sp 0x7ffdd3546850
error	26-Nov-2020 14:35:02	READ of size 4 at 0x631000024838 thread T0
error	26-Nov-2020 14:35:02	    #0 0x55cefae977e8 in ldpe_imsg_compose_parent_sync ldpd/ldpe.c:256
error	26-Nov-2020 14:35:02	    #1 0x55cefae9ab13 in vlog ldpd/log.c:53
error	26-Nov-2020 14:35:02	    #2 0x55cefae9b21f in log_info ldpd/log.c:102
error	26-Nov-2020 14:35:02	    #3 0x55cefae96eae in ldpe_shutdown ldpd/ldpe.c:237
error	26-Nov-2020 14:35:02	    #4 0x55cefae99254 in ldpe_dispatch_main ldpd/ldpe.c:585
error	26-Nov-2020 14:35:02	    #5 0x55cefaf93875 in thread_call lib/thread.c:1681
error	26-Nov-2020 14:35:02	    #6 0x55cefae97304 in ldpe ldpd/ldpe.c:136
error	26-Nov-2020 14:35:02	    #7 0x55cefae5a2e2 in main ldpd/ldpd.c:322
error	26-Nov-2020 14:35:02	    #8 0x7f4ef0c33b96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
error	26-Nov-2020 14:35:02	    #9 0x55cefae525e9 in _start (/usr/lib/frr/ldpd+0xb35e9)
error	26-Nov-2020 14:35:02
error	26-Nov-2020 14:35:02	0x631000024838 is located 65592 bytes inside of 65632-byte region [0x631000014800,0x631000024860)
error	26-Nov-2020 14:35:02	freed by thread T0 here:
error	26-Nov-2020 14:35:02	    #0 0x7f4ef21e37a8 in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xde7a8)
error	26-Nov-2020 14:35:02	    #1 0x55cefae96e91 in ldpe_shutdown ldpd/ldpe.c:234
error	26-Nov-2020 14:35:02	    #2 0x55cefae99254 in ldpe_dispatch_main ldpd/ldpe.c:585
error	26-Nov-2020 14:35:02	    #3 0x55cefaf93875 in thread_call lib/thread.c:1681
error	26-Nov-2020 14:35:02	    #4 0x55cefae97304 in ldpe ldpd/ldpe.c:136
error	26-Nov-2020 14:35:02	    #5 0x55cefae5a2e2 in main ldpd/ldpd.c:322
error	26-Nov-2020 14:35:02	    #6 0x7f4ef0c33b96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
error	26-Nov-2020 14:35:02
error	26-Nov-2020 14:35:02	previously allocated by thread T0 here:
error	26-Nov-2020 14:35:02	    #0 0x7f4ef21e3d28 in __interceptor_calloc (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xded28)
error	26-Nov-2020 14:35:02	    #1 0x55cefae9725d in ldpe ldpd/ldpe.c:127
error	26-Nov-2020 14:35:02	    #2 0x55cefae5a2e2 in main ldpd/ldpd.c:322
error	26-Nov-2020 14:35:02	    #3 0x7f4ef0c33b96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)

Clean this problem up in the same way as the previous commit

Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp added a commit that referenced this pull request Jan 21, 2021
When zebra is running with debugs turned on there
is a use after free reported by the address sanitizer:

2020/10/16 12:58:02 ZEBRA: rib_delnode: (0:254):4.5.6.16/32: rn 0x60b000026f20, re 0x6080000131a0, removing
2020/10/16 12:58:02 ZEBRA: rib_meta_queue_add: (0:254):4.5.6.16/32: queued rn 0x60b000026f20 into sub-queue 3
=================================================================
==3101430==ERROR: AddressSanitizer: heap-use-after-free on address 0x608000011d28 at pc 0x555555705ab6 bp 0x7fffffffdab0 sp 0x7fffffffdaa8
READ of size 8 at 0x608000011d28 thread T0
    #0 0x555555705ab5 in re_list_const_first zebra/rib.h:222
    #1 0x555555705b54 in re_list_first zebra/rib.h:222
    #2 0x555555711a4f in process_subq_route zebra/zebra_rib.c:2248
    #3 0x555555711d2e in process_subq zebra/zebra_rib.c:2286
    #4 0x555555711ec7 in meta_queue_process zebra/zebra_rib.c:2320
    #5 0x7ffff74701f7 in work_queue_run lib/workqueue.c:291
    #6 0x7ffff7450e9c in thread_call lib/thread.c:1581
    #7 0x7ffff738eaf7 in frr_run lib/libfrr.c:1099
    #8 0x55555561a578 in main zebra/main.c:455
    #9 0x7ffff7079cc9 in __libc_start_main ../csu/libc-start.c:308
    #10 0x5555555e3429 in _start (/usr/lib/frr/zebra+0x8f429)
0x608000011d28 is located 8 bytes inside of 88-byte region [0x608000011d20,0x608000011d78)
freed by thread T0 here:
    #0 0x7ffff768bb6f in __interceptor_free (/lib/x86_64-linux-gnu/libasan.so.6+0xa9b6f)
    #1 0x7ffff739ccad in qfree lib/memory.c:129
    #2 0x555555709ee4 in rib_gc_dest zebra/zebra_rib.c:746
    #3 0x55555570ca76 in rib_process zebra/zebra_rib.c:1240
    #4 0x555555711a05 in process_subq_route zebra/zebra_rib.c:2245
    #5 0x555555711d2e in process_subq zebra/zebra_rib.c:2286
    #6 0x555555711ec7 in meta_queue_process zebra/zebra_rib.c:2320
    #7 0x7ffff74701f7 in work_queue_run lib/workqueue.c:291
    #8 0x7ffff7450e9c in thread_call lib/thread.c:1581
    #9 0x7ffff738eaf7 in frr_run lib/libfrr.c:1099
    #10 0x55555561a578 in main zebra/main.c:455
    #11 0x7ffff7079cc9 in __libc_start_main ../csu/libc-start.c:308
previously allocated by thread T0 here:
    #0 0x7ffff768c037 in calloc (/lib/x86_64-linux-gnu/libasan.so.6+0xaa037)
    #1 0x7ffff739cb98 in qcalloc lib/memory.c:110
    #2 0x555555712ace in zebra_rib_create_dest zebra/zebra_rib.c:2515
    #3 0x555555712c6c in rib_link zebra/zebra_rib.c:2576
    #4 0x555555712faa in rib_addnode zebra/zebra_rib.c:2607
    #5 0x555555715bf0 in rib_add_multipath_nhe zebra/zebra_rib.c:3012
    #6 0x555555715f56 in rib_add_multipath zebra/zebra_rib.c:3049
    #7 0x55555571788b in rib_add zebra/zebra_rib.c:3327
    #8 0x5555555e584a in connected_up zebra/connected.c:254
    #9 0x5555555e42ff in connected_announce zebra/connected.c:94
    #10 0x5555555e4fd3 in connected_update zebra/connected.c:195
    #11 0x5555555e61ad in connected_add_ipv4 zebra/connected.c:340
    #12 0x5555555f26f5 in netlink_interface_addr zebra/if_netlink.c:1213
    #13 0x55555560f756 in netlink_information_fetch zebra/kernel_netlink.c:350
    #14 0x555555612e49 in netlink_parse_info zebra/kernel_netlink.c:941
    #15 0x55555560f9f1 in kernel_read zebra/kernel_netlink.c:402
    #16 0x7ffff7450e9c in thread_call lib/thread.c:1581
    #17 0x7ffff738eaf7 in frr_run lib/libfrr.c:1099
    #18 0x55555561a578 in main zebra/main.c:455
    #19 0x7ffff7079cc9 in __libc_start_main ../csu/libc-start.c:308
SUMMARY: AddressSanitizer: heap-use-after-free zebra/rib.h:222 in re_list_const_first

This is happening because we are using the dest pointer after a call into
rib_gc_dest.  In process_subq_route, we call rib_process() and if the
dest is deleted dest pointer is now garbage.  We must reload the
dest pointer in this case.

Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp added a commit that referenced this pull request Jan 21, 2021
Current code in bgp bestpath selection would accept the newest
locally originated path as the best path.  Making the selection
non-deterministic.  Modify the code to always come to the
same bestpath conclusion when you have multiple locally originated
paths in bestpath selection.

Before:

eva# conf
eva(config)# router bgp 323
eva(config-router)# address-family ipv4 uni
eva(config-router-af)# redistribute connected
eva(config-router-af)# network 192.168.161.0/24
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:02:52 2020
eva(config-router-af)# no redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (First path received)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#  redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #2, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:03:32 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#

Notice the route choosen depends on order received

Fixed behavior:

eva# conf
eva(config)# router bgp 323
eva(config-router)# address-family ipv4 uni
eva(config-router-af)# redistribute connected
eva(config-router-af)# network 192.168.161.0/24
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:02:52 2020
eva(config-router-af)# no redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (First path received)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#  redistribute connected
eva(config-router-af)# do show bgp ipv4 uni 192.168.161.0
BGP routing table entry for 192.168.161.0/24
Paths: (2 available, best #2, table default)
  Not advertised to any peer
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin incomplete, metric 0, weight 32768, valid, sourced
      Last update: Wed Sep 16 15:03:32 2020
  Local
    0.0.0.0(eva) from 0.0.0.0 (192.168.161.245)
      Origin IGP, metric 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (Origin)
      Last update: Wed Sep 16 15:03:03 2020
eva(config-router-af)#

Ticket: CM-31490
Found-by: Trey Aspelund <taspelund@nvidia.com>
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp pushed a commit that referenced this pull request Jan 22, 2021
rfc7999:
A BGP speaker receiving an announcement tagged with the BLACKHOLE
community SHOULD add the NO_ADVERTISE or NO_EXPORT community as
defined in [RFC1997], or a similar community, to prevent propagation
of the prefix outside the local AS.  The community to prevent
propagation SHOULD be chosen according to the operator's routing
policy.

Sent:
```
router bgp 65534
 no bgp ebgp-requires-policy
 neighbor 192.168.0.2 remote-as 65030
 !
 address-family ipv4 unicast
  redistribute connected
  neighbor 192.168.0.2 route-map spine out
 exit-address-family
 !
!
ip prefix-list self seq 5 permit 192.168.100.1/32
!
route-map spine permit 10
 match ip address prefix-list self
 set community blackhole
!
```

Received:
```
spine1-debian-9# show ip bgp 192.168.100.1/32
BGP routing table entry for 192.168.100.1/32
Paths: (1 available, best #1, table default, inform peer to blackhole prefix)
  Not advertised to any peer
  65534
    192.168.0.1 from 192.168.0.1 (192.168.100.1)
      Origin incomplete, metric 0, valid, external, best (First path received)
      Community: blackhole no-advertise
      Last update: Thu Jan 21 12:56:39 2021
```

Signed-off-by: Donatas Abraitis <donatas.abraitis@gmail.com>
donaldsharp added a commit that referenced this pull request Mar 6, 2021
When dumping data about prefixes in bgp.  Let's dump the
rpki validation state as well:

Output if rpki is turned on:
janelle# show rpki prefix 2003::/19
Prefix                                   Prefix Length  Origin-AS
2003::                                      19 -  19         3320
janelle# show bgp ipv6 uni 2003::/19
BGP routing table entry for 2003::/19
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  15096 6939 3320
    ::ffff:4113:867a from 65.19.134.122 (193.72.216.231)
    (fe80::e063:daff:fe79:1dab) (used)
      Origin IGP, valid, external, best (First path received), validation-state: valid
      Last update: Sat Mar  6 09:20:51 2021
janelle# show rpki prefix 8.8.8.0/24
Prefix                                   Prefix Length  Origin-AS
janelle# show bgp ipv4 uni 8.8.8.0/24
BGP routing table entry for 8.8.8.0/24
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  100.99.229.142
  15096 6939 15169
    65.19.134.122 from 65.19.134.122 (193.72.216.231)
      Origin IGP, valid, external, best (First path received), validation-state: not found
      Last update: Sat Mar  6 09:21:25 2021

Example output when rpki is not configured:
eva# show bgp ipv4 uni 8.8.8.0/24
BGP routing table entry for 8.8.8.0/24
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  janelle(192.168.161.137)
  64539 15096 6939 15169
    192.168.161.137(janelle) from janelle(192.168.161.137) (192.168.44.1)
      Origin IGP, valid, external, bestpath-from-AS 64539, best (First path received)
      Last update: Sat Mar  6 09:33:51 2021

Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.