FreeBSD: Routes are not updated in kernel if nexthop changes #20
Description
Issue by mwinter-osr
Monday Dec 12, 2016 at 09:24 GMT
Originally opened as https://github.com/opensourcerouting/cumulus-private_quagga/issues/14
Tested based on stable/2.0, commit 2580192 (Dec 8, 2016)
Test results point this to be a common zebra issue (Seen similar failure on ISIS/OSPF/BGP4), but description here is based on BGP4
A new route gets installed correctly to the Kernel, but an update with a new nexthop is not installed.
Example Tests:
ANVL BGP4-9.5(and others)- BGPTOOL (git
https://git.netdef.org/scm/netdef/bgptool.git, testtest_2ibgp_localpref)
Running the test_2ibgp_localpref from the bgptool will send the same route from 2 peers, with the 2nd announcement having a higher local pref and a different nexthop. The 1st route gets installed into the kernel, but the 2nd route is not installed in the kernel
testroute is 30.0.9.0/24:
comp2-dut# sh ip bgp
BGP table version is 2, local router ID is 192.168.1.101
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*>i30.0.9.0/24 192.168.2.1 160 0 i
* i 192.168.1.1 150 0 i
Displayed 1 out of 2 total prefixes
and
comp2-dut# sh ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, P - PIM, T - Table, v - VNC,
V - VPN,
> - selected route, * - FIB route
C>* 1.1.1.0/24 is directly connected, eth1
B>* 30.0.9.0/24 [200/0] via 192.168.2.1, eth3, 00:00:03
C>* 192.168.0.101/32 is directly connected, lo
C>* 192.168.1.0/24 is directly connected, eth2
C>* 192.168.2.0/24 is directly connected, eth3
C>* 192.168.3.0/24 is directly connected, eth4
C>* 192.168.122.0/23 is directly connected, eth0
But FreeBSD Kernel shows:
root@comp2-dut:~# netstat -rn
Routing tables
Internet:
Destination Gateway Flags Netif Expire
1.1.1.0/24 link#2 U eth1
1.1.1.2 link#2 UHS lo
30.0.9.0/24 192.168.1.1 UG1 eth2
127.0.0.1 link#7 UH lo
192.168.0.101 link#7 UH lo
[...]
From bgpd.log:
2016/12/12 01:21:16 BGP: 192.168.1.1 rcvd UPDATE w/ attr: nexthop 192.168.1.1, origin i, localpref 150, path
2016/12/12 01:21:16 BGP: 192.168.1.1 rcvd UPDATE wlen 0 attrlen 21 alen 4
2016/12/12 01:21:16 BGP: 192.168.1.1 rcvd 30.0.9.0/24
2016/12/12 01:21:17 BGP: Tx IPv4 route add VRF 0 30.0.9.0/24 metric 0 tag 0 count 1
2016/12/12 01:21:17 BGP: IPv4 [nexthop 1] 192.168.1.1
2016/12/12 01:21:23 BGP: 192.168.1.1 KEEPALIVE rcvd
2016/12/12 01:21:23 BGP: 192.168.1.1 KEEPALIVE rcvd
2016/12/12 01:21:27 BGP: 192.168.2.1 rcvd UPDATE w/ attr: nexthop 192.168.2.1, origin i, localpref 160, path
2016/12/12 01:21:27 BGP: 192.168.2.1 rcvd UPDATE wlen 0 attrlen 21 alen 4
2016/12/12 01:21:27 BGP: 192.168.2.1 rcvd 30.0.9.0/24
2016/12/12 01:21:27 BGP: Tx IPv4 route add VRF 0 30.0.9.0/24 metric 0 tag 0 count 1
2016/12/12 01:21:27 BGP: IPv4 [nexthop 1] 192.168.2.1
and zebra.log:
2016/12/12 01:21:17 ZEBRA: zebra message comes from socket [13]
2016/12/12 01:21:17 ZEBRA: zebra message received [ZEBRA_IPV4_ROUTE_ADD] 25 in VRF 0
2016/12/12 01:21:17 ZEBRA: 0:30.0.9.0/24: Inserting route rn 0x8025d5600, rib 0x8025d0860 (type 9) existing 0x0
2016/12/12 01:21:17 ZEBRA: 0:30.0.9.0/24: Adding route rn 0x8025d5600, rib 0x8025d0860 (type 9)
2016/12/12 01:21:17 ZEBRA: kernel_rtm_ipv4: 30.0.9.0/24: successfully did NH 192.168.1.1
2016/12/12 01:21:17 ZEBRA: 0:30.0.9.0/24: Redist update rib 0x8025d0860 (type 9), old 0x0 (type -1)
2016/12/12 01:21:17 ZEBRA: 0:192.168.1.1/32: Evaluate RNH, type 0
2016/12/12 01:21:17 ZEBRA: 0:192.168.2.1/32: Evaluate RNH, type 0
2016/12/12 01:21:17 ZEBRA: Kernel: Len: 200 Type: RTM_ADD
2016/12/12 01:21:17 ZEBRA: Kernel: UP GATEWAY DONE PROTO1
2016/12/12 01:21:17 ZEBRA: Kernel: message seq 19
2016/12/12 01:21:17 ZEBRA: Kernel: pid 41959, rtm_addrs 0x7
2016/12/12 01:21:17 ZEBRA: rtm_read: got rtm of type 1 (RTM_ADD)
2016/12/12 01:21:17 ZEBRA: rtm_read: RTM_ADD 30.0.9.0/24: done Ok
2016/12/12 01:21:17 ZEBRA: rib_lookup_and_dump: rn 0x8025d5600, rib 0x8025d0860: NOT removed, selected
2016/12/12 01:21:17 ZEBRA: rib_lookup_and_dump: dumping RIB entry 0x8025d0860 for 30.0.9.0/24 vrf 0
2016/12/12 01:21:17 ZEBRA: rib_lookup_and_dump: refcnt == 0, uptime == 1481534477, type == 9, instance == 0, table == 0
2016/12/12 01:21:17 ZEBRA: rib_lookup_and_dump: metric == 0, mtu == 0, distance == 200, flags == 25, status == 10
2016/12/12 01:21:17 ZEBRA: rib_lookup_and_dump: nexthop_num == 1, nexthop_active_num == 1, nexthop_fib_num == 0
2016/12/12 01:21:17 ZEBRA: rib_lookup_and_dump: NH 192.168.1.1 with flags ACTIVE FIB
2016/12/12 01:21:17 ZEBRA: rib_lookup_and_dump: dump complete
2016/12/12 01:21:18 ZEBRA: Kernel: Len: 168 Type: RTM_MISS
2016/12/12 01:21:18 ZEBRA: Kernel: DONE
2016/12/12 01:21:18 ZEBRA: Kernel: message seq 0
2016/12/12 01:21:18 ZEBRA: Kernel: pid 0, rtm_addrs 0x1
2016/12/12 01:21:18 ZEBRA: Unprocessed RTM_type: 7
2016/12/12 01:21:18 ZEBRA: Kernel: Len: 168 Type: RTM_MISS
2016/12/12 01:21:18 ZEBRA: Kernel: DONE
2016/12/12 01:21:18 ZEBRA: Kernel: message seq 0
2016/12/12 01:21:18 ZEBRA: Kernel: pid 0, rtm_addrs 0x1
2016/12/12 01:21:18 ZEBRA: Unprocessed RTM_type: 7
2016/12/12 01:21:27 ZEBRA: zebra message comes from socket [13]
2016/12/12 01:21:27 ZEBRA: zebra message received [ZEBRA_IPV4_ROUTE_ADD] 25 in VRF 0
2016/12/12 01:21:27 ZEBRA: 0:30.0.9.0/24: Inserting route rn 0x8025d5600, rib 0x8025d08c0 (type 9) existing 0x8025d0860
2016/12/12 01:21:27 ZEBRA: rib_delnode: 0:30.0.9.0/24: rn 0x8025d5600, rib 0x8025d0860, removing
2016/12/12 01:21:27 ZEBRA: 0:30.0.9.0/24: Updating route rn 0x8025d5600, rib 0x8025d08c0 (type 9) old 0x8025d0860 (type 9)
2016/12/12 01:21:27 ZEBRA: kernel_rtm_ipv4: odd command RTM_DELETE for flags 1
2016/12/12 01:21:27 ZEBRA: kernel_rtm_ipv4: No useful nexthops were found in RIB entry 0x8025d08c0
2016/12/12 01:21:27 ZEBRA: kernel_rtm_ipv4: No useful nexthops were found in RIB entry 0x8025d08c0
2016/12/12 01:21:27 ZEBRA: 0:30.0.9.0/24: Redist update rib 0x8025d08c0 (type 9), old 0x8025d0860 (type 9)
2016/12/12 01:21:27 ZEBRA: rib_process: 0:30.0.9.0/24: rn 0x8025d5600, removing rib 0x8025d0860
2016/12/12 01:21:27 ZEBRA: rib_unlink: 0:30.0.9.0/24: rn 0x8025d5600, rib 0x8025d0860
2016/12/12 01:21:27 ZEBRA: 0:192.168.1.1/32: Evaluate RNH, type 0
2016/12/12 01:21:27 ZEBRA: 0:192.168.2.1/32: Evaluate RNH, type 0
2016/12/12 01:21:27 ZEBRA: Kernel: Len: 200 Type: RTM_ADD
2016/12/12 01:21:27 ZEBRA: Kernel: UP GATEWAY PROTO1
2016/12/12 01:21:27 ZEBRA: Kernel: message seq 20
2016/12/12 01:21:27 ZEBRA: Kernel: pid 41959, rtm_addrs 0x7
2016/12/12 01:21:28 ZEBRA: Kernel: Len: 168 Type: RTM_MISS
2016/12/12 01:21:28 ZEBRA: Kernel: DONE
2016/12/12 01:21:28 ZEBRA: Kernel: message seq 0
2016/12/12 01:21:28 ZEBRA: Kernel: pid 0, rtm_addrs 0x1
2016/12/12 01:21:28 ZEBRA: Unprocessed RTM_type: 7
2016/12/12 01:21:28 ZEBRA: Kernel: Len: 168 Type: RTM_MISS
2016/12/12 01:21:28 ZEBRA: Kernel: DONE
2016/12/12 01:21:28 ZEBRA: Kernel: message seq 0
2016/12/12 01:21:28 ZEBRA: Kernel: pid 0, rtm_addrs 0x1
2016/12/12 01:21:28 ZEBRA: Unprocessed RTM_type: 7