Mastering Default Routing on Linux

Connecting a Linux server to a LAN or the Internet requires careful configuration of routing rules. The default route plays a critical role in traffic flow, directing packets when there is no explicit match for the destination. As a network engineer or Linux system administrator, understanding in-depth management of the default gateway is a crucial skill.

This extensive 3200+ word guide aims to fully cover default routing on Linux. Follow along as we dive into traffic flow analysis, troubleshooting connectivity issues, utilizing advanced features like metrics and equal-cost multipath routing, and explore alternatives to the standard static default route.

Grasping these Linux routing concepts will provide expertise to handle real-world network deployments and outages.

Overview of Default Routing

Before jumping into syntax and commands, let‘s briefly recap the role of the default route:

• The routing table consists of destination network prefixes matched to gateways used to reach them.

• When a packet comes in and does not match any route prefixes, the system forwards it to the default gateway.

• This gateway (router) handles sending the traffic to its ultimate destination on another network.

Without this route defined, attempts to communicate with unfamiliar destinations will fail. Configuring the default route enables Internet access and connectivity to other networks.

Here is a simple example routing table with a default route:

Destination     Gateway         Interface
0.0.0.0/0       10.0.0.1        eth0
10.0.0.0/24     On-link         eth0
172.16.1.0/24   On-link         eth1

The first entry using 0.0.0.0/0 matches all destination IP addresses and sends traffic towards 10.0.0.1 from the eth0 interface. This handles reaching anything unavailable directly through the local subnet or other defined routes.

Verifying the Current Default Gateway

Before making any changes, we should verify the current default route. Use either ip route or route -n to check the routing table:

$ ip route show
default via 10.0.2.2 dev eth0 proto dhcp metric 100  
10.0.2.0/24 dev eth0 proto kernel scope link src 10.0.2.15
172.16.1.0/24 dev eth1 proto kernel scope link src 172.16.1.10

Here we see traffic to unfamiliar destinations goes through 10.0.2.2 out the eth0 interface. Let‘s break down some key attributes:

• default – Signifies this is the default route
• via 10.0.2.2 – The nexthop IP address
• dev eth0 – Egress interface
• proto dhcp – Route was auto-configured via DHCP
• metric 100 – Route priority

With this one rule, we can reach the rest of the routes in our infrastructure and Internet. But if this route gets deleted or incorrect, wide-scale outage occurs.

Traffic Flow Analysis

We can leverage packet captures to visualize traffic flow and confirm routing. For example, run tcpdump on the gateway router while pinging Google‘s DNS server from the Linux server:

$ tcpdump -ni eth0 host 8.8.8.8

This shows the ICMP and UDP traffic getting encapsulated and forwarded through the 10.0.2.2 gateway, verifying connectivity.

Complement this with traces and logs on the server side:

$ traceroute 8.8.8.8
$ tail -f /var/log/messages | grep -i network

Graphing communication paths and cross-referencing logs builds strong troubleshooting skills.

Now let‘s explore how to manipulate the default route…

Adding a New Default Gateway

Use the ip route add command to insert a default route, specifying the nexthop gateway and egress interface:

$ sudo ip route add default via 192.168.1.1 dev eth1

This configures a default route through the gateway 192.168.1.1 out the eth1 interface.

Verify the new rule lands in our routing table:

$ ip route show 
default via 192.168.1.1 dev eth1
10.0.2.0/24 dev eth0 proto kernel scope link src 10.0.2.15 
172.16.1.0/24 dev eth1 proto kernel scope link src 172.16.1.10

Success! All traffic without an explicit route now flows through 192.168.1.1 from eth1.

Note: These modifications only persist until the next reboot. To make permanent, must configure network scripts outlined later.

Deleting a Default Gateway

Removing an existing default gateway is just as straightforward using ip route delete:

$ sudo ip route delete default

This completely deletes the route without specifying a particular nexthop or interface.

For targeting a specific default gateway such as 192.168.1.1, use:

$ sudo ip route delete default via 192.168.1.1

And verify the route removed from routing table output.

Persisting Default Routes

There is one major catch to using ip route – changes do not withstand restarts or network disruption!

To persist default gateway configuration, the details must get saved in network configuration files and scripts. Unfortunately exact formats vary widely:

• Ubuntu Desktop – /etc/netplan/ yaml files
• RHEL/CentOS 7 – /etc/sysconfig/network-scripts/ ifcfg files
• RHEL/CentOS 8 – /etc/NetworkManager/system-connections/ nmconnection files
• Arch Linux – /etc/systemd/network/ networking configs
• And many more variants…

Here are some concrete examples for adding a persistent default route on different distros:

Ubuntu 20.04 Static Route

1. Edit /etc/netplan/01-netcfg.yaml:

   network:
     version: 2
     ethernets:
       eno1:
         gateway4: 192.168.1.1 
         routes:
         - to: default
           via: 192.168.1.1

2. Run sudo netplan apply

RHEL/CentOS 7 Default Gateway

1. Edit /etc/sysconfig/network-scripts/ifcfg-eth0:

   GATEWAY=192.168.1.1

2. Restart network service:

   $ sudo systemctl restart network

RHEL/CentOS 8 NetworkManager

1. Create /etc/NetworkManager/system-connections/default-eth1.nmconnection:

   [ipv4]
   method=auto
   routes=0.0.0.0/0,192.168.1.1

2. Restart NetworkManager:

   $ sudo systemctl restart NetworkManager

This covers some common scenarios, but many other network files and tools exist like systemd-networkd. The key is finding where default route configuration gets stored on your Linux distribution.

Troubleshooting Connectivity Issues

If running into network issues after changing routes:

• Trace Paths – traceroute 8.8.8.8 watch traffic stops

• Ping Remote Hosts – ping google.com tests DNS and connectivity

• Ping Next Hops – ping 192.168.1.1 verifies gateway responsiveness

• Review Interface Configuration – ip addr show check for IPs

• Check System Logs – grep network /var/log/messages

Also confirm cables plugged in, interface brought up properly, no firewall rules blocking traffic, and test from another machine on same network.

Methodically verifying each step identifies the failure point.

Leveraging iptables for Traffic Control

The Linux kernel‘s iptables firewall provides powerful routing policies. For example, to prefer routing Internet traffic through eth0 instead of eth1:

$ sudo iptables -A OUTPUT -o eth1 -p tcp \
    --dport 80 -j REDIRECT --to-port 8080

This redirects outbound HTTP connections from eth1 to port 8080 locally, failing cleanly. Smarter firewall rules combined with multiple default gateways provide flexible traffic shaping.

Default Gateway Metric Priority

When defining multiple default routes, we can assign metric values to influence precedence. The lower the metric, the higher priority selecting that route:

$ sudo ip route add default via 10.0.0.1 metric 10
$ sudo ip route add default via 192.168.1.1 metric 20 

The route through 10.0.0.1 now has first priority for matching traffic because of a lower metric value.

If we deleted the first route, all packets would start flowing via 192.168.1.1 instead. Defining route priority is useful for traffic engineering and failover scenarios.

Equal Cost Multipath (ECMP) Routing

ECMP allows next-hop packet forwarding to occur across multiple equal routes simultaneously through a routing algorithm. This achieves better load balancing compared to relying on one single default gateway.

Here is an ECMP configuration with a default route split across two gateways:

 default via 10.0.0.1 dev eth0 proto static metric 10
 default via 10.1.0.1 dev eth1 proto static metric 10

By assigning both routes a low equal cost metric of 10, outbound traffic will balance across both gateways. So long as the networks path equally to the destination, flows should distribute 1:1.

Monitoring traffic volume with iftop on each interface verifies equal splitting. Keep in mind return paths may skew based on external router hashing algorithms.

ECMP scales best across high bandwidth links when datacenters connect to redundant dedicated routers. But even simple configurations can better utilize multiple WAN links.

Leveraging Dynamic Routing Protocols

Thus far we have only discussed static default routing. However, Linux supports dynamic interior gateway protocols like OSPF and BGP. Instead of manually configuring routes, these complex routing algorithms automatically share network topology and policy:

• Quagga – Provides OSPF, RIP, BGP configured through vtysh CLI
• FRR – More modern fork improving on Quagga
• BIRD – BGP focused daemon for full routing table control
• GoBGP – Next-gen BGP stack implemented in Go language

Explaining protocol nuances requires dedicated guides. Check best practices depending on scale and needs before deploying in production environments. But huge efficiency gains possible by intelligently handling dynamic routes.

High Availability Configurations

Mission critical infrastructure demands high reliability from routing. Instead of a single default gateway, we can provide redundancy through Virtual Router Redundancy Protocol (VRRP):

interface: eth0
    backup: 192.168.1.2
    master: 192.168.1.1

If the master router at 192.168.1.1 fails, the backup automatically takes over.

When configuring highly available default gateways, also consider:

• Ping or arping monitoring scripts to rapidly handle failover
• Redundant physical line cards and fiber paths
• Leveraging anycast IPs for multipath efficiency
• Running routing daemons on backup devices
• Session preservation for zero packet loss failover

Enterprise-grade solutions provide many additional failover optimization mechanisms, at greater complexity.

Configuring Default Routes via nmcli

Rather than directly editing network configuration files, Linux provides alternatives like NetworkManager‘s nmcli tool:

$ sudo nmcli connection add ifname eth1 type ethernet \
   ipv4.routes "0.0.0.0/0 192.168.1.1"

This quickly creates a new connection named "ethernet-eth1" with our default route. To instantiate it:

$ sudo nmcli connection up "ethernet-eth1"

The nmcli tool manages NetworkManager connections from the CLI. Changes persist across reboots providing a simple configuration method.

See the nmcli manpage for advanced capability like bonding, bridging, teaming, and full LifeCycle management control.

Summary

Configuring the Linux default network gateway provides the foundation for infrastructure access and DNS resolution. Carefully managing this route prevents nasty outages and customer impacting critical systems reliance.

We covered analyzing traffic flow,Adding and removing default routes, route prioritization, monitoring and troubleshooting connectivity issues leveraging iptables for more advanced traffic steering up to highly redundant enterprise solutions and different configuration tooling like nmcli.

With routing table mastery, we can securely architect networks to deliver business results. I hope this comprehensive deep dive into the world of Linux default gateways proves useful for your system administration career!