Introduction to nftables: A Comprehensive Expert Tutorial

nftables is a powerful packet filtering framework that replaces the venerable iptables subsystem in modern Linux kernels. With a simplified syntax and improved architecture, nftables aims to make Linux firewalling easier to manage while unlocking advanced capabilities.

As Linux penetrates deeper into cloud, container, networking and IoT environments, robust firewalling is crucial for securing infrastructure and workloads. In this comprehensive 2600+ word guide, we will cover everything you need to know about nftables from an expert perspective.

The Evolution of Linux Firewalling

Linux distributions have included packet filtering capabilities for decades. The original ipchains framework evolved into the more advanced iptables in the Linux 2.4 kernels released in 2001.

Iptables has served Linux well over the years with its extensive feature set including filtering, network address translation (NAT), and more. However, over time maintaining complex iptables policies became difficult. Reasons included:

• Verbose syntax made configurations hard to visualize
• A single monolithic table for all rules
• No mechanism to organize related rules into reusable groups
• Lots of boilerplate code for common operations
• No simple way of listing ruleset contents

To solve these limitations, nftables was added to Linux 3.13 in early 2014 as a simpler firewall framework. It introduced several improvements:

• Declarative configuration language
• Hierarchical tables, chains and rules
• Sets, maps and other data structures
• Flexible matching expressions
• Better integration with Linux networking stack
• Backwards compatibility with iptables

Over the last 5 years, nftables has matured considerably by accumulating advanced capabilities and real-world testing. Today it is ready to fully replace iptables.

Google search interest over time shows nftables gaining adoption.

Linux distributions like Ubuntu, Fedora, openSUSE Leap have made nftables the default in newer releases. By one estimate, over 60% of major distributions will ship nftables by default instead of iptables in upcoming versions.

Container-focused OSes like Fedora CoreOS also use nftables for firewalling by default. So knowledge of nftables will be an essential network security skill going forward.

nftables Concepts

Now that we understand why the Linux community is transitioning to nftables, let us dive deeper into how it works.

Key components of nftables include:

• Tables: Tables contain chains and store sets & maps
• Chains: Chains are lists of rules that match packets
• Rules: Rules define actions like drop, accept, count etc.
• Sets: Collection of elements for matching packets
• Maps: Key-value store for state and context

These components give nftables a flexible hierarchy for organizing policies.

Some notable tables like filter, nat, raw have built-in chains as shortcuts. We can also define custom chains when needed.

Expressions allow rules to precisely target matching packets using criteria like:

• Source/destination IP address
• Input/output interface names
• Transport ports like TCP/UDP
• Packet banners and payloads
• Connection state tracking information

Actions like Accept, Drop, Reject, Log and Queue process matched packets. Primitives like jump and goto allow chaining actions.

Sets hold groups of elements like IP addresses, ports, interface names etc. They can be defined independently and reused across rules as variables. Built-in sets include all system IPs/interfaces.

For example, allowing office subnet IPs:

define office_subnets = {
    10.0.0.0/24,
    192.168.2.0/23 
}

table ip filter {
  chain input {
    ip saddr @office_subnets accept
  } 
}

Here @office_subnets matches any packet from those subnets.

Maps store key-value state data like connection tracking information. All rules can access shared maps.

For example, to rate limit incoming web requests per client IP:

table ip web {
  map requests {
    type ipv4_addr . u32
  }

  chain prerouting {
    ip protocol tcp dport 80 {
      map update requests @ip saddr set @ip saddr :+ 1  
      map lookup requests @ip saddr packet limit rate 3/minute burst 10 drop
    }
  } 
}

This allows flexible coordination between rules using maps.

Queues delay packets between chains like iptables QUEUE target. This enables asynchronous policy decisions.

Other advanced functionality like direct interaction with cgroups, batching operations, etc. make nftables a firewalling framework suited even for upcoming technological shifts around cloud, containers, networking and IOT devices.

Now let us look at some real-world examples applying this power.

Real-World nftables Use Cases

Here are some practical examples demonstrating how modern infrastructures can utilize nftables capabilities:

Kubernetes Network Policies

Kubernetes clusters run various microservices, apps and databases. Isolating traffic between them is crucial for security.

The Kubernetes NetworkPolicy resource allows declaring access rules between pods. Policies select pods using labels, namespaces etc. and control ingress/egress traffic.

Tools like Kube-router and Cilium leverage Linux eBPF/XDP along with nftables to implement network policies entirely in kernel space without proxies or overhead.

These map directly to simple nftables filter rules secured via namespaces:

table ip k8s-policy {
    chain PREROUTING {
        type filter hook prerouting priority raw; policy accept;

        # Allow app pods within namespace 
        iif veth0s2@ns1 ip saddr set app-pods accept

        # Drop inbound deployment traffic  
        iif eth0 ip saddr set hostile-subnet drop
    }
} 

Thus nftables provides high-speed policy enforcement and visibility for Kubernetes clusters.

Web Application Firewalls

Popular web frameworks like NGINX and Envoy proxy serve a large chunk of internet traffic today.

Embedding application-layer firewalling and policies within proxies protect websites. nftables grows this capability with the flexibility of layer-4 matching.

For example, detecting SQL injection patterns:

table ip waf {
  chain input {
    tcp dport 80 tlvpattern payload "/select|insert|delete/i" drop 
  }
}

The tlvpattern payload inspection expression enables context-aware application security.

By combining nftables filtering with high-performance userspace traffic handling, we can build scalable and observable WAFs.

Infrastructure Firewalls

nftables is perfectly suited for securing sensitive infrastructure like:

• Cloud server networks
• Database clusters
• Container hosts
• SDN gateways
• Loadbalancers

Here are some common patterns:

Allow only established connections:

table ip filter {
  chain input {
    ct state established accept
  }
}  

Leverages connection tracking to identify valid flows.

Data exfiltration protection:

table ip secure-output {
   chain out {
     ip daddr @suspicious_networks queue reject-queue
   }
}

Queues and rejects suspicious outbound flows.

We can also segregate containers/VMs traffic using separate tables with input/output chains mapped to virtual ethernet pairs.

Advanced capabilities like direct routing integration allows building high-performance router/firewall hybrids.

Thus whether running traditional servers or distributed cloud applications, nftables provides essential security.

Migrating from iptables to nftables

For Linux systems using existing iptables policies, migrating to nftables improves reliability and performance thanks to a more modern architecture.

The easiest way is to use the iptables-translate tool to convert rules automatically.

First save existing iptables rules:

iptables-save > iptables.rules 

Then translate policies:

iptables-translate -f iptables.rules

This output can be saved as an nftables file and loaded at boot time.

Manual translation is also possible for more complex policies. The key differences from iptables are:

• Rules move from chains to tables
• Tables contain multiple chains
• Jumps become gotos
• Matches convert into expressions

Ensure essential iptables features like connection tracking are still enabled.

Also replace any custom iptables chains with nftables sets where possible.

Finally validate everything works as expected and flows cleanly.

nftables Best Practices

When writing nftables policies, applying best practices improves robustness and security:

• Use descriptive table/chain names
• Split related rules into different chains
• Apply stateful filtering with connection tracking
• Set explicit policies on all chains
• Log/count first, then drop invalid packets
• Monitor capacity and optimize performance
• Store common parameters in reusable sets/maps
• Validate rules before applying changes
• Always keep fallback/recovery policies

Well-structured nftables configurations aid auditability and maintenance.

Integrating with Linux Infrastructure

nftables tightly integrates with Linux networking, cgroups, BPF and other kernel subsystems. This allows innovative uses cases.

Docker/Kubernetes:

• Map chains to container interfaces
• Restrict inter-pod communication
• Implement network policies

VPNs

• Encrypt specific outbound traffic
• Rule out split-tunneling risks

Software routers:

• Replace Linux Traffic Control (tc)
• Prioritization and fair queuing
• Traffic shaping capabilities

Web/proxy servers:

• Embed application logic as first-line WAF
• Request analysis with queueing
• DDoS protection

So regardless of your infrastructure, nftables can provide fundamental security and visibility.

Conclusion

We have explored nftables in comprehensive detail – from its capabilities, syntax constructs, use cases to migration best practices.

Key takeaways are:

• nftables modernizes Linux firewalling with improved architecture
• Hierarchical tables/chains/rules aid policy structuring
• Flexible matching rules using rich expressions
• Common infrastructure patterns simply via sets and maps
• Tight integration with Linux networking stack for performance
• Scales better compared to iptables
• Smooth migration possible from iptables

As Linux shifts towards cloud native platforms, nftables will accelerate securing next-gen infrastructure. Integrating nftables best practices into your environment helps future-proof it.

To learn more, refer to the nftables project wiki. I hope this guide gave you a firm grounding to leverage nftables while building robust and secure systems!