The Domain Name System (DNS) forms the foundation for connectivity on modern networks by mapping human-readable domain names like google.com to underlying IP addresses. It has become such ubiquitous infrastructure that its critical nature is often overlooked. However, DNS is a single point of failure – when it goes down, access to websites, email, and other services grinds to a halt. As such, every Linux systems administrator should have sharp troubleshooting skills for tackling DNS outages or performance issues when they inevitably crop up.

A key tool that should be part of any sysadmin‘s DNS toolkit is dig – the flexible DNS interrogation utility that comes bundled with most Linux distributions. Mastering dig allows deep inspection into your DNS infrastructure for troubleshooting, auditing or maintenance.

This comprehensive guide to installing and using dig on Debian 9 will give you DNS superpowers allowing you to debug tricky DNS issues for your infrastructure like a seasoned pro.

DNS Crash Course

Before diving into usage of dig itself, let‘s do a quick primer on DNS components and terminology that will be useful context throughout this article:

Domain Name System (DNS)

DNS provides a distributed lookup service that maps easy to remember domain names like linuxhaxor.net to hard to remember IP addresses like 172.104.28.29. This abstraction allows hosting services at human friendly locations rather than requiring endpoints be accessed directly via raw IPs.

Some key statistics on DNS adoption from [Cloudflare]:

  • Over 331 million registered domain names worldwide
  • Average of 77 billion DNS queries handled daily
  • Critical infrastructure with 99.999% uptime expected

DNS Record Types

DNS servers store different types of records that each serve distinct purposes:

  • A Records – Map hostnames to IPv4 addresses
  • AAAA Records – Map hostnames to IPv6 addresses
  • CNAME Records – Alias one name to another canonical name
  • MX Records – Define mail servers for accepting messages
  • NS Records – Define authoritative nameservers for a domain
  • SRV Records – Define services at domain and their ports
  • TXT Records – Provide arbitrary additional text data
  • CAA Records – Certification authority authorization records

There are many more exotic record types, but these make up the bulk of common records you will encounter.

DNS Resolvers

Clients have local DNS resolvers like systemd-resolved or dnsmasq that cache queries and talk to upstream recursive resolvers provided by ISPs. These in turn fan out queriesauthoritatively answer via additional hops to root and Top Level Domain (TLD) nameservers as needed.

DNS Traffic

DNS predominantly utilizes UDP port 53 for standard lookups. TCP over port 53 is used for zone transfers or very large responses and lookups.

There is also growing adoption of encryption via DNS-over-HTTPS and DNS-over-TLS especially amongst major public resolvers.

Importance of Dig for Sysadmins

As you can see even from the quick primer above, DNS can be complex with a lot of moving parts. When issues crop up troubleshooting and root causing the source of failures can become tricky.

This is where a tool like dig comes in – it allows interrogating each component of the DNS ecosystem to shine light on discrepancies and unlock what exactly is going on:

Investigate Fundamentals

  • Is the domain responding at all? Are records being returned?
  • What are query, response latency metrics?
  • Which nameservers is query being routed to?

Inspect Records

  • Verify if expected records like MX, NS, CNAMEs exist
  • Check for incorrect out of date records
  • Identify unauthorized added records

Compare Providers

  • Match same queries across multiple servers
  • Analyze performance differences
  • Uncover configuration mismatches

And much more. Dig gives visibility DNS admins need.

Installing Dig on Debian

Dig ships as part of the dnsutils package on Debian.

To install:

$ sudo apt update
$ sudo apt install dnsutils

Verify dig command is now available and print help:

$ dig -h

If you don‘t see the usage manual, double check dnsutils was installed correctly.

Now that we have dig ready, lets get into actually putting it to use across some common sysadmin tasks.

Getting Help

Dig has powerful extensive functionality – far beyond what we can reasonably cover in this guide alone.

Thankfully, the utility self-documents a lot of this via the -h flag:

$ dig -h

Common Options

Here are some of the most pertinent options sysadmins regularly utilize:

  • +trace – Enables tracing to dig deeper on failures
  • +bufsize – Sets UDP buffer size
  • +nostats – Disables statistics
  • +tcp – Forces query over TCP
  • +notcp – Disables TCP even for large queries
  • +ignore – Disables checking IPv6/IPv4 records
  • +noall – Disables showing empty records
  • +short – Disables verbosity leaving just answer

Be sure to scan through the full usage guide to uncover additional functionality that may assist for specialized troubleshooting.

Now onto actually using dig in practice.

Simple DNS Lookups

The most basic invocation of dig allows querying for standard A records:

$ dig google.com  

; <<>> DiG 9.16.6-Debian <<>> google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35146
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
...
;; QUESTION SECTION:
;google.com.            IN  A

;; ANSWER SECTION:  
google.com.     55  IN  A   142.250.191.238

We can instantly see that google.com resolves to the IP 142.250.191.238 from the configured default nameservers in /etc/resolv.conf.

In addition to this basic lookup, we also get a lot of metadata about the query and answer:

  • NOERROR status indicates successful
  • Question details
  • Authoritative answer
  • Server statistics
  • Query paths/tracing

Having insight into all these additional dimensions beyond just the raw IP returned allows diagnosing obscure issues where DNS may be functioning but exhibiting subpar performance for example.

Now that we have basic A record lookups understood, let‘s move on to more advanced record types.

Exploring Advanced DNS Record Types

Sysadmins often need to retrieve info beyond simple A/AAAA records as part of troubleshooting DNS issues or just general maintenance.

Thankfully, dig provides easy access to all record types with the same interface.

MX Records

To query the MX records by hostname:

$ dig google.com MX 

;; ANSWER SECTION:
google.com.     5183    IN  MX  40 alt4.aspmx.l.google.com.
google.com.     5183    IN  MX  30 alt3.aspmx.l.google.com.
google.com.     5183    IN  MX  20 alt2.aspmx.l.google.com.  
google.com.     5183    IN  MX  10 aspmx.l.google.com.
google.com.     5183    IN  MX  50 alt1.aspmx.l.google.com.

Here we can validate all the frontend mailservers that accept messages for the google.com domain along with their priorities. This allows troubleshooting mail delivery issues.

NS Records

Next, querying the NS records displays the nameservers in charge of the google.com DNS zone itself:

$ dig google.com NS

;; ANSWER SECTION:  
google.com.     21599   IN  NS  ns3.google.com.
google.com.     21599   IN  NS  ns4.google.com.
google.com.     21599   IN  NS  ns2.google.com.
google.com.     21599   IN  NS  ns1.google.com.

Usually verifying whether expected NS records are present eliminates an entire class of issues around missing or outdated delegations.

TXT Records

For manual administrative records, TXT entries come in handy like in this SPF policy example:

$ dig linuxhaxor.net TXT

;; ANSWER SECTION:
linuxhaxor.net.   3600    IN  TXT "v=spf1 -all"

And many more records types beyond just this small subset have dedicated functionality making them indispensable for DNS admins.

Debugging DNS Issues

Up to this point we have focused primarily on dig‘s utility for DNS operations, maintenance and auditability during normal operations.

However, the tool‘s flexible nature makes it invaluable for stressed troubleshooting scenarios when DNS issues crop up and threaten infrastructure availability and connectivity.

Here are some common issues and how dig can provide guidance during outages:

Zone Disappears Entirely

If a zone like linuxhaxor.net drops offline entirely, dig will showcase that:

$ dig linuxhaxor.net

; <<>> DiG 9.16.6-Debian <<>> linuxhaxor.net  
;; global options: +cmd   
;; connection timed out; no servers could be reached

This points us clearly at name resolution failing with no server reachable rather than say empty records being returned.

Potential causes could then be:

  • Nameservers themselves down or blocking traffic
  • Network outage cutting off DNS traffic
  • Misconfigured resolver blocking access

Having this breadcrumb eliminates tons of potential red herrings.

Latency Spikes

During intermittent DNS slowness, we can employ +stats for quantitive visibility:

$ dig +stats linuxhaxor.net  

;; Query time: 552 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Tue Jan 17 09:38:32 EST 2023
...
;; MSG SIZE  rcvd: 107

The 552 millisecond query time clearly signals sluggishness talking to this particular nameserver. We can then investigate connectivity to that specific NS and upstream performance beyond it.

Without timing quantification, such latency issues tend to be extremely tricky to identify the root cause of.

Compare Across Nameservers

When troubleshooting, administrators often want to compare lookup behavior across different servers.

Dig simplifies this via the @ syntax:

$ dig @8.8.8.8 linuxhaxor.net 
$ dig @1.1.1.1 linuxhaxor.net

This analysis can definitely highlight discrepancies such as certain nameservers returning SERVFAIL while others work fine pointing to delegation configuration issues.

Without consistency checks, administrators waste tons of time barking up the wrong trees chasing failures.

Exploring Additional Digging

We have really just scratched the surface of critical DNS issues DIG can debug here. Some additional examples:

  • Reverse lookups failing for some hosts
  • Zone serving different data to different geographic regions
  • Queries returning warning flags indicating issues
  • Heavyweight requests overflowing resolver recursors

Spend time becoming familiar with the various metadata and analysis DIG outputs during sane operations to build pattern recognition for when things go awry. This enables rapid fault isolation.

Tuning DIG Performance

When utilizing DIG to troubleshoot obtuse DNS infrastructure issues or outages, query performance and reliability matter. Thankfully, we can customize some knobs and flags to assist with stability.

A few key performance tuning options:

Increase Buffer Size
For unreliable connections, use +bufsize to allow more data without truncating:

dig +bufsize=4096 linuxhaxor.net ANY

Enable Retries
Specify number of times to retry failed lookups before giving up:

dig +retry=3 linuxhaxor.net

Force TCP Transport
TCP mode avoids UDP fragmentation and loss issues by guaranteeing full payloads:

dig +tcp linuxhaxor.net ANY

There are many additional options for tuning transfer sizes, histograms, avoiding fallback protocols etc. Tweak based on your specific environment challenges.

Dig Scripting

Up to this point, we have utilized DIG interactively on the command line. However, the tool‘s flexible nature also allows usage programatically to perform bulk querrying, checks and automated alerting.

For example, gathering metrics on an entire environment‘s DNS infrastructure reliability:

#!/bin/bash

domains=(google.com linuxhaxor.net cloudflare.com) 

for i in "${domains[@]}"
do
   latency=$(dig +tries=2 +stats $i | grep -i ‘query time‘ | cut -d " " -f 3)  
   echo "$i lookup took ${latency}ms"
done

Dumping this into a script allows running on an interval and triggering alerts when certain thresholds are exceeded or failures occur.

DIG output contains a bounty of data like this that can be easily parsed for automation purposes.

Conclusion

Dig represents a key diagnostic capability in the DNS troubleshooting toolbox for sysadmins. Leveraging its powerful flexible nature allows inspecting DNS infrastructure to uncover discrepancies and bottlenecks.

We have really only scratched the surface of dig‘s extensive functionality in this guide focused on fundamentals. The man pages and online docs contain a wealth of advanced features around trace analysis, DNSSEC, latency measurement and more.

Mastering dig both interactively and programatically will give any Linux admin confidence to weather tricky obscure DNS outages and issues. Start querying and unpacking output now during boring times to build pattern recognition experience that pays dividends during incidents.

What tips do you have for creative dig usage? Share your tricks below!

Similar Posts