As a full-stack developer, having a deep understanding of core infrastructure like DNS is critical for building and maintaining complex web applications. DNS outages can completely break an app‘s functionality and devastate user experience. In my role optimizing performance and reliability, I rely daily on DNS diagnostic tools to ensure systems are operating smoothly.
This comprehensive guide will explore essential DNS troubleshooting utilities through the lens of a hands-on full-stack engineer. I‘ll share insights from years of experience on the capabilities and appropriate situations to leverage each tool based on real-world troubleshooting scenarios. My goal is to provide the perspective of an expert developer well-versed in utilizing these DNS tools to inspect records, trace issues, and inform solutions.
Nslookup – Quick Interactive Lookups
Nslookup is one of the simplest tools for interrogating DNS, making it fast and easy to check basic record mappings. With full-stack development, quick checks help validate assumptions during builds and narrow the scope during incident response. Though nslookup lacks some advanced functions, its interactive mode helpsdsp quickly isolate or confirm DNS issues.
For example, while developing a NodeJS application that connects to a third-party payments API, I suspected cached DNS entries were resulting in intermittent connection failures. A quick nslookup helped inspect current record mappings from my local dev environment against the production DNS servers:
> server 8.8.8.8
Default server: 8.8.8.8
Address: 8.8.8.8#53
> paymentsapi.com
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
Name: paymentsapi.com
Address: 93.123.40.201Comparing these results against our internal DNS showed significantly different IP addresses, confirming stale DNS entries in my workspace. An ipconfig /flushdns cleared the outdated records. By using nslookup, I could rapidly inspect mappings and prove the cached record theory matching the symptoms we saw in the app.
In terms of capabilities, key things I use nslookup for include:
- Interactive queries with ability to switch between DNS servers for comparison
- Confirm A, AAAA, CNAME and other simple record mappings
- Ease-of-use for rapid checks – easily fired off against unfamiliar domains
However, for more involved analysis nslookup lacks certain features:
- No detailed metadata in query responses
- Limited RR type support – no MX, TXT etc
- No DNSSEC validation capability
- No underlying protocol inspection
So while an indispensable quick-check tool, true in-depth analysis requires leveling up to dig.
Dig – Advanced Troubleshooting and Analysis
Dig provides extensive control and verbose outputs perfect for advanced DNS analysis by full-stack developers. During launches, I rely on dig‘s detail-rich results to confirm proper configuration across wide array of DNS record types necessary for complex web apps. Dig‘s flexibility is also invaluable in debugging obscure issues that require correlation across multiple DNS queries.
Recently I used dig while investigating launch issues around SMTP delivery failures for transactional emails. Our app servers connected to load-balanced SMTP relays which should have routed perfectly to available mail gateways. Oddly though, servers kept reporting connection timeouts. Dig provided the missing insight:
$ dig mx companydomain.com @8.8.8.8
; <<>> DiG 9.16.1-Ubuntu <<>> mx companydomain.com @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64284
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 6
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4000
;; QUESTION SECTION:
;companydomain.com. IN MX
;; ANSWER SECTION:
companydomain.com. 86400 IN MX 10 mx1.companydomain.com.
companydomain.com. 86400 IN MX 10 mx2.companydomain.com.
companydomain.com. 86400 IN MX 20 mx3.companydomain.com.
companydomain.com. 86400 IN MX 20 mx4.companydomain.com.
companydomain.com. 86400 IN MX 30 mxbackup.companydomain.com.
;; ADDITIONAL SECTION:
mx1.companydomain.com. 86400 IN AAAA 2601::1d
mx2.companydomain.com. 86400 IN A 104.28.10.11
mx2.companydomain.com. 86400 IN AAAA 2601::22
mx3.companydomain.com. 86400 IN A 104.28.27.52
mx4.companydomain.com. 86400 IN A 208.110.125.92
mx4.companydomain.com. 86400 IN AAAA 2601::42
;; Query time: 532 msec
;; WHEN: Sun Feb 12 10:37:41 EST 2023
;; MSG SIZE rcvd: 307Here dig exposed that while our primary MX records resolved IPv6 addresses, the secondary and tertiaries only had IPv4 entries. Our SMTP relays could route to the primary targets but failed over to backups due to Happy Eyeballs priorities favoring AAAA records. Reconfiguring the IPv6 entries allowed failover and restored email deliverability.
This example highlights just some of dig‘s key advantages:
- Fully-customizable queries to specify record type, DNS class, name servers, etc
- Verbose metadata in responses like refresh times, TTLs, and DNSSEC info
- Output easy to pipeline into other tools for further processing & analysis
- Lightweight and fast enough for automation in scripts checking groups of domains
For advanced DNS interrogation or working strange issues like this one, dig is an invaluable tool in my full-stack troubleshooting arsenal.
Host – Simple But Versatile DNS Lookup Tool
Host is my go-to for basic DNS checks due its unique ability to perform both forward (A/AAAA) and reverse (PTR) lookups. Its output is simple – just the domain and IP or vice versa. This makes host great for quick confirmation tests without unneeded data.
I often use host to check both directions of a DNS mapping when reviewing PRs altering our zone records. No need to cross reference nslookup or dig outputs – host lets me validate a change in a single step:
$ host www.companydomain.com
www.companydomain.com has address 104.28.10.11
$ host 104.28.10.11
11.10.28.104.in-addr.arpa domain name pointer lb1.companydomain.comHere I can rapidly confirm the hostname maps to the intended IP, and the reverse pointer is set correctly to our load balancer alias. This two way check gives me confidence in commiting these infrastructure changes.
Another area host assists with is service discovery. When integrating with new SaaS platforms, I leverage host to check how the provider has configured their side:
$ host api.newproduct.com
api.newproduct.com has address 172.67.19.106
api.newproduct.com mail is handled by 10 mx1.newproduct.com.
api.newproduct.com mail is handled by 20 mx2.newproduct.com.Now I can validate they have A records, MX records are present should we need to whitelist for service communications, and I can pass these details on to our security and networking teams.
For simplicity, availability by default on Linux, and bidirectional lookups, host remains a vital DNS tool for any full-stack developer‘s toolkit.
Traffic Analysis with Wireshark
Network traffic analysis with Wireshark provides visibility DNS troubleshooting can’t get from traditional lookup tools. By capturing packets sent between DNS clients and servers, Wireshark grants unique insight into query frequency, performance, protocol errors, and more.
Recently Wireshark was instrumental in identifying an anomaly where API response times for one of our NodeJS services had increased 30-40%. Dig and host reported DNS was functioning perfectly. Yet correlating traffic captures from the affected subnet showed odd ties to DNS behavior:
1821 1.962838 172.58.10.123 → 9.9.9.9 DNS 87 Standard query 0x1d65 A api.mycoolservice.com OPT
1822 1.963177 9.9.9.9 → 172.58.10.123 DNS 119 Standard query response 0x1d65 A 23.23.242.175 A 23.23.242.176 A 98.124
18.187 A 204.246.172.100 OPT
1823 2.261405 172.58.10.115 → 9.9.9.9 DNS 87 Standard query 0xbe66 A api.mycoolservice.com OPT
1824 2.272424 9.9.9.9 → 172.58.10.115 DNS 119 Standard query response 0xbe66 A 23.23.242.175 A 23.23.242.176 A 98.124
.18.187 A 204.246.172.100 OPTHere the timing between repeated queries was much greater than expected – nearly 300ms latency before the NodeJS service would retry DNS after receiving results. Since the API domain resolved to four IP addresses, extra processing was required to iterate through the list, resulting in a full added RTT fetching A records. Working with our CDN team, reducing the resolved entries to our two closest edge nodes restored expected behavior.
This again showcases the unique visibility analyzing raw DNS traffic provides. By inspecting DNS conversations rather than just entries, Wireshark enabled identifying the root cause invisible to other tools. Introducing traffic analysis greatly expands troubleshooting perspective.
DNSPerf – Critical for Quantifying Impact of Changes
DNS performance is a key consideration during continuous delivery as new features and fixes flow through environments. DNSPerf provides an objective way to baseline metrics like lookup latency, cache performance, query volumes, and DNSSEC overhead both before and after infrastructure or application changes.
Recently my team added compliance headers to API responses served through our NGINX proxies. This transparently added the additional hop without requiring application logic changes. But we wanted to quantify the impact on DNS lookup time from passing through the extra proxy layer:
$ dnsperf -s 9.9.9.9 -d api.mycoolapp.com
sending bursts of 10 queries
response times: min = 32 ms, max = 98 ms, avg = 45 ms
sent=100000, rcvd=98998, loss=1.00%
$ dnsperf -s 9.9.9.9 -d api.mycoolapp.com
sending bursts of 10 queries
response times: min = 62 ms, max = 152 ms, avg = 86 ms
sent=100000, rcvd=99023, loss=0.97%After the change, average lookup time had clearly doubled. By providing measurable KPI around DNS speed, DNSPerf enabled quantitatively confirming the performance impact of the architectural adjustment.
DNSPerf has become an invaluable tool as we scale our platform while balancing complexity, security, and speed. Easy automation via the CLI guides decision tradeoffs using objective metrics derived from DNSPerf testing before and after proposed alterations.
Make DNS Visibility a Priority
As a full-stack developer, the complexity we orchestrate mandates having clear visibility into foundational components like DNS. The tools explored in this guide help establish that comprehensive insight from multiple vantage points – quickly querying records, analyzing protocol transactions, inspecting traffic flows, and benchmarking performance attributes.
Mastering utilities like nslookup, dig, host, Wireshark and DNSPerf has given me the capability to not only architect systems, but also optimize them. With the power to deeply understand DNS behavior and pivot when needed, I can build and run higher quality, more resilient applications. I encourage all engineers expand their visibility through adept use of these DNS troubleshooting tools.
