systemd resolved 127.0.0.53 stub is broken by default #10298
Description
systemd version the issue has been seen with
10.3
ii systemd 237-3ubuntu10.3 amd64 system and service manager
$ uname -a
Linux (snip) 4.15.0-36-generic #39-Ubuntu SMP Mon Sep 24 16:19:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Used distribution
Ubuntu 18.04 server
Expected behaviour you didn't see
install Ubuntu 18.04, ping/nslookup a local hostname (which is a static DHCP entry on tomato router firmware), and it should resolve to local IPv4 address
Unexpected behaviour you saw
but it does not resolve on newer systems that default to systemd-resolve :(
BASIC TEST:
$ systemd-resolve fermmy-git
fermmy-git: resolve call failed: No appropriate name servers or networks for name found
Steps to reproduce the problem
0. baseline, Ubuntu 18.04 fresh install, greenfield; I am not interested in this stupid "netplan" crap for configuring static IP addresses ... so ...
- revert system to ifdown (/etc/network/interfaces) w/ static IP for server
$ cat /etc/network/interfaces
(snip)
auto ens3
iface ens3 inet static
address 1.0.0.51
netmask 255.255.255.0
broadcast 1.0.0.255
gateway 1.0.0.1
dns-nameservers 1.0.0.1
- nslookup/ping local host (via router DNS/gateway) doesn't work
$ ping fermmy-git
ping: fermmy-git: Temporary failure in name resolution
$ nslookup fermmy-git
Server: 127.0.0.53
Address: 127.0.0.53#53
** server can't find fermmy-git: SERVFAIL
$ systemd-resolve fermmy-git
fermmy-git: resolve call failed: No appropriate name servers or networks for name found
resolvconf
$ cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
# 127.0.0.53 is the systemd-resolved stub resolver.
# run "systemd-resolve --status" to see details about the actual nameservers.
nameserver 127.0.0.53
clearly, resolveconf is setup to utilize the systemd-resolve stub service...
$ systemd-resolve --status
Global
DNSSEC NTA: 10.in-addr.arpa
16.172.in-addr.arpa
168.192.in-addr.arpa
17.172.in-addr.arpa
18.172.in-addr.arpa
19.172.in-addr.arpa
20.172.in-addr.arpa
21.172.in-addr.arpa
22.172.in-addr.arpa
23.172.in-addr.arpa
24.172.in-addr.arpa
25.172.in-addr.arpa
26.172.in-addr.arpa
27.172.in-addr.arpa
28.172.in-addr.arpa
29.172.in-addr.arpa
30.172.in-addr.arpa
31.172.in-addr.arpa
corp
d.f.ip6.arpa
home
internal
intranet
lan
local
private
test
Link 2 (ens3)
Current Scopes: DNS
LLMNR setting: yes
MulticastDNS setting: no
DNSSEC setting: no
DNSSEC supported: no
DNS Servers: 1.0.0.1
well it DOES have my router IP as the DNS server ... wtf
So then ...
** Workarounds **
$ cat /etc/resolv.conf
(snip)
nameserver 1.0.0.1 <--- ADD THIS
nameserver 127.0.0.53
Then tada:
$ ping fermmy-git
PING fermmy-git (1.0.0.54) 56(84) bytes of data.
64 bytes from fermmy-git (1.0.0.54): icmp_seq=1 ttl=64 time=0.873 ms
^C
--- fermmy-git ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.873/0.873/0.873/0.000 ms
$ nslookup fermmy-git
Server: 1.0.0.1
Address: 1.0.0.1#53
Name: fermmy-git
Address: 1.0.0.54
Other notes:
- at some point in my debug I apt installed resolveconf btw to try to get basic local name resolution working .. (not sure if/when this is required - but it wasn't working before that)
- my older/existing server systems (Ubuntu 16.04 that still have resolveconf by default w/ static IP and force local router nameserver work fine ... ); existing desktop systems on DHCP also work fine; TLDR not my router/network problem
Why doesn't the systemd-resolve stub service work?
(if we're going to switch to systemd services galore by default for Linux systems everywhere, can we make sure they work? so frustrating!)