Networking · lesson 4 of 8
Beginner
4Intermediate
4Also part of Networking
11- + Linux Networking: Routes, Interfaces, and Bind Addresses from Linux
- + Bash Networking from Bash
- + Docker Networking from Docker
- + FastAPI HTTP Methods and Status Codes from FastAPI
- + FastAPI Headers and Cookies from FastAPI
- + Cloud Networking Basics: What's the Same Everywhere from Multi-Cloud
- + VPC and Networking Basics from AWS
- + GCP VPC Networking Basics from Google Cloud
- + Azure Virtual Network Basics from Azure
- + The TLS Handshake, Run Locally with openssl s_server from Cryptography
- + Load Balancing: Round Robin vs Least Connections from System Design
DNS: From a Name to an Address, and the Ways It Fails
Query a local dnsmasq with dig: an A record and its TTL, NXDOMAIN vs SERVFAIL vs no answer, what Python sees for each, and /etc/hosts beating DNS.
By SkillByExample · Updated
Every connection in the earlier lessons used a name like server, and something turned it into an
address before the first packet. That something is DNS. This lesson runs a small DNS server,
dnsmasq, in Docker, queries it with dig, and breaks resolution in three different ways. Each failure
looks different in dig and in Python, and telling them apart saves hours. Tested with Docker Engine
29.8.2, Alpine 3.22.6, dnsmasq 2.91, dig from BIND 9.20.29 and Python 3.12.15 (python:3.12-slim).
A resolver for your own zone
One image holds both the server and the client tools:
FROM alpine:3.22
RUN apk add --no-cache dnsmasq bind-toolsThe config answers for a private zone, lab.test, from its own records, and forwards one other domain,
broken.test, to an upstream server at 172.30.0.99.
# dnsmasq.conf
no-resolv
no-hosts
log-queries
log-facility=-
local=/lab.test/
local-ttl=300
host-record=api.lab.test,172.30.0.10
host-record=db.lab.test,172.30.0.11
server=/broken.test/172.30.0.99no-resolv stops dnsmasq from forwarding anything else to your real resolver, and the network is
--internal, so no query can leave the lab. The .test top-level domain is reserved for exactly this
kind of testing.
docker build -t dns-lab .
docker network create --internal --subnet 172.30.0.0/24 dns-lab
docker run -d --name dns --network dns-lab --ip 172.30.0.53 -v "$PWD/dnsmasq.conf":/etc/dnsmasq.conf:ro dns-lab dnsmasq -kA successful lookup
$ docker run --rm --network dns-lab dns-lab dig @172.30.0.53 api.lab.test
; <<>> DiG 9.20.29 <<>> @172.30.0.53 api.lab.test
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56099
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;api.lab.test. IN A
;; ANSWER SECTION:
api.lab.test. 300 IN A 172.30.0.10
;; Query time: 0 msec
;; SERVER: 172.30.0.53#53(172.30.0.53) (UDP)
;; WHEN: Tue Oct 13 19:09:06 UTC 2026
;; MSG SIZE rcvd: 57Three parts matter. status: NOERROR in the header is the outcome. The answer line reads “the name
api.lab.test has an Internet (IN) address record (A) of 172.30.0.10, and you may cache it for
300 seconds”. That number is the TTL. The aa flag means authoritative answer: this server owns the
zone and isn’t repeating something it heard elsewhere. @172.30.0.53 told dig which server to ask, and
the SERVER line confirms it went over UDP port 53.
Failure 1: NXDOMAIN, the name doesn’t exist
$ docker run --rm --network dns-lab dns-lab dig @172.30.0.53 nope.lab.test
; <<>> DiG 9.20.29 <<>> @172.30.0.53 nope.lab.test
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 49872
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;nope.lab.test. IN A
;; Query time: 0 msec
;; SERVER: 172.30.0.53#53(172.30.0.53) (UDP)
;; WHEN: Tue Oct 13 19:09:07 UTC 2026
;; MSG SIZE rcvd: 42The server answered, instantly, and the answer is “no such name”. NXDOMAIN is a fact about the zone, and resolvers cache it like any other answer. Retrying won’t help. The fix is a record or a spelling correction.
Failure 2: no answer at all
broken.test is forwarded to 172.30.0.99, and nothing runs there yet:
$ docker run --rm --network dns-lab dns-lab dig @172.30.0.53 www.broken.test
;; communications error to 172.30.0.53#53: timed out
;; communications error to 172.30.0.53#53: timed out
;; communications error to 172.30.0.53#53: timed out
; <<>> DiG 9.20.29 <<>> @172.30.0.53 www.broken.test
; (1 server found)
;; global options: +cmd
;; no servers could be reached
$ docker logs dns 2>&1 | tail -3
dnsmasq[1]: forwarded www.broken.test to 172.30.0.99
dnsmasq[1]: query[A] www.broken.test from 172.30.0.2
dnsmasq[1]: forwarded www.broken.test to 172.30.0.99dig blames 172.30.0.53, which is the server it asked, and it got nothing back after three tries of
five seconds each. The server log shows the real story: dnsmasq was alive and forwarding, but waiting on
an upstream that never answered, so it had nothing to send back. “No servers could be reached” means
no DNS response arrived, not that the name is wrong. It’s the DNS version of the timeout from the
sockets lesson.
Failure 3: SERVFAIL, an answer that says “I failed”
Now put a server at 172.30.0.99 that is up but broken. It replies to every query with response code 2,
SERVFAIL, which is what a real upstream sends when its own backend or DNSSEC validation fails.
# broken_upstream.py
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.bind(("0.0.0.0", 53))
while True:
query, addr = s.recvfrom(512)
# same ID and question, flags = response, RCODE 2 (SERVFAIL)
header = query[:2] + b"\x81\x82" + query[4:6] + b"\x00\x00" * 3
end = query.index(b"\x00", 12) + 5 # end of the question section
s.sendto(header + query[12:end], addr)$ docker run -d --name upstream --network dns-lab --ip 172.30.0.99 -v "$PWD":/work -w /work python:3.12-slim python broken_upstream.py
$ docker run --rm --network dns-lab dns-lab dig @172.30.0.53 www.broken.test
; <<>> DiG 9.20.29 <<>> @172.30.0.53 www.broken.test
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 61628
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;www.broken.test. IN A
;; Query time: 0 msec
;; SERVER: 172.30.0.53#53(172.30.0.53) (UDP)
;; WHEN: Tue Oct 13 19:09:27 UTC 2026
;; MSG SIZE rcvd: 33
$ docker logs dns 2>&1 | tail -3
dnsmasq[1]: query[A] www.broken.test from 172.30.0.2
dnsmasq[1]: forwarded www.broken.test to 172.30.0.99
dnsmasq[1]: reply error is SERVFAILThe whole DNS message format is visible in those few lines. A response reuses the query’s 2-byte ID so
the client can match them up, sets flag bits that mark it as a response, and puts the outcome in the
low 4 bits of the flags (0x82 ends in 2). SERVFAIL means “I couldn’t get an answer”, not “there’s no
such name”. The name may well exist, so you look at the servers in the chain, not at the record.
What your application sees
Programs don’t run dig. They call the system resolver, which in Python is getaddrinfo:
# resolve.py
import socket, sys
for name in sys.argv[1:]:
try:
infos = socket.getaddrinfo(name, 80, type=socket.SOCK_STREAM)
print(f"{name:<16} {infos[0][4][0]}")
except socket.gaierror as e:
print(f"{name:<16} gaierror {e.errno}: {e.strerror}")--dns makes Docker’s embedded resolver in the container forward to the lab’s dnsmasq:
$ docker run --rm --network dns-lab --dns 172.30.0.53 -v "$PWD":/work -w /work python:3.12-slim python resolve.py api.lab.test nope.lab.test www.broken.test
api.lab.test 172.30.0.10
nope.lab.test gaierror -2: Name or service not known
www.broken.test gaierror -3: Temporary failure in name resolutionThe two failures arrive as two different error numbers. -2 is NXDOMAIN, which is permanent. -3 is
SERVFAIL, which is “temporary”, so a retry might succeed. A log line that only says “could not resolve
host” has thrown that difference away. Log the error number.
/etc/hosts wins, but only for some tools
--add-host writes an entry into the container’s /etc/hosts, pointing api.lab.test somewhere else:
$ docker run --rm --network dns-lab --dns 172.30.0.53 --add-host api.lab.test:10.9.9.9 dns-lab sh -c "grep lab.test /etc/hosts; getent hosts api.lab.test; dig +short api.lab.test"
10.9.9.9 api.lab.test
10.9.9.9 api.lab.test api.lab.test
172.30.0.10getent hosts asks the system resolver, the same path curl, Python and almost every program take. That
path reads /etc/hosts before DNS, so it returns 10.9.9.9. dig only speaks DNS, so it never reads
the file and returns the real record. When “dig says one thing and my app does another”, look in
/etc/hosts. A stale entry left over from testing is a classic cause. Container-name DNS on Docker
networks is covered in Docker networking. Clean up with
docker rm -f dns upstream and docker network rm dns-lab.
Practice
- Add an alias. Give
api.lab.testa second name,www.lab.test, with a CNAME record, and query it with+noall +answerto see only the answer section.
Solution
dnsmasq takes extra settings on the command line as well as from the file:
$ docker rm -f dns
$ docker run -d --name dns --network dns-lab --ip 172.30.0.53 -v "$PWD/dnsmasq.conf":/etc/dnsmasq.conf:ro dns-lab dnsmasq -k --cname=www.lab.test,api.lab.test
$ docker run --rm --network dns-lab dns-lab dig @172.30.0.53 +noall +answer www.lab.test
www.lab.test. 300 IN CNAME api.lab.test.
api.lab.test. 300 IN A 172.30.0.10A CNAME says “this name is really that name”, and the server helpfully includes the A record for the target in the same answer. Two hops, one round trip.
- Go backwards. Look up the name for
172.30.0.10withdig -x. What name does the query actually ask for?
Solution
$ docker run --rm --network dns-lab dns-lab dig @172.30.0.53 +noall +answer -x 172.30.0.10
10.0.30.172.in-addr.arpa. 300 IN PTR api.lab.test.A reverse lookup asks for a PTR record under in-addr.arpa, with the address’s octets reversed, so the
most general part comes last, as in a normal name. host-record created the PTR record automatically.
- Watch a TTL count down. Start a second dnsmasq at
172.30.0.54that caches and forwards everything to172.30.0.53. Ask it forapi.lab.testtwice, about ten seconds apart, then ask the authoritative server.
Solution
$ docker run -d --name cache --network dns-lab --ip 172.30.0.54 dns-lab dnsmasq -k --no-resolv --server=172.30.0.53
$ docker run --rm --network dns-lab dns-lab dig @172.30.0.54 +noall +answer api.lab.test
api.lab.test. 300 IN A 172.30.0.10
$ docker run --rm --network dns-lab dns-lab dig @172.30.0.54 +noall +answer api.lab.test
api.lab.test. 289 IN A 172.30.0.10
$ docker run --rm --network dns-lab dns-lab dig @172.30.0.53 +noall +answer api.lab.test
api.lab.test. 300 IN A 172.30.0.10The authoritative server always says 300. The cache says how much of the 300 is left. If you changed the record now, clients of this cache would keep getting the old address for 289 more seconds. That’s why you lower a TTL before a migration, not during it.
- Name each failure. For each case, say what you’d check first: dig prints
status: NXDOMAIN; dig printsstatus: SERVFAIL; dig printsno servers could be reached; dig is right butgetentisn’t.
Solution
| Symptom | Meaning | Check first |
|---|---|---|
NXDOMAIN | the zone says the name doesn’t exist | spelling, then whether the record was created |
SERVFAIL | the resolver got no usable answer | the upstream servers for that domain, then DNSSEC |
no servers could be reached | no DNS response arrived | that the resolver is up and reachable on port 53 |
getent disagrees with dig | something before DNS answered | /etc/hosts |