Why did deleting the default route break internet access but not local-subnet access?

The routing table has two kinds of entries: subnet routes for destinations directly reachable over an interface, and the default route for everything else. Deleting only the default route leaves the subnet route intact, so a host on the same /16 still resolves via ARP and responds — but any destination outside that subnet has no matching route left, and the kernel refuses to send the packet at all.

What does 'Network is unreachable' actually mean, versus a timeout?

'Network is unreachable' is an immediate, local decision — the kernel checked its own routing table, found no matching route, and refused to even hand the packet to a network interface. A timeout means the packet was sent but no reply came back, which points to a problem somewhere downstream (firewall, dead host, blocked port) rather than in your own routing table.

Why did the same nc listener work from inside the container but get refused from another container?

A socket bound to 127.0.0.1 only accepts connections whose destination is the loopback interface itself — which is only true for processes inside that same container, since each container has its own private loopback. A process on another container reaches it via the real network interface (eth0), and the kernel refuses that connection before it ever reaches the listening application, because no socket is bound to that interface.

How do I tell what address a listening service is actually bound to?

ss -tlnp shows a Local Address:Port column for every listening socket. 127.0.0.1:9000 means loopback-only; 0.0.0.0:9000 means it accepts connections on every interface. This is the single fastest way to confirm whether a 'service is up but nobody can reach it' report is a bind-address problem before checking firewalls or DNS.

Is binding to 0.0.0.0 always the right fix?

No — it's the right fix when the service is meant to be reached from other hosts or containers. A database or internal cache that should only ever be reached by processes on the same machine is more safely left bound to 127.0.0.1, so this is a real security decision, not just a connectivity toggle.

Linux Networking: Routes, Interfaces, and Bind Addresses

Two real networking failures at the OS layer: a deleted default route that leaves the local subnet reachable but the internet not, and a socket bound to 127.0.0.1 that refuses every connection from outside the container.

By SkillByExample · Updated Sep 14, 2026

linux networking ip routing sockets

Two failures that look identical from a client’s point of view — “can’t reach it” — come from completely different layers, and the fix for one does nothing for the other.

Interfaces and routes: what a container actually has

docker run -d --name net-lesson --cap-add=NET_ADMIN ubuntu:22.04 sleep infinity
docker exec net-lesson ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0@if89: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 72:48:d5:8a:1a:dd brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.17.0.9/16 brd 172.17.255.255 scope global eth0
       valid_lft forever preferred_lft forever

Two interfaces: lo, the loopback that only talks to processes on this same container, and eth0, the real interface with the address 172.17.0.9 that other containers see. Having an address on an interface doesn’t say anything about where packets to other networks go — that’s a separate table:

docker exec net-lesson ip route

default via 172.17.0.1 dev eth0
172.17.0.0/16 dev eth0 proto kernel scope link src 172.17.0.9

Two rules: anything in 172.17.0.0/16 goes straight out eth0, and everything else (default) goes via the gateway at 172.17.0.1.

The bug: deleting the default route

docker exec net-lesson ip route del default
docker exec net-lesson ip route

172.17.0.0/16 dev eth0 proto kernel scope link src 172.17.0.9

The subnet route survived; only default is gone.

docker exec net-lesson ping -c 2 -W 2 8.8.8.8

ping: connect: Network is unreachable

Not a timeout — the kernel never sent a packet. It checked the routing table, found nothing that matches 8.8.8.8, and refused immediately. Compare a destination still inside the subnet:

docker exec net-lesson ping -c 2 -W 2 172.17.0.1

PING 172.17.0.1 (172.17.0.1) 56(84) bytes of data.
64 bytes from 172.17.0.1: icmp_seq=1 ttl=64 time=3.26 ms
64 bytes from 172.17.0.1: icmp_seq=2 ttl=64 time=0.093 ms

--- 172.17.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.093/1.674/3.256/1.581 ms

Same interface, same container, still works — because the subnet route never left. This is the actual value of separating the two checks: a working subnet-local ping with a failing internet ping points straight at the default route, not the interface itself.

The fix: restore the route

docker exec net-lesson ip route add default via 172.17.0.1 dev eth0
docker exec net-lesson ping -c 2 -W 2 8.8.8.8

PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=63 time=18.0 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=63 time=19.3 ms

--- 8.8.8.8 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 17.981/18.645/19.310/0.664 ms

A second failure: the same address doesn’t mean the same route

docker exec -d net-lesson bash -c "nc -l 127.0.0.1 9000"
docker exec net-lesson ss -tlnp | grep 9000

LISTEN 0      1          127.0.0.1:9000      0.0.0.0:*    users:(("nc",pid=434,fd=3))

Local Address is 127.0.0.1 — this socket only accepts connections addressed to loopback. From inside the same container, that’s every connection:

docker exec net-lesson bash -c "echo hi | nc -w 2 127.0.0.1 9000 && echo LOCAL_OK"

LOCAL_OK

From a second container on a shared network, the destination is eth0, not loopback:

docker network create net-lesson-net
docker network connect net-lesson-net net-lesson
docker run --rm --network net-lesson-net ubuntu:22.04 bash -c \
  "apt-get update -qq && apt-get install -y -qq netcat-openbsd >/dev/null 2>&1; \
   echo hi | nc -w 2 -v net-lesson 9000"

nc: connect to net-lesson (172.19.0.2) port 9000 (tcp) failed: Connection refused

The route to the container is fine — this is a different rejection than the missing-route case, refused instantly by the kernel because no socket anywhere is listening on eth0’s address for that port, only on loopback’s.

The fix: bind to every interface

docker exec net-lesson pkill -f "nc -l"
docker exec -d net-lesson bash -c "nc -l 0.0.0.0 9000"
docker exec net-lesson ss -tlnp | grep 9000

LISTEN 0      1            0.0.0.0:9000       0.0.0.0:*    users:(("nc",pid=460,fd=3))

docker run --rm --network net-lesson-net ubuntu:22.04 bash -c \
  "apt-get update -qq && apt-get install -y -qq netcat-openbsd >/dev/null 2>&1; \
   echo hi | nc -w 2 -v net-lesson 9000; echo EXIT_CODE=\$?"

Connection to net-lesson (172.19.0.2) 9000 port [tcp/*] succeeded!
EXIT_CODE=0

Same container, same port, same client command — the only thing that changed is which interfaces the listening socket accepts connections on.

Practice

1. Confirm ip route get shows exactly which rule a given destination would use.

docker exec net-lesson ip route get 8.8.8.8

8.8.8.8 via 172.17.0.1 dev eth0 src 172.17.0.9 uid 0
    cache

via 172.17.0.1 dev eth0 is the default route being applied — this is the fastest way to confirm which routing table entry would actually handle a specific address, without having to read the whole table and reason about longest-prefix match yourself.

2. Distinguish "no route" from "route exists but nothing answers" using a made-up subnet.

docker exec net-lesson ping -c 1 -W 2 10.99.99.99

PING 10.99.99.99 (10.99.99.99) 56(84) bytes of data.

--- 10.99.99.99 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

No “Network is unreachable” this time — 10.99.99.99 matches the default route (since nothing more specific exists for it), so the kernel does send the packet; it just never gets a reply. 100% packet loss after a real wait is a downstream failure; an instant “Network is unreachable” is a local routing-table failure. They look similar but mean very different things to fix.

3. Check both listening sockets side by side after fixing the bind address.

docker exec net-lesson ss -tln

State  Recv-Q Send-Q Local Address:Port  Peer Address:Port
LISTEN 0      4096      127.0.0.11:33563      0.0.0.0:*
LISTEN 0      1            0.0.0.0:9000       0.0.0.0:*

The 127.0.0.11 line is Docker’s own embedded DNS resolver for this container, bound to loopback only — a real example of a service that’s deliberately loopback-bound, right next to 0.0.0.0:9000 which is deliberately not. The Local Address column is what to check first any time “the service is up but nothing can reach it” comes up.

Next: package management — installing, upgrading, and untangling a broken dependency with apt and dpkg.