Host Firewalls with nftables: DROP Times Out, REJECT Refuses

A firewall decides what happens to a packet before any program sees it. The sockets lesson showed two kinds of failure, an instant refusal and a slow timeout. This lesson produces both on purpose, from the same running server, by changing one word in a firewall rule. It then lets exactly one client back in. You configure only the firewall of your own lab container, inside its own network namespace. Your machine’s firewall is never touched. Tested with Docker Engine 29.8.2, Python 3.12.15 (python:3.12-slim) and nftables 1.1.3.

The lab

The server and clients are the echo server and client.py from the sockets lesson. The client prints the error type and how long the attempt took, with a two-second timeout.

# client.py
import socket, sys, time

host, port = sys.argv[1], int(sys.argv[2])
timeout = float(sys.argv[3]) if len(sys.argv) > 3 else 2
start = time.monotonic()
try:
    with socket.create_connection((host, port), timeout=timeout) as s:
        print("local ", s.getsockname())
        print("remote", s.getpeername())
        s.sendall(b"hello")
        print("reply ", s.recv(1024))
except OSError as e:
    print(type(e).__name__, "-", e)
print(f"took {time.monotonic() - start:.2f}s")

# echo_server.py
import socket

srv = socket.create_server(("0.0.0.0", 9000))
print("listening on", srv.getsockname(), flush=True)
while True:
    conn, addr = srv.accept()
    with conn:
        data = conn.recv(1024)
        print("from", addr, "got", data, flush=True)
        conn.sendall(data.upper())

The image adds the nft tool:

FROM python:3.12-slim
RUN apt-get update && apt-get install -y --no-install-recommends nftables \
    && rm -rf /var/lib/apt/lists/*
WORKDIR /work

The first rule file drops every packet sent to TCP port 9000:

# drop.nft
table inet lab
delete table inet lab
table inet lab {
    chain input {
        type filter hook input priority 0; policy accept;
        tcp dport 9000 counter drop
    }
}

The inet family covers IPv4 and IPv6 at once. The chain hooks into input, the path that packets addressed to this host take. policy accept means anything no rule matches gets through. The first two lines are a reload idiom: create the table if it’s missing, delete it, then define it fresh. You can apply the file any number of times. (nft also has a one-line destroy table, but the kernel under Docker Desktop rejected it with "destroy" command is not supported.)

Without the capability, nothing happens

docker build -t fw-lab .
docker network create --internal --subnet 172.31.0.0/24 fw-lab

$ docker run --rm --network fw-lab -v "$PWD":/work fw-lab nft -f drop.nft
netlink: Error: cache initialization failed: Operation not permitted

A container can’t even read the firewall without CAP_NET_ADMIN. Grant it to the server container only, and never use --privileged. The rules you write then live in that container’s network namespace. They filter its traffic, and nobody else’s.

docker run -d --name server --network fw-lab --ip 172.31.0.10 --cap-add NET_ADMIN -v "$PWD":/work fw-lab python echo_server.py

The ruleset you start with isn’t empty

$ docker exec server sh -c "nft list ruleset 2>/dev/null"
table ip nat {
	chain DOCKER_OUTPUT {
		ip daddr 127.0.0.11 tcp dport 53 counter packets 0 bytes 0 xt target "DNAT"
		ip daddr 127.0.0.11 udp dport 53 counter packets 0 bytes 0 xt target "DNAT"
	}

	chain OUTPUT {
		type nat hook output priority dstnat; policy accept;
		ip daddr 127.0.0.11 counter packets 0 bytes 0 jump DOCKER_OUTPUT
	}

	chain DOCKER_POSTROUTING {
		ip saddr 127.0.0.11 tcp sport 33131 counter packets 0 bytes 0 xt target "SNAT"
		ip saddr 127.0.0.11 udp sport 37354 counter packets 0 bytes 0 xt target "SNAT"
	}

	chain POSTROUTING {
		type nat hook postrouting priority srcnat; policy accept;
		ip daddr 127.0.0.11 counter packets 0 bytes 0 jump DOCKER_POSTROUTING
	}
}
$ docker exec server sh -c "nft list ruleset 2>&1 >/dev/null"
# Warning: table ip nat is managed by iptables-nft, do not touch!
# Warning: XT target DNAT not found
# Warning: XT target DNAT not found
# Warning: XT target SNAT not found
# Warning: XT target SNAT not found

Docker put this here. These NAT rules send the container’s DNS queries for 127.0.0.11 to Docker’s embedded resolver, the one that turns server into an address. They were written by iptables, which nowadays is a front end to the same nftables engine, so nft can list them but warns you off. This is why the rule files use their own table, inet lab, and not flush ruleset. Flushing everything would silently break name resolution inside the container.

Before any rule, a client at 172.31.0.30 gets its answer:

$ docker run --rm --network fw-lab --ip 172.31.0.30 -v "$PWD":/work fw-lab python client.py server 9000
local  ('172.31.0.30', 53486)
remote ('172.31.0.10', 9000)
reply  b'HELLO'
took 0.03s

DROP: the client times out

$ docker exec server nft -f drop.nft
$ docker exec server nft list table inet lab
table inet lab {
	chain input {
		type filter hook input priority filter; policy accept;
		tcp dport 9000 counter packets 0 bytes 0 drop
	}
}
$ docker run --rm --network fw-lab --ip 172.31.0.30 -v "$PWD":/work fw-lab python client.py server 9000
TimeoutError - timed out
took 2.03s
$ docker exec server nft list table inet lab
table inet lab {
	chain input {
		type filter hook input priority filter; policy accept;
		tcp dport 9000 counter packets 2 bytes 120 drop
	}
}

The server is running and listening, exactly as before, and the client can’t tell. It sent a SYN, heard nothing, sent it again a second later, and gave up when its two seconds ran out. The counter shows both attempts: 2 packets of 60 bytes, each one a SYN. From the outside, a dropped port looks the same as a host that doesn’t exist. That’s the point of drop, and it’s also why a timeout is the slowest failure to debug.

REJECT: the client is refused

One word changes:

# reject.nft
table inet lab
delete table inet lab
table inet lab {
    chain input {
        type filter hook input priority 0; policy accept;
        tcp dport 9000 counter reject with tcp reset
    }
}

$ docker exec server nft -f reject.nft
$ docker run --rm --network fw-lab --ip 172.31.0.30 -v "$PWD":/work fw-lab python client.py server 9000
ConnectionRefusedError - [Errno 111] Connection refused
took 0.03s
$ docker exec server nft list table inet lab
table inet lab {
	chain input {
		type filter hook input priority filter; policy accept;
		tcp dport 9000 counter packets 1 bytes 60 reject with tcp reset
	}
}

The firewall now answers on the server’s behalf with the same TCP reset a closed port sends, so the client fails in 30 ms. One packet, one answer, no retry. This is the catch with “refused means nothing is listening” from the sockets lesson: a reject rule produces exactly the same error while the service is running fine. Choose drop facing the internet, where an answer only helps someone mapping your network. Choose reject inside your own network, where fast and clear failures save on-call time.

An allow-list: one source gets in

# allow.nft
table inet lab
delete table inet lab
table inet lab {
    chain input {
        type filter hook input priority 0; policy accept;
        ip saddr 172.31.0.20 tcp dport 9000 counter accept
        tcp dport 9000 counter drop
    }
}

$ docker exec server nft -f allow.nft
$ docker run --rm --network fw-lab --ip 172.31.0.20 -v "$PWD":/work fw-lab python client.py server 9000
local  ('172.31.0.20', 41408)
remote ('172.31.0.10', 9000)
reply  b'HELLO'
took 0.02s
$ docker run --rm --network fw-lab --ip 172.31.0.30 -v "$PWD":/work fw-lab python client.py server 9000
TimeoutError - timed out
took 2.03s
$ docker exec server nft list table inet lab
table inet lab {
	chain input {
		type filter hook input priority filter; policy accept;
		ip saddr 172.31.0.20 tcp dport 9000 counter packets 5 bytes 273 accept
		tcp dport 9000 counter packets 2 bytes 120 drop
	}
}

Rules run top to bottom, and the first verdict wins. Packets from .20 hit the accept rule and never reach the drop. Everything else falls through to the drop. The counters prove it: the allowed client’s 5 packets (handshake, data, close) were accepted, and the other client’s 2 SYNs were dropped. The allow-list is an addition to the drop rule, which is the fail-closed shape: a new client is denied until someone lists it. Cloud security groups and NSGs apply the same idea at the network edge, as Networking basics everywhere shows. Clean up with docker rm -f server and docker network rm fw-lab.

Practice

1. Reject without a reset. Change reject with tcp reset to a plain reject, which answers with an ICMP “port unreachable” message instead. Does the client notice the difference?

# icmp.nft
table inet lab
delete table inet lab
table inet lab {
    chain input {
        type filter hook input priority 0; policy accept;
        tcp dport 9000 counter reject
    }
}

$ docker exec server nft -f icmp.nft
$ docker run --rm --network fw-lab --ip 172.31.0.30 -v "$PWD":/work fw-lab python client.py server 9000
ConnectionRefusedError - [Errno 111] Connection refused
took 0.02s

2. Allow a list, not a host. Replace the single address with a named set holding 172.31.0.20 and 172.31.0.21, then test from .21 and from .30.

# set.nft
table inet lab
delete table inet lab
table inet lab {
    set allowed {
        type ipv4_addr
        elements = { 172.31.0.20, 172.31.0.21 }
    }
    chain input {
        type filter hook input priority 0; policy accept;
        ip saddr @allowed tcp dport 9000 counter accept
        tcp dport 9000 counter drop
    }
}

$ docker exec server nft -f set.nft
$ docker run --rm --network fw-lab --ip 172.31.0.21 -v "$PWD":/work fw-lab python client.py server 9000
local  ('172.31.0.21', 56416)
remote ('172.31.0.10', 9000)
reply  b'HELLO'
took 0.02s
$ docker run --rm --network fw-lab --ip 172.31.0.30 -v "$PWD":/work fw-lab python client.py server 9000
TimeoutError - timed out
took 2.02s
$ docker exec server nft list set inet lab allowed
table inet lab {
	set allowed {
		type ipv4_addr
		elements = { 172.31.0.20, 172.31.0.21 }
	}
}

3. Find the ordering bug. Someone “tidied” the allow-list file and put the drop rule first. Predict what .20 sees, then confirm it with the counters.

# order.nft
table inet lab
delete table inet lab
table inet lab {
    chain input {
        type filter hook input priority 0; policy accept;
        tcp dport 9000 counter drop
        ip saddr 172.31.0.20 tcp dport 9000 counter accept
    }
}

$ docker exec server nft -f order.nft
$ docker run --rm --network fw-lab --ip 172.31.0.20 -v "$PWD":/work fw-lab python client.py server 9000
TimeoutError - timed out
took 2.02s
$ docker exec server nft list chain inet lab input
table inet lab {
	chain input {
		type filter hook input priority filter; policy accept;
		tcp dport 9000 counter packets 2 bytes 120 drop
		ip saddr 172.31.0.20 tcp dport 9000 counter packets 0 bytes 0 accept
	}
}

4. Undo it. Remove your table and show that the client gets in again and that Docker’s NAT table is still intact.

$ docker exec server nft delete table inet lab
$ docker run --rm --network fw-lab --ip 172.31.0.30 -v "$PWD":/work fw-lab python client.py server 9000
local  ('172.31.0.30', 35966)
remote ('172.31.0.10', 9000)
reply  b'HELLO'
took 0.03s
$ docker exec server sh -c "nft list ruleset 2>/dev/null"
table ip nat {
	chain DOCKER_OUTPUT {
		ip daddr 127.0.0.11 tcp dport 53 counter packets 0 bytes 0 xt target "DNAT"
		ip daddr 127.0.0.11 udp dport 53 counter packets 0 bytes 0 xt target "DNAT"
	}

	chain OUTPUT {
		type nat hook output priority dstnat; policy accept;
		ip daddr 127.0.0.11 counter packets 0 bytes 0 jump DOCKER_OUTPUT
	}

	chain DOCKER_POSTROUTING {
		ip saddr 127.0.0.11 tcp sport 33131 counter packets 0 bytes 0 xt target "SNAT"
		ip saddr 127.0.0.11 udp sport 37354 counter packets 0 bytes 0 xt target "SNAT"
	}

	chain POSTROUTING {
		type nat hook postrouting priority srcnat; policy accept;
		ip daddr 127.0.0.11 counter packets 0 bytes 0 jump DOCKER_POSTROUTING
	}
}