Ports and Sockets: Connection Refused vs Timed Out

An IP address gets a packet to a host. A port gets it to one program on that host. The pair is a socket address, and a TCP connection is two of them joined together. This lesson runs a small TCP echo server and a client in Docker, then breaks the connection on purpose in four different ways. Each way produces a different error after a different wait, and telling them apart is most of network debugging. Tested with Docker Engine 29.8.2 and Python 3.12.15 (python:3.12-slim). Everything runs on one internal Docker network that has no route out of the lab.

A server and a client

# echo_server.py
import socket

srv = socket.create_server(("0.0.0.0", 9000))
print("listening on", srv.getsockname(), flush=True)
while True:
    conn, addr = srv.accept()
    with conn:
        data = conn.recv(1024)
        print("from", addr, "got", data, flush=True)
        conn.sendall(data.upper())

The client takes a host, a port and an optional timeout, and reports what went wrong and how long that took.

# client.py
import socket, sys, time

host, port = sys.argv[1], int(sys.argv[2])
timeout = float(sys.argv[3]) if len(sys.argv) > 3 else 2
start = time.monotonic()
try:
    with socket.create_connection((host, port), timeout=timeout) as s:
        print("local ", s.getsockname())
        print("remote", s.getpeername())
        s.sendall(b"hello")
        print("reply ", s.recv(1024))
except OSError as e:
    print(type(e).__name__, "-", e)
print(f"took {time.monotonic() - start:.2f}s")

Create a lab network with a known subnet and start the server at a fixed address. --internal means the network has no gateway, so nothing in this lesson can leave your machine.

docker network create --internal --subnet 172.28.0.0/24 sock-lab
docker run -d --name server --network sock-lab --ip 172.28.0.10 -v "$PWD":/work -w /work python:3.12-slim python echo_server.py

$ docker run --rm --network sock-lab -v "$PWD":/work -w /work python:3.12-slim python client.py server 9000
local  ('172.28.0.2', 54920)
remote ('172.28.0.10', 9000)
reply  b'HELLO'
took 0.03s
$ docker logs server
listening on ('0.0.0.0', 9000)
from ('172.28.0.2', 54920) got b'hello'

The connection has two ends. The server’s end is the port you chose, 9000. The client’s end is 54920, an ephemeral port the kernel picked for this one connection. The server’s log shows the same pair from the other side. A second client run gets a different ephemeral port, which is how one server port holds thousands of connections at once: each one is a different four-part tuple of client IP, client port, server IP and server port. Docker’s embedded DNS turned the name server into 172.28.0.10.

Failure 1: Connection refused

Same host, a port with nothing behind it:

$ docker run --rm --network sock-lab -v "$PWD":/work -w /work python:3.12-slim python client.py server 9001
ConnectionRefusedError - [Errno 111] Connection refused
took 0.03s

Refused is an answer. The packet crossed the network, the server’s kernel saw no socket listening on 9001, and it replied with a TCP reset straight away. So a refusal in 30 ms tells you a lot: the address is right, the route works, and the problem is the service. Either it isn’t running, it’s on another port, or it’s bound to 127.0.0.1 only. The next lesson shows that reset as a packet.

Failure 2: Timed out

Now an address on the subnet where no container exists:

$ docker run --rm --network sock-lab -v "$PWD":/work -w /work python:3.12-slim python client.py 172.28.0.99 9000
TimeoutError - timed out
took 2.03s

A timeout is silence. Nothing replied, so the client waited for its full two seconds. Silence has more causes than a refusal: no such host, a firewall that drops packets, a broken route, or a server too overloaded to answer. That’s why “it times out” needs more investigation than “it’s refused”. The firewall lesson produces both on purpose from the same server.

Failure 3: wait longer, get a different answer

Give the same dead address ten seconds:

$ docker run --rm --network sock-lab -v "$PWD":/work -w /work python:3.12-slim python client.py 172.28.0.99 9000 10
OSError - [Errno 113] No route to host
took 3.17s

The error changed because something did eventually answer: the client’s own kernel. 172.28.0.99 is on the local subnet, so before sending a TCP packet the kernel asks the subnet “who has 172.28.0.99?” using ARP. After three unanswered ARP requests, about one second apart, it gives up and fails the connect with EHOSTUNREACH. With a two-second timeout your code never saw that, because the timeout fired first. The error you see depends on which clock runs out first. The tcpdump lesson captures those ARP requests with timestamps.

Failure 4: no route at all

An address outside the subnet, on a network with no gateway:

$ docker run --rm --network sock-lab -v "$PWD":/work -w /work python:3.12-slim python client.py 10.9.9.9 9000
OSError - [Errno 101] Network is unreachable
took 0.02s
$ docker run --rm --network sock-lab alpine:3.22 ip route
172.28.0.0/24 dev eth0 scope link  src 172.28.0.2

The route table has a single entry and no default route, which is what --internal gives you. There’s nowhere to send 10.9.9.9, so the kernel fails instantly without a packet leaving the container. On an ordinary network that address would go to the default gateway instead, and you’d get a timeout.

Two programs, one port

Start a second copy of the server in the same container:

$ docker exec server python echo_server.py
Traceback (most recent call last):
  File "/work/echo_server.py", line 4, in <module>
    srv = socket.create_server(("0.0.0.0", 9000))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/socket.py", line 950, in create_server
    raise error(err.errno, msg) from None
OSError: [Errno 98] Address already in use (while attempting to bind on address ('0.0.0.0', 9000))

A listening port belongs to one socket per address. The first server holds 0.0.0.0:9000, so the second can’t bind it. This is the error behind “my app won’t start after a crash” when the old process is still running. Each container has its own network namespace, so two containers can both listen on 9000. Two processes in the same container can’t.

UDP: no handshake, no guarantee

UDP sends single datagrams with no connection set up first. Send one to the server’s port 9000, where only TCP is listening:

# udp_probe.py
import socket, sys

s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.settimeout(2)
s.connect((sys.argv[1], int(sys.argv[2])))
print("sent", s.send(b"ping"), "bytes")
try:
    print("reply", s.recv(1024))
except OSError as e:
    print(type(e).__name__, "-", e)

$ docker run --rm --network sock-lab -v "$PWD":/work -w /work python:3.12-slim python udp_probe.py server 9000
sent 4 bytes
ConnectionRefusedError - [Errno 111] Connection refused

The send succeeded. UDP has no handshake, so there’s nothing to fail at send time. TCP 9000 and UDP 9000 are separate ports, so the datagram arrived at a port with no UDP listener. The server’s kernel replied with an ICMP “port unreachable” message, and Linux reported it on the next recv. It only does that because the socket is connected. An unconnected UDP socket using sendto would just time out. Silence is the normal UDP failure, which is why UDP protocols such as DNS retry on their own.

When you’re done: docker rm -f server and docker network rm sock-lab.

Practice

1. Give UDP a listener. Write a UDP echo server for port 9000, run it inside the existing server container next to the TCP one, and probe again. Does the TCP client still work?

# udp_echo.py
import socket

s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.bind(("0.0.0.0", 9000))
while True:
    data, addr = s.recvfrom(1024)
    s.sendto(data.upper(), addr)

$ docker exec -d server python udp_echo.py
$ docker run --rm --network sock-lab -v "$PWD":/work -w /work python:3.12-slim python udp_probe.py server 9000
sent 4 bytes
reply b'PING'
$ docker run --rm --network sock-lab -v "$PWD":/work -w /work python:3.12-slim python client.py server 9000
local  ('172.28.0.2', 50838)
remote ('172.28.0.10', 9000)
reply  b'HELLO'
took 0.02s

2. Let the kernel choose. Bind a server to port 0 three times and print the port you got. Where have you seen numbers like these before?

# port_zero.py
import socket

for _ in range(3):
    with socket.create_server(("0.0.0.0", 0)) as srv:
        print("the kernel picked port", srv.getsockname()[1])

$ docker exec server python port_zero.py
the kernel picked port 41521
the kernel picked port 39915
the kernel picked port 45657

3. A fifth failure. Run the client against sevrer (a typo) on port 9000. Which line of client.py raised, and why is it not a connection error?

$ docker run --rm --network sock-lab -v "$PWD":/work -w /work python:3.12-slim python client.py sevrer 9000
gaierror - [Errno -3] Temporary failure in name resolution
took 0.02s

4. Read the four errors as a table. Without rerunning anything, write down, for refused, timed out, no route to host and network unreachable: did a packet leave the client, did anything answer, and roughly how long did it take?