Ports, Sockets, TCP, UDP, and Connection Diagnostics
Understand how Linux processes bind sockets and how TCP, UDP, ports, namespaces, firewalls, and application protocols interact during connection troubleshooting.
Learning objectives
By the end of this lesson
- Explain sockets, local and remote address-port tuples, and wildcard binding.
- Distinguish TCP connection state from UDP datagram behavior.
-
Use
ss,lsof,nc,curl, and OpenSSL for layered diagnostics. - Identify common causes of timeout, refusal, reset, and protocol mismatch.
- Run a safe loopback-only service test and preserve evidence.
1. A port belongs to a protocol, address, namespace, and socket
A port number alone does not identify a service. TCP port 53 and UDP
port 53 are separate transport endpoints. Binding to
127.0.0.1 accepts only local IPv4 traffic; binding to
0.0.0.0 requests all local IPv4 addresses; an IPv6
wildcard may or may not accept mapped IPv4 connections depending on
system and socket options. Containers and network namespaces can
have independent socket tables.
flowchart TD A["Client process"] --> B["Local IP:ephemeral port"] B --> C["TCP or UDP transport"] C --> D["Routing and firewall path"] D --> E["Server IP:service port"] E --> F["Listening or bound socket"] F --> G["Server process"] H["Network namespace"] -. scopes .-> B H -. scopes .-> E
2. TCP creates state and reports different failure classes
TCP uses a handshake before application data. A successful handshake proves bidirectional transport to a listening socket, not that TLS or the application protocol is healthy. A prompt “connection refused” usually means a reachable host actively rejected the connection because no listener or firewall reject rule exists. A timeout suggests dropped probes, an unreachable path, filtering, or a nonresponsive destination. A reset means an established or attempted connection was actively terminated.
ss -lntp
ss -tn state syn-sent
ss -tn state established
ss -tn state time-wait
# Extended information for active TCP sockets
ss -tnoi
# Which process has a local TCP listener? Permission may limit output.
lsof -nP -iTCP -sTCP:LISTEN 2>/dev/null | head -n 40
3. UDP is connectionless at the protocol level
UDP sockets can be bound or connected from the application’s perspective, but the transport does not perform a handshake or guarantee delivery, ordering, or retransmission. A successful send often proves only that the local kernel accepted a datagram. ICMP errors may arrive later or be filtered. Diagnose UDP with application-aware queries, packet capture where authorized, counters, and server-side evidence.
ss -lnup
ss -uap
# UDP error and protocol counters
nstat -az 2>/dev/null | grep -E 'Udp|Ip.*NoRoute' | head -n 60 || true
# Example DNS query is more meaningful than a generic UDP probe
# dig @resolver.example.net example.com A
A raw port check cannot validate an application exchange. Test DNS with a DNS query, HTTP with an HTTP request, TLS with a handshake, and databases with their client protocol.
4. Move from socket existence to application behavior
# TCP connect test with a bounded timeout
nc -vz -w 3 example.com 443 2>&1 || true
# HTTP headers, redirects, timing, and errors
curl --connect-timeout 5 --max-time 15 -I https://example.com/
# Separate DNS, connect, TLS, first-byte, and total timing
curl -sS -o /dev/null \
--connect-timeout 5 --max-time 20 \
-w 'remote=%{remote_ip} code=%{http_code} dns=%{time_namelookup} connect=%{time_connect} tls=%{time_appconnect} first_byte=%{time_starttransfer} total=%{time_total}\n' \
https://example.com/
# Inspect TLS handshake and certificate chain
openssl s_client -connect example.com:443 -servername example.com </dev/null 2>/dev/null | sed -n '1,45p'
Server Name Indication matters for virtual-hosted TLS services, so
the hostname passed to -servername can change the
certificate and backend. HTTP Host headers, proxies, redirects,
authentication, and application routing occur after transport
connectivity.
5. A local listener can still be unreachable
Check the exact bind address, network namespace, host firewall, cloud security controls, container publishing, and reverse proxy configuration. A service listening on loopback is intentionally unavailable on external interfaces. A container listener may exist only in the container namespace unless a port is published or routed.
# Exact listener addresses and process ownership
ss -lntup
# List network namespaces managed by iproute2
ip netns list 2>/dev/null || true
# Read-only firewall views; availability varies by distribution
sudo nft list ruleset 2>/dev/null | sed -n '1,180p' || true
sudo iptables -S 2>/dev/null | sed -n '1,120p' || true
# Socket-owning systemd units may also be relevant
systemctl list-sockets --all --no-pager 2>/dev/null || true
Removing policy destroys evidence and can expose services. Capture the ruleset, identify the matching chain and counter, and make a reviewed minimal change with a rollback path.
6. Hands-on lab: test a loopback-only HTTP service
This lab starts a temporary Python server bound only to loopback, inspects its socket, performs an HTTP request, and stops it cleanly.
lab="$HOME/devops-academy/linux/chapter12/lesson04"
mkdir -p "$lab/site"
cd "$lab"
printf '<h1>DevOps Academy socket lab</h1>\n' > site/index.html
python3 -m http.server 18080 --bind 127.0.0.1 --directory site \
> server.log 2>&1 &
server_pid=$!
trap 'kill "$server_pid" 2>/dev/null || true' EXIT
sleep 1
ss -lntp 'sport = :18080' > listener.txt 2>&1 || true
curl --fail --show-error --max-time 5 \
http://127.0.0.1:18080/ > response.html
printf 'pid=%s\n' "$server_pid" > process.txt
ps -p "$server_pid" -o pid,ppid,stat,comm,args >> process.txt
kill "$server_pid"
wait "$server_pid" 2>/dev/null || true
trap - EXIT
sha256sum listener.txt response.html process.txt server.log > evidence.sha256
Verification checklist
7. Common connection-diagnostic mistakes
“Port 443 is open, so HTTPS works.”
The TCP handshake may succeed while TLS, certificate selection, HTTP routing, or the application fails.
“UDP connect succeeded.”
A local send does not prove the server received or answered the datagram.
“0.0.0.0 is a remote address.”
In a listener it represents all local IPv4 addresses, not a destination clients connect to.
“The process is listening on the host.”
It may be listening inside another network namespace or container.
8. Knowledge check
Question 1. What usually distinguishes connection refused from a timeout?
Question 2. Why is an HTTP request stronger evidence than a generic TCP port check?
Question 3. Why might ss on the host not show a
container process listener?
9. Summary
A service endpoint is defined by protocol, local address, port, namespace, and process. TCP provides handshake and lifecycle states; UDP requires application-aware evidence. Diagnose from local binding through route and firewall to TLS and application behavior, and distinguish refusal, timeout, reset, and protocol errors rather than calling all of them “network failures.”
10. Further reading
-
socket(7),tcp(7),udp(7), andss(8). -
curl(1),openssl-s_client(1),nc(1), andlsof(8). - nftables documentation and container runtime networking documentation.
Keep the academy open
Support free, practical DevOps education.
Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0
Send only Ethereum/ERC-20 compatible assets to this
address.