Container Hosts, Rootless Workloads, and Runtime Basics

1. A container host is a layered control plane

The word “runtime” is often used for several components. An engine such as Docker Engine or Podman manages images and workload lifecycle. A high-level runtime such as containerd or CRI-O provides orchestration-facing services. A low-level OCI runtime such as runc or crun creates the process boundary. The kernel supplies namespaces, cgroups, capabilities, seccomp, filesystems, and networking.

flowchart TD
  U["CLI, API, CI system, or orchestrator"] --> E["Engine or CRI implementation"]
  E --> I["Image store and snapshotter"]
  E --> H["High-level runtime and lifecycle state"]
  H --> O["OCI runtime"]
  O --> K["Linux kernel primitives"]
  K --> W["Container processes"]
  E --> N["Network and volume integration"]
  E --> L["Logs, events, metrics, policy"]

Security boundaries follow control authority. A user who can command a rootful runtime API may be able to create privileged workloads, mount host paths, access devices, or alter host networking. Protect runtime sockets and APIs as administrative interfaces.

2. Inventory the host before choosing operational defaults

Record distribution, kernel, cgroup mode, storage filesystem, security modules, runtime versions, registries, proxy settings, time synchronization, and update ownership. Container hosts are sensitive to kernel, filesystem, networking, and runtime interactions; an apparently small host change can affect every workload.

report="$HOME/devops-academy/linux/chapter19/lesson02/host-inventory"
mkdir -p "$report"

{
  date -u --iso-8601=seconds
  cat /etc/os-release
  uname -a
  findmnt -t cgroup2
  findmnt -T /var/lib/containers 2>/dev/null || true
  findmnt -T /var/lib/docker 2>/dev/null || true
  getenforce 2>/dev/null || true
  aa-status 2>/dev/null | head -n 30 || true
} > "$report/platform.txt"

for cmd in podman docker containerd crio runc crun; do
  if command -v "$cmd" >/dev/null; then
    "$cmd" --version >> "$report/runtime-versions.txt" 2>&1 || true
  fi
done

systemctl --no-pager --type=service --state=running   | grep -E 'docker|containerd|crio|podman' > "$report/services.txt" || true
ss -lxnp > "$report/unix-sockets.txt" 2>/dev/null || true
sha256sum "$report"/* > "$report/SHA256SUMS"

3. Image layers and writable layers have different durability semantics

OCI images are content-addressed and normally assembled from read-only layers. A container receives a writable layer through a storage driver or snapshotter. Copy-on-write improves reuse, but the writable layer is not an appropriate substitute for durable application data. Databases and stateful services need explicit volumes, backup scope, ownership, and restore testing.

# Inspect storage configuration without changing it.
podman info --format json 2>/dev/null | head -n 120 || true
docker info 2>/dev/null | sed -n '1,120p' || true

# Identify image and container storage consumption.
podman system df -v 2>/dev/null || docker system df -v 2>/dev/null || true

# Inspect mounts for a selected container.
name=${1:-example}
podman inspect "$name" --format '{{json .Mounts}}' 2>/dev/null ||   docker inspect "$name" --format '{{json .Mounts}}' 2>/dev/null || true

Pruning is destructive. Establish retention rules for images, stopped containers, build cache, volumes, and snapshots. Never run an unconditional prune on a shared host without inventory, ownership, exclusion, and recovery procedures.

4. Rootless execution reduces daemon and container privilege

Rootless Docker runs both daemon and containers inside a user namespace. Rootless Podman is daemonless for ordinary CLI use and stores user-owned state under the user’s data directories. Rootless containers cannot acquire more host privilege than the launching account, but networking, subordinate-ID mappings, low ports, device access, cgroup delegation, and filesystem ownership require deliberate design.

# Inspect subordinate IDs and lingering user services.
grep -E "^${USER}:" /etc/subuid /etc/subgid 2>/dev/null || true
loginctl show-user "$USER" -p Linger -p UID -p State

# Podman rootless state.
podman info --format '{{.Host.Security.Rootless}} {{.Store.GraphRoot}}'   2>/dev/null || true
systemctl --user status podman.socket --no-pager 2>/dev/null || true

# Docker rootless context and daemon state.
docker context ls 2>/dev/null || true
systemctl --user status docker --no-pager 2>/dev/null || true

# Confirm host identity mapping for a running container process.
# cat /proc/HOST_PID/uid_map

5. Runtime sockets are privileged control interfaces

A rootful Docker socket, containerd socket, CRI socket, or remotely exposed API can permit administrative actions far beyond starting one application. Unix-socket permissions, group membership, TLS policy, namespace scope, and network exposure determine who can control the host.

# Enumerate likely runtime sockets and owners.
for socket in /run/docker.sock /run/containerd/containerd.sock               /var/run/crio/crio.sock /run/podman/podman.sock; do
  [[ -S $socket ]] && stat -Lc '%A %U:%G %n' "$socket"
done

# Audit users in groups commonly granted runtime control.
getent group docker 2>/dev/null || true
getent group podman 2>/dev/null || true

# Look for unexpected TCP listeners related to runtime APIs.
ss -lntp 2>/dev/null | grep -Ei 'docker|containerd|podman|2375|2376' || true

6. Pull by digest when identity must be immutable

A tag is a movable registry reference. A digest identifies a specific manifest. Record both the human-friendly source and resolved digest in deployment evidence. Verify registry trust, TLS, authentication, image provenance, signatures or attestations, vulnerability status, and policy before execution.

image=${1:-docker.io/library/alpine:latest}

# Pull and record the resolved identity.
podman pull "$image" 2>/dev/null || docker pull "$image"
podman image inspect "$image"   --format '{{.Id}} {{join .RepoDigests " "}}' 2>/dev/null ||   docker image inspect "$image"   --format '{{.Id}} {{join .RepoDigests " "}}'

# Save an inspection record; do not assume labels prove provenance.
mkdir -p "$HOME/devops-academy/linux/chapter19/lesson02"
podman image inspect "$image" 2>/dev/null >   "$HOME/devops-academy/linux/chapter19/lesson02/image-inspect.json" ||   docker image inspect "$image" >   "$HOME/devops-academy/linux/chapter19/lesson02/image-inspect.json"

7. Harden the workload specification and the host together

Prefer a non-root application UID, read-only root filesystem where practical, explicit writable mounts, dropped capabilities, no-new-privileges, default or stricter seccomp policy, LSM confinement, bounded CPU/memory/PID resources, no host PID/network namespace, and no runtime socket mount. Avoid privileged mode and unrestricted device access.

# Example using Podman; Docker supports comparable flags.
podman run --rm   --user 65532:65532   --read-only   --tmpfs /tmp:rw,noexec,nosuid,nodev,size=64m   --cap-drop=all   --security-opt no-new-privileges   --memory=256m --cpus=0.5 --pids-limit=128   docker.io/library/alpine:latest   sh -c 'id; mount | head; grep -E "^(CapEff|NoNewPrivs|Seccomp):" /proc/self/status'

# Inspect the effective result after creation.
podman inspect CONTAINER 2>/dev/null | less ||   docker inspect CONTAINER 2>/dev/null | less

Do not copy flags blindly. Some applications require writable paths, signals, IPC, low ports, or specific capabilities. Discover requirements in a test environment and grant the narrowest documented exception.

8. Lifecycle, restart policy, health, and logs must agree

A restart policy is not application recovery. The process may repeatedly restart while dependencies fail or data remains inconsistent. Define startup ordering, readiness, liveness, graceful termination, stop timeout, log retention, and escalation. On systemd hosts, decide whether the engine or systemd owns restart behavior to avoid competing supervisors.

# Runtime lifecycle and events.
podman ps --all --no-trunc 2>/dev/null || docker ps --all --no-trunc
podman events --since 10m --stream=false 2>/dev/null ||   docker events --since 10m --until "$(date --iso-8601=seconds)" 2>/dev/null || true

# Inspect restart, health, and log configuration.
name=${1:-example}
podman inspect "$name" 2>/dev/null | less || docker inspect "$name" | less
podman logs --since 10m "$name" 2>/dev/null || docker logs --since 10m "$name" 2>/dev/null || true

# Runtime service logs.
journalctl -u docker -u containerd -u crio --since -30min --no-pager 2>/dev/null || true
journalctl --user -u podman.socket -u docker --since -30min --no-pager 2>/dev/null || true

9. Hands-on lab: run and verify a constrained disposable workload

The lab chooses Podman when available and otherwise Docker. It pulls a small public image, runs a disposable constrained process, records effective controls, and removes the container automatically.

set -Eeuo pipefail
lab="$HOME/devops-academy/linux/chapter19/lesson02/lab"
mkdir -p "$lab"
image=docker.io/library/alpine:latest

if command -v podman >/dev/null; then
  engine=podman
elif command -v docker >/dev/null; then
  engine=docker
else
  printf 'Install no runtime for this lab; review commands only.
' > "$lab/result.txt"
  exit 0
fi

"$engine" pull "$image" > "$lab/pull.txt"
"$engine" image inspect "$image" > "$lab/image.json"
"$engine" run --rm   --read-only --cap-drop=all --security-opt no-new-privileges   --memory=128m --cpus=0.25 --pids-limit=32   --tmpfs /tmp:rw,noexec,nosuid,nodev,size=16m   "$image" sh -c '
    id
    grep -E "^(CapEff|NoNewPrivs|Seccomp):" /proc/self/status
    printf "cgroup="; cat /proc/self/cgroup
    touch /tmp/allowed
    touch /root/blocked 2>&1 || true
  ' > "$lab/runtime-evidence.txt" 2>&1

sha256sum "$lab"/* > "$lab/SHA256SUMS"
printf 'Engine=%s; review %s
' "$engine" "$lab"

10. Container-host readiness checklist

• Supported distribution, kernel, filesystem, cgroup mode, and runtime versions are defined.
• Runtime APIs and groups are restricted, inventoried, and monitored.
• Rootless or user-namespace designs are tested against workload requirements.
• Registry trust, digest recording, provenance, vulnerability response, and retention are owned.
• Workloads have non-root identity, resource limits, minimal capabilities, seccomp, LSM policy, and bounded mounts.
• Logs, events, metrics, storage growth, image garbage collection, updates, reboot, backup, and recovery are operationalized.

11. Knowledge check

12. Summary

• A container host combines engines, runtimes, storage, networking, kernel controls, and administrative APIs.
• Rootless execution reduces privilege but requires compatibility and resource-control validation.
• Runtime sockets and groups are administrative security boundaries.
• Digest-pinned images, provenance, constrained specifications, and effective-state inspection support repeatability.
• Host operations must cover lifecycle, logs, updates, storage growth, and recovery.

13. Further reading

• Docker Engine — Rootless mode
• podman(1)
• OCI Image Format Specification
• containerd operations documentation
• systemd resource control