Docker Engine Components: dockerd, containerd, runc, BuildKit, APIs, and Container Lifecycle: Diagnostics, Failure Modes, Security, and Performance

1. Preserve evidence before touching the layer you suspect

The fastest-looking fix—restart Docker, delete runtime files, run a lower-level tool, add privilege—often erases the information that would identify the actual failing layer. Start with a timestamped evidence packet.

date -u
docker context show
docker version
docker info
docker buildx version
docker buildx ls
docker ps -a --no-trunc
docker events --since 10m --until "$(date -u +%Y-%m-%dT%H:%M:%SZ)" 2>/dev/null | tail -n 100

2. Diagnostic sequence by causal layer

1. Client/context/endpoint: did the request reach the intended daemon?
2. Engine API/dockerd: did the daemon accept and validate the operation?
3. Image/build input: did the exact content exist and resolve?
4. Runtime creation: did containerd/shim/runc create the task/process?
5. Process/health: did PID 1 start, stay running, and behave correctly?
6. Storage/resource/security: did mounts, cgroups, capabilities, LSM policy, or permissions block the process?
7. Network/external dependency: did post-start communication fail?

Only move downward when the previous layer has evidence of success.

3. Intentionally broken example: wrong context, not broken runtime

This safe exercise creates a local Docker context that points to a nonexistent Unix socket. It does not change the daemon.

docker context create da-ch03-broken   --docker 'host=unix:///tmp/da-ch03-missing.sock'

docker --context da-ch03-broken version || true

docker context show
docker --context default version 2>/dev/null || true

docker context rm da-ch03-broken

The expected failure occurs before dockerd, containerd, or runc on the real host receives a request. The repair is to select/correct the endpoint, not to restart runtimes.

4. Intentionally broken example: forced obsolete API version

Engine 29.8 supports API negotiation, with the current matrix listing minimum API 1.40. Forcing a much older API version is a reversible way to demonstrate an API-layer failure.

DOCKER_API_VERSION=1.24 docker version || true
unset DOCKER_API_VERSION
docker version

Preserve the exact error. If normal negotiation works immediately afterward, there is no evidence that runc or containerd was the problem.

5. “Build succeeded but image missing”: builder/exporter diagnosis

A custom builder can successfully solve a build while leaving the result only in BuildKit cache because no local-load or push exporter was requested.

WORK="${TMPDIR:-/tmp}/da-ch03-missing-image"
mkdir -p "$WORK"
printf 'FROM busybox:1.36.1
RUN echo ok > /ok
' > "$WORK/Dockerfile"

docker buildx create --name da-ch03-isolated --driver docker-container --use --bootstrap
docker buildx build --progress=plain -t da-ch03-cacheonly:local "$WORK"
docker image inspect da-ch03-cacheonly:local || true

# Correct the output contract instead of blaming Engine image storage.
docker buildx build --progress=plain --load -t da-ch03-cacheonly:local "$WORK"
docker image inspect da-ch03-cacheonly:local --format '{{.Id}}'

docker buildx use default 2>/dev/null || true
docker buildx rm da-ch03-isolated
docker image rm da-ch03-cacheonly:local 2>/dev/null || true
rm -rf "$WORK"

6. Daemon restart is not container restart

If dockerd restarts, the control plane changes. If a workload process restarts, application runtime state changes. Live-restore can intentionally keep certain standalone containers running across daemon unavailability, but only within documented constraints. Therefore a daemon PID change does not by itself prove the application process restarted, and an application PID change does not prove the daemon restarted.

7. Unsafe shortcuts to reject

8. Performance: locate the bottleneck before tuning

Slow docker build can be context transfer, cache miss, BuildKit worker CPU/I/O, registry latency, snapshotter/storage behavior, or exporter time. Slow docker run can be image pull/unpack, runtime setup, mount/network initialization, or application startup. Measure the phase and preserve timestamps before increasing resources or concurrency.

Knowledge check

Official references and version notes

• Docker architecture overview — client/server ownership and the daemon's responsibility for Docker objects.
• Docker Engine API — API negotiation and the current Engine/API compatibility matrix.
• Docker Engine 29 release notes — current Engine 29 packaging, component, security, and compatibility changes.
• Alternative container runtimes — Docker Engine's use of containerd for container lifecycle and runc as the default OCI runtime.
• containerd image store with Docker Engine — current Engine 29 storage architecture and upgrade/fresh-install differences.
• Run containerd in the Docker daemon — current experimental embedded-containerd boundary introduced in Engine 29.7.
• BuildKit — Docker's modern build backend and its graph/cache model.
• Builders — default and custom BuildKit builder ownership.
• Docker build driver — integrated BuildKit behavior and local-image loading.
• Docker-container build driver — isolated, configurable BuildKit builder behavior and output semantics.
• Live restore — daemon-unavailability behavior, scope, caveats, and upgrade constraints.
• docker system events — event-stream evidence for object lifecycle transitions.