Docker Engine Components: dockerd, containerd, runc, BuildKit, APIs, and Container Lifecycle: Concepts, Architecture, and Mental Model

1. The problem: one command crosses several ownership boundaries

Chapter 01 established that a container is an isolated process environment rather than a miniature virtual machine, and Chapter 02 separated the Docker client, selected context, daemon endpoint, installation source, and host prerequisites. The next failure mode is subtler: after the client reaches a healthy daemon, engineers often treat every internal failure as “Docker failed.” That phrase is too coarse to troubleshoot.

A request such as docker run begins at the CLI or SDK, crosses the Engine API, is interpreted by dockerd, uses Docker-managed containerd services for content and task lifecycle, invokes a containerd shim and an OCI runtime such as runc, and finally asks the kernel to create the workload process with its namespaces, cgroups, mounts, and security settings. A build request shares the client/API/control-plane entrance but branches into BuildKit, whose workers, cache, frontends, and exporters are a different execution path.

2. The Engine request path

The useful mental model is a control path plus a runtime path. The CLI is a client. dockerd is the Docker Engine control plane. containerd is a managed runtime/content service. A shim maintains the task boundary independently of the short-lived runc invocation. The Linux kernel actually schedules and isolates the resulting process.

flowchart TD
  A[Docker CLI or SDK] --> B[Engine API]
  B --> C[dockerd control plane]
  C --> D[Docker-managed containerd]
  D --> E[containerd shim]
  E --> F[runc OCI runtime]
  F --> G[Kernel process + namespaces + cgroups]
  C --> H[Docker object metadata]
  D --> I[content / snapshots / task state]
  G --> J[logs + exit + resource evidence]
            

The arrows are ownership transitions, not a promise that every Docker package exposes each component as a separately managed host service. Engine 29.7 introduced an experimental embedded-containerd mode; the default remains Docker-managed containerd behavior, and shims still remain separate task processes. Course labs therefore inspect the installed environment instead of assuming process topology.

3. Object state, task state, and process state are different

4. The build path branches through BuildKit

BuildKit is the modern Docker build backend. The default Buildx builder uses the docker driver and the BuildKit library bundled with the Engine; custom builders can run BuildKit in a dedicated container, Kubernetes, or a remote daemon. This means “Docker build succeeded” is incomplete unless you know which builder solved the graph and which exporter handled the result.

flowchart TD
  A[Docker CLI / buildx] --> B[Selected builder]
  B --> C{Driver}
  C -->|docker| D[BuildKit bundled with Engine]
  C -->|docker-container| E[Dedicated BuildKit container]
  C -->|remote / kubernetes| F[External BuildKit worker]
  D --> G[cache + result exporter]
  E --> G
  F --> G
  G --> H[local load / registry push / OCI output / cache-only]
            

5. Read-only baseline: identify the components you actually have

docker version
docker context show
docker context inspect "$(docker context show)"
docker info

docker buildx version
docker buildx ls
docker buildx inspect --bootstrap

# Native Linux Engine only; these may not be host-visible on Docker Desktop.
command -v containerd && containerd --version || true
command -v runc && runc --version || true
ps -eo pid,ppid,comm,args | grep -E '[d]ockerd|[c]ontainerd|[c]ontainerd-shim' || true

Do not treat a missing containerd or runc binary in the outer user shell as proof that Docker does not use those components. Docker Desktop places the Linux Engine inside a managed VM, and package layouts differ. The supported Docker-facing evidence is still docker version, docker info, and builder inspection.

6. Current Engine 29 baseline and why it is only a baseline

As of 2026-09-20, Docker Engine 29.8.1 is current. The Engine 29.8 API matrix lists API 1.55 maximum and 1.40 minimum. Engine 29.8 packaging includes BuildKit 0.33.0 and runc 1.5.1; 29.8.1 updates the static-binary containerd package to 2.3.5. Those numbers are useful compatibility anchors, but your package source or Docker Desktop can expose a different packaging surface. Diagnostics must start from installed evidence.

7. Why this model matters in DevOps

CI systems, release pipelines, incident responders, and platform teams need to distinguish source/build identity from Engine objects and runtime processes. A reliable incident note can say: “CLI 29.8.1 using context X negotiated API 1.55 with daemon Y; build builder Z produced image ID A; container B was created and task PID C started; the first failure occurred after start in application health.” That statement is actionable because every claim has an owner and observable evidence.

Knowledge check

Official references and version notes

• Docker architecture overview — client/server ownership and the daemon's responsibility for Docker objects.
• Docker Engine API — API negotiation and the current Engine/API compatibility matrix.
• Docker Engine 29 release notes — current Engine 29 packaging, component, security, and compatibility changes.
• Alternative container runtimes — Docker Engine's use of containerd for container lifecycle and runc as the default OCI runtime.
• containerd image store with Docker Engine — current Engine 29 storage architecture and upgrade/fresh-install differences.
• Run containerd in the Docker daemon — current experimental embedded-containerd boundary introduced in Engine 29.7.
• BuildKit — Docker's modern build backend and its graph/cache model.
• Builders — default and custom BuildKit builder ownership.
• Docker build driver — integrated BuildKit behavior and local-image loading.
• Docker-container build driver — isolated, configurable BuildKit builder behavior and output semantics.
• Live restore — daemon-unavailability behavior, scope, caveats, and upgrade constraints.
• docker system events — event-stream evidence for object lifecycle transitions.