Docker Engine Components: dockerd, containerd, runc, BuildKit, APIs, and Container Lifecycle: Configuration, Design Choices, and Tradeoffs
Choose deliberately between Docker-managed runtime state and direct lower-level tools, CLI versus Engine API automation, integrated versus isolated BuildKit builders, and daemon versus workload recovery scopes.
Learning objectives
- Choose the supported control surface that owns the state being changed.
- Compare CLI and Engine API automation without weakening Docker socket permissions.
- Compare integrated and isolated BuildKit builder designs and their output contracts.
- Distinguish daemon recovery from container/workload recovery and live-restore behavior.
- Explain why Docker-managed containerd is not a routine second administration plane.
docker version, docker info,
docker buildx version, and platform-specific runtime
evidence on the machine that actually runs the lab.
1. Four decisions that look similar but affect different trust boundaries
Engine internals are useful only if they improve design decisions. The correct question is not “Can I reach containerd?” but “Which supported control surface should own this change, and what evidence will prove it?”
2. Docker-managed runtime state versus direct containerd administration
Docker Engine uses containerd for container lifecycle and, on fresh Engine 29 installations, uses the containerd image store by default. That does not mean Docker-managed containerd should be treated as a second independent administration surface. Docker's object metadata, networking, volumes, restart policy, security configuration, and API behavior remain Engine concerns.
| Approach | Good fit | Main risk | Evidence |
|---|---|---|---|
| Docker CLI / Engine API | Normal Docker lifecycle and automation | Wrong context/API permissions if poorly scoped | API response, object ID, events, inspect |
| Read-only lower-level inspection | Deep incident diagnosis on an authorized host | Misinterpreting namespace/runtime internals | processes, versions, sockets, logs |
| Direct mutation of Docker-managed containerd | Generally not a supported routine repair path | State divergence from Docker Engine ownership | Hard to reconcile safely |
3. Engine API automation versus CLI wrappers
The Docker CLI is excellent for humans and many scripts. For long-lived automation, the Engine API or supported SDK can provide structured responses, HTTP status codes, and explicit API negotiation. Both still require the exact context/endpoint and authorization boundary to be understood.
# Read-only API evidence on a local rootful Linux Engine socket.
# Run only if your account is already authorized to the Docker socket.
curl --unix-socket /var/run/docker.sock http://localhost/_ping
curl --unix-socket /var/run/docker.sock http://localhost/version
Socket access is high privilege. Do not grant it merely to make an
API lab work. On rootless Docker the socket path differs; on Desktop
the transport is managed by Desktop. The correct lab fallback is
docker version, not permission weakening.
4. Integrated BuildKit versus an isolated builder
| Builder | Strength | Tradeoff | Output behavior to verify |
|---|---|---|---|
docker driver |
No extra configuration; bundled BuildKit | BuildKit version/options are controlled by Engine | Local load is natural/default for supported result |
docker-container |
Custom BuildKit image/config, isolated builder state | Extra container/volume and explicit lifecycle |
Use --load or --push when needed
|
| remote / Kubernetes | Separate capacity/trust domain | Network/auth/availability complexity | Exporter target must be explicit |
For Chapter 03, the default builder is the simplest baseline. An isolated builder becomes useful when you need a different BuildKit version, advanced cache/export behavior, or a separate trust/resource boundary.
5. Daemon recovery versus workload recovery
By default, loss of dockerd affects running containers.
Docker's live-restore feature can keep eligible
standalone Linux containers running while the daemon is unavailable,
but it is not the same thing as a container restart policy and has
explicit upgrade/configuration caveats. It is also not a general
high-availability system.
6. Engine 29.7+ embedded containerd: useful nuance, not a default assumption
Current Docker documentation describes an experimental mode in which
dockerd can run containerd in the same process. The
default still starts and manages containerd separately. This is a
strong reason to diagnose from supported API/object evidence first:
process topology can evolve while the Engine ownership model remains
the stable abstraction.
7. Worked decision scenario
A CI team needs reproducible builds with an exact BuildKit version
and wants build cache state isolated from the host's default Docker
builder. They do not need to manage Docker containers through
containerd directly. The appropriate choice is a dedicated
docker-container Buildx builder pinned to a reviewed
BuildKit image, with explicit --push or
--load output and lifecycle/cleanup ownership. Direct
ctr mutation would solve the wrong problem.
A different operations team needs to restart
dockerd for a patch update on a dedicated Linux host
with strict downtime requirements. Their design decision concerns
live-restore compatibility, log buffering, changed daemon options,
and rollback—not a custom builder or container restart policy.
8. Decision checklist
- Which layer owns the state I intend to change?
- Is the interface supported and versioned?
- What exact object/process/build IDs will prove success?
- Can a failed attempt be rolled back without deleting unrelated state?
- Does the choice expand daemon/socket/runtime privileges?
- Will a future operator understand which configuration source is authoritative?
Knowledge check
When is the default docker Buildx driver a good choice?
When the integrated Engine builder is sufficient and automatic loading into the local image store is desirable. It minimizes extra moving parts but offers less BuildKit configurability.
What is the main operational difference between daemon restart and container restart?
A daemon restart changes the control plane. A container restart stops and starts workload processes. Live-restore may keep eligible standalone containers running while dockerd is unavailable, so the two events are not equivalent.
Why is Engine API automation often preferable to parsing docker CLI text output?
The API provides structured, versioned data and explicit status codes. CLI wrappers are excellent for humans, but scripts that parse presentation text can be brittle. API negotiation and authorization still need explicit handling.
What is the risk of treating the moby containerd namespace as a general administration surface?
It bypasses Docker Engine ownership. Direct changes can create unsupported state divergence and make subsequent Docker operations misleading or destructive.
Official references and version notes
- Docker architecture overview — client/server ownership and the daemon's responsibility for Docker objects.
- Docker Engine API — API negotiation and the current Engine/API compatibility matrix.
- Docker Engine 29 release notes — current Engine 29 packaging, component, security, and compatibility changes.
- Alternative container runtimes — Docker Engine's use of containerd for container lifecycle and runc as the default OCI runtime.
- containerd image store with Docker Engine — current Engine 29 storage architecture and upgrade/fresh-install differences.
- Run containerd in the Docker daemon — current experimental embedded-containerd boundary introduced in Engine 29.7.
- BuildKit — Docker's modern build backend and its graph/cache model.
- Builders — default and custom BuildKit builder ownership.
- Docker build driver — integrated BuildKit behavior and local-image loading.
- Docker-container build driver — isolated, configurable BuildKit builder behavior and output semantics.
- Live restore — daemon-unavailability behavior, scope, caveats, and upgrade constraints.
- docker system events — event-stream evidence for object lifecycle transitions.
Verified 2026-09-20: Docker Engine 29.8.1 is current; the Engine 29.8 API matrix lists maximum API 1.55 and minimum API 1.40. Engine 29.8 packaging includes BuildKit 0.33.0 and runc 1.5.1; 29.8.1 updates the static-binary containerd package to 2.3.5. Package-managed distributions and Docker Desktop can bundle or expose components differently. Record the actual client/server/API/component/builder versions on the learner's environment before diagnosing compatibility.
Keep the academy open
Support free, practical DevOps education.
Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0
Send only Ethereum/ERC-20 compatible assets to this
address.