Docker Engine Components: dockerd, containerd, runc, BuildKit, APIs, and Container Lifecycle: Guided Hands-On Workflow and Core Operations

1. Lab scope and preflight

This workflow uses one disposable container and one tiny build. It does not change daemon configuration, mount the Docker socket into a container, use privileged mode, or directly modify containerd. Run it only on a personal/disposable Docker environment or an authorized lab host.

set -eu
LAB=da-ch03-runtime
LABEL='devops-academy.lab=chapter03'
WORK="${TMPDIR:-/tmp}/da-ch03-build"
mkdir -p "$WORK"

docker version
docker context show
docker info --format 'Server={{.ServerVersion}} Driver={{.Driver}} Cgroup={{.CgroupVersion}}'
docker buildx version
docker buildx ls

2. Record an image input before creating a container

Use a small versioned base image and record the resolved immutable digest when the registry supplies one. The version tag remains human-readable; the digest tells you which bytes were actually selected.

docker pull busybox:1.36.1
docker image inspect busybox:1.36.1   --format 'ImageID={{.Id}} RepoDigests={{json .RepoDigests}}'

3. Separate create from start

docker create creates the Docker object without starting its process. This makes the state boundary visible.

docker create \
  --name "$LAB" \
  --label "$LABEL"   busybox:1.36.1   sh -c 'echo "container pid1=$$"; trap "echo TERM-received; exit 0" TERM; while :; do sleep 5; done'

docker inspect "$LAB" --format 'ID={{.Id}} Status={{.State.Status}} PID={{.State.Pid}} Image={{.Image}}'

Before start, the object exists but .State.Pid should not represent a running workload process. That is the first proof that Docker object lifecycle and process lifecycle are distinct.

4. Start it, then correlate Docker state with runtime state

START=$(date -u +%Y-%m-%dT%H:%M:%SZ)
docker start "$LAB"
sleep 1
PID=$(docker inspect -f '{{.State.Pid}}' "$LAB")
docker inspect "$LAB" --format 'Status={{.State.Status}} PID={{.State.Pid}} Started={{.State.StartedAt}}'
docker logs "$LAB"

# Native Linux Engine host only:
ps -fp "$PID" || true
ls -l "/proc/$PID/ns" 2>/dev/null || true
cat "/proc/$PID/cgroup" 2>/dev/null || true
ps -eo pid,ppid,comm,args | grep '[c]ontainerd-shim' || true

On Docker Desktop, the PID belongs to the Linux VM and normally will not map to an outer Windows/macOS PID. Record that limitation instead of calling the evidence “missing.”

5. Capture the lifecycle timeline with events

Current state answers “what exists now.” Events answer “what happened.” After stopping the container, query the bounded timeline.

docker stop --time 10 "$LAB"
END=$(date -u +%Y-%m-%dT%H:%M:%SZ)
docker events --since "$START" --until "$END" --filter "container=$LAB"
docker inspect "$LAB" --format 'Status={{.State.Status}} Exit={{.State.ExitCode}} Finished={{.State.FinishedAt}}'

Look for create/start/stop/die-related events and correlate timestamps with the inspect state and logs. Event vocabulary can vary with operation and platform, so preserve raw output rather than forcing a prewritten story onto it.

6. Run one BuildKit build and identify the builder path

The Dockerfile is intentionally trivial because this chapter is about build ownership, not Dockerfile instruction mastery.

cat > "$WORK/Dockerfile" <<'EOF'
FROM busybox:1.36.1
RUN printf 'chapter03-build-evidence\n' > /evidence.txt
CMD ["cat", "/evidence.txt"]
EOF

cd "$WORK"
docker buildx ls
docker buildx inspect --bootstrap
docker buildx build --progress=plain --load -t da-ch03-build:local .
docker image inspect da-ch03-build:local --format 'ImageID={{.Id}} Created={{.Created}}'

With the default docker driver, the BuildKit backend is integrated with Docker Engine and a single-platform result is loaded locally. If your selected builder uses another driver, --load is the explicit contract that makes a Docker Engine image-store result part of this lab.

7. Inspect daemon/runtime evidence without mutating lower-level state

# Native Linux Engine examples; read-only.
systemctl status docker --no-pager 2>/dev/null || true
journalctl -u docker --since "$START" --no-pager 2>/dev/null | tail -n 80 || true
containerd --version 2>/dev/null || true
runc --version 2>/dev/null || true

8. Layer-selection challenge

Classify each symptom before proposing a fix: (a) docker version cannot reach the selected endpoint; (b) daemon accepts create but start fails with an OCI runtime error; (c) Buildx reports success but the image is absent locally; (d) container PID exists but application health fails. The likely first layers are respectively context/API transport, runtime/process creation, builder/exporter contract, and application/runtime health—not one generic “Docker problem.”

9. Exact cleanup

docker rm -f "$LAB" 2>/dev/null || true
docker image rm da-ch03-build:local 2>/dev/null || true
rm -rf "$WORK"

docker ps -a --filter "label=$LABEL"
docker image inspect da-ch03-build:local >/dev/null 2>&1 && echo 'image still present' || echo 'lab image removed'

Do not substitute docker system prune. Exact cleanup is part of the evidence chain because it proves which resources the lab owned.

Knowledge check

Official references and version notes

• Docker architecture overview — client/server ownership and the daemon's responsibility for Docker objects.
• Docker Engine API — API negotiation and the current Engine/API compatibility matrix.
• Docker Engine 29 release notes — current Engine 29 packaging, component, security, and compatibility changes.
• Alternative container runtimes — Docker Engine's use of containerd for container lifecycle and runc as the default OCI runtime.
• containerd image store with Docker Engine — current Engine 29 storage architecture and upgrade/fresh-install differences.
• Run containerd in the Docker daemon — current experimental embedded-containerd boundary introduced in Engine 29.7.
• BuildKit — Docker's modern build backend and its graph/cache model.
• Builders — default and custom BuildKit builder ownership.
• Docker build driver — integrated BuildKit behavior and local-image loading.
• Docker-container build driver — isolated, configurable BuildKit builder behavior and output semantics.
• Live restore — daemon-unavailability behavior, scope, caveats, and upgrade constraints.
• docker system events — event-stream evidence for object lifecycle transitions.