Checkpoint Lab — Containerd Image Store, Snapshotters, Runtime Internals, OCI Runtime Specs, and Docker Engine Evolution

1. Checkpoint outcome

You will trace alpine:3.22.1 through a complete but bounded chain: registry metadata → local Docker image identity → reported store/snapshotter mode → Docker container object → running process → optional read-only containerd correlation → exact cleanup. The checkpoint is successful only if every observation is labeled with the layer it proves.

2. Assumptions and platform matrix

3. Preflight: host, context, credentials, disk, and preexisting resources

set -eu
LAB=dca31-checkpoint
EVIDENCE="$LAB/evidence"
mkdir -p "$EVIDENCE"

date -u +%Y-%m-%dT%H:%M:%SZ | tee "$EVIDENCE/time.txt"
docker context show | tee "$EVIDENCE/context.txt"
docker version | tee "$EVIDENCE/docker-version.txt"
docker info | tee "$EVIDENCE/docker-info.txt"
docker buildx version 2>/dev/null | tee "$EVIDENCE/buildx-version.txt" || true
docker compose version 2>/dev/null | tee "$EVIDENCE/compose-version.txt" || true
docker system df | tee "$EVIDENCE/system-df-before.txt"

# Guard exact lab name.
if docker container inspect dca31-checkpoint >/dev/null 2>&1; then
  echo 'Refusing to overwrite existing container dca31-checkpoint' >&2
  exit 1
fi

if docker image inspect alpine:3.22.1 >/dev/null 2>&1; then
  echo preexisting > "$EVIDENCE/image-preflight.txt"
else
  echo absent > "$EVIDENCE/image-preflight.txt"
fi

The lab uses public read-only image pulls and no credentials. If your environment requires an authenticated registry mirror, use a disposable training account and do not save credentials inside the evidence directory.

4. Prediction sheet: write before acting

cat > "$EVIDENCE/predictions.md" <<'EOF'
# Predictions
1. Pulling alpine:3.22.1 will create/update local image/content metadata but will not create a Docker container or running process.
2. docker create will create a container object with State.Status=created and PID=0; docker start will transition to a running task/process with a nonzero host PID.
3. The immutable registry/image digest should not change merely because a container starts.
4. If DriverStatus reports io.containerd.snapshotter.v1, Engine is using the containerd snapshotter-backed image store; if not, document the classic/other reported mode rather than forcing a change.
EOF
cat "$EVIDENCE/predictions.md"

If your host differs, do not rewrite the prediction after seeing the result. Record the mismatch and explain it.

5. Registry-side image evidence

docker buildx imagetools inspect alpine:3.22.1   | tee "$EVIDENCE/registry-index.txt"
docker buildx imagetools inspect alpine:3.22.1 --raw   > "$EVIDENCE/registry-index-raw.json"
sha256sum "$EVIDENCE/registry-index-raw.json"   | tee "$EVIDENCE/registry-index-raw.sha256"

This captures OCI index/manifest metadata before local rootfs state exists. Record which platform matches the daemon host.

6. Pull, identify, and record storage mode

docker pull alpine:3.22.1 | tee "$EVIDENCE/pull.txt"

docker image inspect alpine:3.22.1 > "$EVIDENCE/image-inspect.json"
docker image inspect alpine:3.22.1 \
  --format 'id={{.Id}} repoDigests={{json .RepoDigests}} platform={{.Os}}/{{.Architecture}}'   | tee "$EVIDENCE/image-summary.txt"

docker info --format '{{json .DriverStatus}}'   | tee "$EVIDENCE/driver-status.json"
docker info --format 'driver={{.Driver}} root={{.DockerRootDir}} runtime={{.DefaultRuntime}}'   | tee "$EVIDENCE/store-runtime-summary.txt"

Now verify prediction 1: docker ps -a --filter name=dca31-checkpoint should still show no lab container. The pull changed image/content state, not container/process state.

7. Create without starting: capture the object boundary

docker create \
  --name dca31-checkpoint \
  --label devops-academy.lab=chapter31-checkpoint   alpine:3.22.1   sh -c 'printf "checkpoint-runtime\n"; sleep 180'   | tee "$EVIDENCE/container-id.txt"

docker inspect dca31-checkpoint > "$EVIDENCE/container-created.json"
docker inspect dca31-checkpoint \
  --format 'status={{.State.Status}} pid={{.State.Pid}} image={{.Image}} runtime={{.HostConfig.Runtime}}'   | tee "$EVIDENCE/container-created-summary.txt"

Prediction 2 says PID should be zero before start. If your platform reports something different, preserve it and explain the product/version behavior instead of forcing expected output.

8. Start and trace runtime/process state

docker start dca31-checkpoint | tee "$EVIDENCE/start.txt"

docker inspect dca31-checkpoint > "$EVIDENCE/container-running.json"
docker inspect dca31-checkpoint \
  --format 'status={{.State.Status}} pid={{.State.Pid}} started={{.State.StartedAt}} image={{.Image}} runtime={{.HostConfig.Runtime}}'   | tee "$EVIDENCE/container-running-summary.txt"

docker top dca31-checkpoint -eo pid,ppid,user,args   | tee "$EVIDENCE/docker-top.txt"
docker logs --timestamps dca31-checkpoint   | tee "$EVIDENCE/logs.txt"

At this point the chain has crossed into task/shim/runc/kernel execution. The image digest is still artifact identity; the PID proves a current process instance only.

9. Capture component/runtime evidence

docker info --format 'default={{.DefaultRuntime}} runtimes={{json .Runtimes}}'   | tee "$EVIDENCE/runtimes.txt"
containerd --version 2>/dev/null | tee "$EVIDENCE/containerd-binary.txt" || true
runc --version 2>/dev/null | tee "$EVIDENCE/runc-binary.txt" || true

docker system df | tee "$EVIDENCE/system-df-after.txt"

Do not claim the optional binaries are the exact daemon-managed components unless you can prove that relationship. The evidence packet should say “host binary reported X” versus “Docker release packaging baseline Y.”

10. Optional Linux-only correlation with ctr

# OPTIONAL READ-ONLY DIAGNOSTIC; skip on Desktop/remote contexts.
sudo ctr --address /var/run/docker/containerd/containerd.sock   --namespace moby containers list   | tee "$EVIDENCE/ctr-containers.txt"

sudo ctr --address /var/run/docker/containerd/containerd.sock   --namespace moby tasks list   | tee "$EVIDENCE/ctr-tasks.txt"

Find the Docker container ID/task correlation if available. Stop there. The evidence packet must explicitly state: Docker Engine owns this namespace; no ctr mutation was performed.

11. Verify the predictions independently

cat > "$EVIDENCE/verification.md" <<'EOF'
# Verification
- Pull changed local image/content state: compare image-preflight, pull output, and image inspect.
- Create produced a Docker object before a running PID: compare container-created-summary with container-running-summary.
- Start produced runtime/process evidence: docker top + logs + StartedAt.
- Image identity remained artifact identity: compare .Image / RepoDigests before and after start.
- Store mode came from Docker info DriverStatus; no backend switch was performed.
- No Docker-managed containerd state was mutated through ctr.
EOF
cat "$EVIDENCE/verification.md"

12. Evidence packet manifest

13. Cleanup and rollback

docker rm -f dca31-checkpoint

# Respect preexisting image state.
if grep -qx absent "$EVIDENCE/image-preflight.txt"; then
  docker image rm alpine:3.22.1 || true
fi

docker system df | tee "$EVIDENCE/system-df-cleanup.txt"

Keep the evidence directory if you want an audit trail. The cleanup intentionally does not prune unrelated images, build cache, networks, volumes, or containerd content.

14. What Chapter 31 adds to the production operating model

You can now trace Docker’s high-level object model into its Engine 29-era storage/runtime internals without confusing implementation state for a supported operator API. That gives production runbooks a sharper rule: verify immutable OCI identity at the registry/image layer, verify materialization at the snapshot layer, verify execution at the task/shim/runtime layer, and preserve Docker as the owner of its managed containerd state.

Knowledge check

Official references and version notes

• Docker Docs — containerd image store with Docker Engine — Engine 29 fresh-install defaults, snapshotters, disk layout, switching, and experimental migration guidance.
• Docker Docs — Storage drivers — distinction between classic graph drivers and the Engine 29 containerd image store.
• Docker Docs — Select a storage driver — current storage-backend matrix and platform notes.
• Docker Docs — Docker daemon configuration overview — /var/lib/docker versus /var/lib/containerd and data-root implications.
• Docker Docs — Run containerd in the Docker daemon — Engine 29.7+ experimental embedded-containerd mode and debugging endpoint warning.
• Docker Engine 29 release notes — Engine 29.8.1 baseline, containerd 2.3.5 static-binary packaging, BuildKit 0.33.0 and runc 1.5.1 updates.
• containerd 2.3 — Features — namespaces, images, root filesystems, snapshots, containers, tasks, and OCI runtime integration.
• containerd — Snapshotters — core snapshotter behavior and the overlayfs naming used by containerd.
• OCI Image Specification 1.1.1 — image indexes, manifests, configs, layers, and content-addressable descriptors.
• OCI Runtime Specification 1.3.0 — runtime bundle, config.json, execution environment, and lifecycle model.