Container Filesystems, Writable Layers, docker cp, exec Workflows, Processes, Signals, and PID 1: Guided Hands-On Workflow and Core Operations

1. Lab scope and ownership

This lab uses a Docker Official Image, resolves its current repository digest, and creates only chapter-owned resources. The main container is intentionally retained until the evidence packet is complete so runtime filesystem changes can be inspected before cleanup.

2. Preflight: context, versions, storage mode, and exact image identity

mkdir -p ch06-lab/evidence
cd ch06-lab

docker context show | tee evidence/00-context.txt
docker version | tee evidence/01-version.txt
docker info | tee evidence/02-info.txt

SOURCE='busybox:1.37.0'
docker pull "$SOURCE"
PINNED=$(docker image inspect "$SOURCE" --format '{{index .RepoDigests 0}}')
printf 'source=%s\npinned=%s\n' "$SOURCE" "$PINNED" | tee evidence/03-image.txt

The tag is used only as a human-readable source to resolve an immutable repository digest. All containers in the lab use $PINNED. The full docker info output is retained because Engine 29 fresh installs commonly use the containerd image store, while upgraded systems can differ.

3. Create the runtime baseline before mutation

MAIN='devops-academy-ch06-main'

docker run -d \
  --name "$MAIN" \
  --label devops-academy.lab=ch06 \
  "$PINNED" \
  sh -c 'trap "echo received-TERM; exit 0" TERM; mkdir -p /academy; echo image-runtime-baseline > /academy/runtime.txt; while :; do sleep 1; done'

docker inspect "$MAIN" --format 'ID={{.Id}} Image={{.Image}} Status={{.State.Status}} Pid={{.State.Pid}}' | tee evidence/04-container.txt
docker inspect "$MAIN" --format '{{json .Mounts}}' | tee evidence/05-mounts.json
docker top "$MAIN" | tee evidence/06-top.txt
docker diff "$MAIN" | tee evidence/07-diff-baseline.txt

The command deliberately creates /academy/runtime.txt at runtime, so the first diff is already instructive. Other paths may also appear because BusyBox or Docker creates runtime metadata. Do not infer intent from a diff line alone.

4. Copy a harmless file in, then copy evidence out

printf 'copied-at=%s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" > local-note.txt

docker cp ./local-note.txt "$MAIN":/academy/copied-note.txt
docker diff "$MAIN" | tee evidence/08-diff-after-cp.txt

docker cp "$MAIN":/academy/copied-note.txt ./roundtrip-note.txt
sha256sum local-note.txt roundtrip-note.txt | tee evidence/09-copy-hashes.txt
ls -ln local-note.txt roundtrip-note.txt | tee evidence/10-copy-ownership.txt

The matching content hash proves the bytes round-tripped. Ownership may differ because docker cp applies documented source/destination ownership rules. Do not use this exercise to copy secrets. Also note that docker cp can work with stopped containers, which is useful when preserving non-secret evidence after a process has exited.

5. Use exec for a bounded diagnostic action

docker exec "$MAIN" sh -c 'printf "exec-created\n" > /academy/exec-note.txt; echo exec-pid=$$; cat /academy/runtime.txt'
docker diff "$MAIN" | tee evidence/11-diff-after-exec.txt
docker top "$MAIN" | tee evidence/12-top-after-exec.txt

The exec process ends, but the file it wrote remains in this container's writable layer. That is the distinction: process lifetime and filesystem lifetime are separate, and neither changes the underlying image. If the container is replaced, the exec-created file disappears.

6. Observe PID 1 and child processes

docker exec "$MAIN" sh -c '
  echo "inside namespace:";
  ps;
  echo "PID1 cmdline:";
  tr "\\0" " " < /proc/1/cmdline; echo
' | tee evidence/13-proc.txt

The exact process listing depends on the image and platform. On Linux containers, /proc/1 describes the process Docker treats as the primary process inside the PID namespace. On Docker Desktop, host-side PID mapping crosses the Desktop VM boundary, so the course records both Docker inspect PID evidence and in-container PID evidence instead of assuming they are directly interchangeable.

7. Prove graceful signal handling before changing the image

docker stop --timeout 5 "$MAIN"
docker inspect "$MAIN" --format 'Status={{.State.Status}} Exit={{.State.ExitCode}} Finished={{.State.FinishedAt}}' | tee evidence/14-stopped.txt
docker logs "$MAIN" | tee evidence/15-logs.txt

Because the primary shell installed a TERM trap, the logs should contain received-TERM and the process should exit without needing timeout escalation. If your evidence differs, preserve it and diagnose rather than forcing the expected result.

While the container is stopped, prove that file-copy and exec have different requirements:

docker cp "$MAIN":/academy/exec-note.txt ./evidence/exec-note-from-stopped.txt

docker exec "$MAIN" sh -c 'echo impossible-while-stopped' 2> evidence/16-exec-stopped-error.txt || true
cat evidence/16-exec-stopped-error.txt

docker cp can read the stopped container filesystem; docker exec requires the primary process to be running.

8. Optional Linux-container experiment: inspect --init

This experiment is read-only except for creating one disposable container. It demonstrates ownership, not a requirement that every container use an init helper.

INIT='devops-academy-ch06-init'
docker run -d --init \
  --name "$INIT" \
  --label devops-academy.lab=ch06 \
  "$PINNED" sh -c 'while :; do sleep 1; done'

docker exec "$INIT" sh -c 'ps; echo; tr "\\0" " " < /proc/1/cmdline; echo' | tee evidence/17-init-proc.txt
docker inspect "$INIT" --format 'Init={{.HostConfig.Init}} Status={{.State.Status}}' | tee evidence/18-init-inspect.txt

docker stop --timeout 5 "$INIT"
docker rm "$INIT"

Current Docker documentation states that --init inserts the daemon's docker-init, backed by Tini in the default installation, to perform init responsibilities such as reaping zombie processes. Use it when those responsibilities are needed; do not add it blindly as a substitute for understanding the application process model.

9. Convert the intended runtime file into declared image state

Now take the intent behind the manual copy—“every container should contain a known training note”—and encode it in an image. Chapter 07 explains Dockerfile instructions in depth; this lab uses the smallest possible build only to demonstrate durability.

printf 'declared-by-image\n' > declared-note.txt
cat > Dockerfile <<'EOF'
ARG BASE_IMAGE
FROM ${BASE_IMAGE}
COPY declared-note.txt /academy/declared-note.txt
CMD ["sh", "-c", "trap 'echo received-TERM; exit 0' TERM; while :; do sleep 1; done"]
EOF

docker build \
  --label devops-academy.lab=ch06 \
  --build-arg BASE_IMAGE="$PINNED" \
  -t devops-academy-ch06:declared . | tee evidence/19-build.txt

docker image inspect devops-academy-ch06:declared --format 'ImageID={{.Id}} BaseLabel={{index .Config.Labels "devops-academy.lab"}}' | tee evidence/20-built-image.txt

The build input pins the base by digest. The resulting local image has its own immutable image ID. Nothing from copied-note.txt or exec-note.txt is included unless explicitly declared in the build context and Dockerfile.

10. Replace the mutated container and prove declared state

docker rm "$MAIN"

docker run -d \
  --name devops-academy-ch06-replacement \
  --label devops-academy.lab=ch06 \
  devops-academy-ch06:declared

docker exec devops-academy-ch06-replacement sh -c '
  echo "declared:"; cat /academy/declared-note.txt;
  echo "manual-copy:"; test -e /academy/copied-note.txt && echo present || echo absent;
  echo "exec-hotfix:"; test -e /academy/exec-note.txt && echo present || echo absent
' | tee evidence/21-replacement-proof.txt

docker diff devops-academy-ch06-replacement | tee evidence/22-replacement-diff.txt

The replacement starts with the declared note but not the copied/exec files. This is the central proof of the chapter: runtime mutation can validate a hypothesis, but only declared image/configuration state reproduces across replacement.

11. Guarded cleanup

docker ps -a --filter label=devops-academy.lab=ch06 --format 'table {{.ID}}\t{{.Names}}\t{{.Status}}'

# Remove only the exact remaining lab container after reviewing the listing.
docker stop --timeout 5 devops-academy-ch06-replacement
docker rm devops-academy-ch06-replacement

# Remove only the local lab image created by this chapter.
docker image rm devops-academy-ch06:declared

Do not prune broadly and do not remove the shared BusyBox source image merely to “clean everything.” The evidence directory is intentionally retained for review.

12. Small challenge: choose the correct layer

A teammate discovers that a production-like container is missing /etc/widget/policy.json. They propose docker cp policy.json container:/etc/widget/. Before doing anything, identify what you would inspect to determine whether the path is image content, hidden by a mount, intentionally generated at runtime, or genuinely absent. Then state how you would test the hypothesis in a disposable container and how the durable fix would be represented.

Knowledge check

Official references and version notes

• Docker storage overview — explains the ephemeral per-container writable layer and why persistent data belongs in explicit mounts rather than the container layer.
• Storage drivers and writable layers — copy-on-write concepts, writable-layer behavior, and the Engine 29 distinction between the containerd image store and classic storage-driver examples.
• docker container diff — reports added, changed, and deleted paths in the container filesystem relative to its initial image-backed state.
• docker container cp — current copy semantics, stopped-container support, destination ownership, archive mode, symlink handling, and path rules.
• docker container exec — starts a new command only while the container's primary PID 1 is running and does not make image changes durable.
• docker container run — current runtime options including --init, stop signals/timeouts, mounts, TTY behavior, and security controls.
• docker container stop — graceful signal delivery and timeout-to-SIGKILL escalation.
• docker container attach — documents special PID 1 signal behavior and terminal signal-proxy implications.
• JSONArgsRecommended build check — why exec-form CMD/ENTRYPOINT avoids an unintended shell parent and improves signal handling.
• Docker build best practices — entrypoint scripts should normally exec the final application so it becomes PID 1 and receives signals directly.
• Bind mounts — mounting over an existing container path obscures image/writable-layer content until the container is recreated without that mount.
• Docker Engine 29 release notes — Engine 29.8.1 is the current patch baseline at verification time; recent 29.x releases also include multiple docker cp security and compatibility fixes.