Container Filesystems, Writable Layers, docker cp, exec Workflows, Processes, Signals, and PID 1: Configuration, Design Choices, and Tradeoffs

1. Design frame: decide ownership before choosing a command

The fastest command is not always the safest design. Before choosing COPY, a volume, bind mount, docker cp, docker exec, shell-form startup, exec-form startup, or --init, ask four questions: who owns the state, how long must it live, who must reproduce it, and which process must receive lifecycle signals?

These decisions affect portability, rollback, incident evidence, host coupling, security exposure, and performance. The chapter keeps the choices bounded; persistent-storage engineering is covered later in Chapters 20–21.

2. Bake into the image, mount, or copy at runtime?

A manual copy that fixes production behavior is a clue that declared state is incomplete, not proof that docker cp should become the deployment mechanism.

3. Writable-layer convenience has lifecycle and performance costs

Docker's storage guidance recommends keeping write-heavy or durable data outside the container writable layer. Copy-on-write can add overhead when a lower-layer file must be copied up before modification, and deleting the container deletes its writable layer. The exact implementation depends on the daemon's storage backend; Engine 29 fresh installs default to the containerd image store, while upgraded hosts may still expose classic storage-driver behavior.

Therefore performance advice should be evidence-based: inspect actual storage mode, measure the workload, and use a volume for durable/write-heavy data rather than assuming every filesystem issue is an overlay2 issue.

4. Mount precedence: visible does not mean stored there

If a mount targets a non-empty directory, its contents obscure what the image or writable layer had at that path. This is useful for configuration/data injection, but it complicates diagnosis. A missing visible file may still exist underneath the mount.

Do not copy a file “into” a mounted path until you know which storage object will actually receive the write.

5. Exec form versus shell form: convenience changes signal ownership

Shell form is convenient when you need shell expansion or pipelines, but it inserts an implicit shell. Docker's current build check warns that shell-form CMD/ENTRYPOINT can prevent the intended application from receiving OS signals. Exec form launches the executable directly and makes process identity clearer.

6. Application PID 1 versus an init helper

A well-designed application can be PID 1 if it handles its signals and child processes. Adding a tiny init is valuable when the workload spawns children that can become orphaned/zombie processes or when signal forwarding needs an init boundary. Docker's --init option uses the daemon's Tini-backed docker-init by default.

Do not use a full systemd-style init merely because the container concept resembles a VM. A container usually has one main service concern. Add the smallest process-management mechanism justified by measured behavior.

7. Runtime debugging versus immutable rebuild

docker exec is strong for inspection, health probes during an incident, or a narrowly authorized administrative command. It is weak as a deployment mechanism because the action is not encoded in the image source and may be lost on replacement.

A robust workflow is:

1. Preserve the first-failure evidence.
2. Use a disposable reproduction or bounded exec/copy action to test a hypothesis.
3. Record exactly what was changed and why.
4. Express the intended fix in Dockerfile/source/configuration.
5. Build a new image from reviewed inputs.
6. Verify the new image identity and behavior.
7. Replace the old container; do not “promote” the hot-fixed writable layer.

8. Secret boundary: neither writable layer nor docker cp is a secret store

Copying a credential into a container creates sensitive data in runtime filesystem state and may leave it inspectable or recoverable longer than intended. Embedding the same credential in an image is worse because it can persist in image layers and caches. Later chapters cover Docker secrets/configuration patterns in depth.

For this chapter, the rule is simple: labs use fake values only, and docker cp examples transfer harmless text. In production design, use an explicit secret-management mechanism with narrow access and lifecycle rather than ad hoc file copying.

9. Worked decision: a service needs config, scratch space, and graceful shutdown

Suppose a service ships a default policy, receives environment-specific configuration, writes transient render output, and spawns helper processes. A defensible design might be:

• Bake the default policy and binary into the image.
• Inject environment-specific non-secret config through a read-only configuration mount or platform mechanism.
• Use tmpfs or a bounded writable path for non-durable scratch output.
• Launch the application in exec form.
• Add --init only if child reaping is genuinely needed.
• Set and test a stop timeout that exceeds measured drain time.
• Use exec/cp for diagnosis only, then rebuild any intended release change.

Evidence should include image digest, mount ownership, writable-delta expectations, docker diff output where relevant, PID 1 command, child-process behavior, stop signal/timeout, shutdown logs, and replacement test.

10. Platform and version prerequisites

Linux containers expose the PID 1 and Unix-signal model discussed here. Windows containers have different process and signal semantics. Docker Desktop adds a managed VM boundary for Linux containers on macOS/Windows, so host PID/filesystem assumptions should not be transferred blindly.

docker cp behavior and security have also changed across Engine releases. Engine 29.5.1 fixed multiple high-severity copy-path issues; 29.7.x fixed additional symlink/kernel compatibility regressions. Record the actual Engine patch level in runbooks and do not rely on an old copy workaround as current guidance.

11. Design challenge

You inherit a container that starts with shell-form CMD, writes 2 GB/day into /var/lib/widget, receives config by docker cp, and is routinely “fixed” with docker exec vi .... Propose a migration that separates release files, durable data, configuration, runtime scratch, process launch, and diagnostic actions. For every change, state the evidence proving the new behavior and the rollback boundary.

Knowledge check

Official references and version notes

• Docker storage overview — explains the ephemeral per-container writable layer and why persistent data belongs in explicit mounts rather than the container layer.
• Storage drivers and writable layers — copy-on-write concepts, writable-layer behavior, and the Engine 29 distinction between the containerd image store and classic storage-driver examples.
• docker container diff — reports added, changed, and deleted paths in the container filesystem relative to its initial image-backed state.
• docker container cp — current copy semantics, stopped-container support, destination ownership, archive mode, symlink handling, and path rules.
• docker container exec — starts a new command only while the container's primary PID 1 is running and does not make image changes durable.
• docker container run — current runtime options including --init, stop signals/timeouts, mounts, TTY behavior, and security controls.
• docker container stop — graceful signal delivery and timeout-to-SIGKILL escalation.
• docker container attach — documents special PID 1 signal behavior and terminal signal-proxy implications.
• JSONArgsRecommended build check — why exec-form CMD/ENTRYPOINT avoids an unintended shell parent and improves signal handling.
• Docker build best practices — entrypoint scripts should normally exec the final application so it becomes PID 1 and receives signals directly.
• Bind mounts — mounting over an existing container path obscures image/writable-layer content until the container is recreated without that mount.
• Docker Engine 29 release notes — Engine 29.8.1 is the current patch baseline at verification time; recent 29.x releases also include multiple docker cp security and compatibility fixes.