Production Container Patterns, Immutable Delivery, Configuration, Statelessness, Sidecars, and Operational Contracts: Concepts, Architecture, and Mental Model
Turn Docker features into an explicit production container contract covering immutable artifacts, external configuration, durable state, identity, health, resources, observability, shutdown, recovery, and companion services.
Learning objectives
- Define a production container contract that separates immutable image identity, runtime configuration, durable state, process identity, health, resources, logs, shutdown, and recovery.
- Explain why a healthy container, a reachable application, durable data, and a correctly verified release digest are different states.
- Distinguish stateless application design from stateful service requirements and keep durable state outside the container writable layer.
- Explain what a companion/sidecar-like service should own—and what hidden coupling makes it unsafe or brittle.
- Inspect the complete runtime contract before changing any production-like object.
1. The practical problem: features are not an operating contract
Docker can start a process, mount a volume, publish a port, run a healthcheck, rotate logs, enforce a memory ceiling, and restart an exited process. None of those features alone tells an operator what the workload is supposed to be. A production container contract connects them: which exact image bytes, which configuration and secret sources, which user, which writable paths, which network exposure, which health semantics, which resource envelope, which log retention, and what replacement/recovery should preserve.
Chapters 13, 20, 23–30, and 36 established those individual pieces. Chapter 38 combines them into one reviewable runtime contract.
2. Mental model: immutable artifact plus external runtime state
flowchart TD
A[Release image digest] --> B[Runtime contract]
C[Config / secret source] --> B
D[Declared volume + networks + resources] --> B
B --> E[Container process as intended user]
E --> F[Health + traffic]
E --> G[stdout/stderr + metrics]
E --> H[Persistent writes only to declared data]
F --> I[Graceful stop or process failure]
I --> J[Replacement / restart]
H --> J
J --> K[Verify same contract + durable state]
L[Companion service] --> F
L --> G
The image digest is the immutable executable artifact. Configuration and secrets enter at runtime through explicit sources. Volumes carry only intended durable data; tmpfs or the writable layer carry ephemeral state. Networks and published ports define reachability. Resource, security, logging, health, stop and restart settings define how the process is allowed to behave. Replacement succeeds only when the new process receives the same required external state and independently passes health/application checks.
3. State domains to inspect before any change
| Domain | Production question | Evidence |
|---|---|---|
| Release identity | Which exact image/index digest is intended? | Compose image ref, registry digest, image inspect |
| Runtime identity | Which UID/GID and security restrictions execute it? | container inspect, id, security options |
| Configuration | Which non-secret values shape behavior? | versioned config file/config object; rendered Compose |
| Secret | Which credential source and grant are used? | secret definition/grant without printing value |
| Durable state | Which data must survive replacement? | volume identity, mount target, checksum/backup evidence |
| Ephemeral state | Where may temporary files exist? | tmpfs/writable-path inventory |
| Network | Which internal service names and host ports exist? | network inspect, published bind address |
| Health | What probe means the process is ready? | healthcheck definition/history plus external request |
| Resources | What CPU/memory/PID envelope is intended? | HostConfig/cgroup evidence and stats |
| Observability | Where do stdout/stderr and metrics go? | logging driver/options, bounded logs, stats |
| Shutdown | Which signal and grace period protect state? | stop signal/timeout and application log |
| Recovery | What restarts automatically and what needs orchestration? | restart policy/count, runbook and event trail |
| Companion | Why does another container exist and what may it access? | service contract, network/mount/secret grants |
4. Immutable delivery means replacement, not patching
An immutable production image is changed by rebuilding and replacing it, not by installing packages or editing code inside a running container. A shell session can be useful for read-only diagnosis, but any emergency mutation creates drift that disappears at replacement and is difficult to reproduce. The corrective artifact should be a new image digest produced from declared source and build inputs.
Tags remain useful human aliases, but the deploy decision should be
bound to a digest or to recorded digest resolution. “Deploy
release-42” is incomplete evidence unless you also know
which digest that tag resolved to at the decision point.
5. Stateless does not mean “never writes”
A stateless application may still write caches, temporary files, sockets, or generated runtime state. The production distinction is that the instance does not own irreplaceable business state in its disposable writable layer. Ephemeral data belongs in tmpfs or replaceable caches; durable data belongs in a declared volume or external service with a backup/recovery contract.
6. Container writable layer versus declared state
| Path type | Lifecycle | Production interpretation |
|---|---|---|
| Image filesystem | immutable image layers | release artifact; never runtime patch target |
| Writable layer | container lifetime | ephemeral only; replacement destroys it |
| Named volume | independent of container | durable data with explicit retention/backup |
| tmpfs | memory-backed runtime lifetime | ephemeral and non-persistent |
| Bind mount | host-coupled lifecycle | use only when host ownership/path coupling is intentional |
7. Identity and filesystem contract
Run as the least-privileged user that can perform the application’s
job. A read-only root filesystem is valuable only when the
application’s legitimate writable paths are separately declared.
Combine non-root execution, dropped capabilities,
no-new-privileges, read-only rootfs, and narrow
writable mounts rather than reaching for broad authority when one
path fails.
8. Health, reachability, and availability are different
A Docker healthcheck runs inside the container and reports a local probe result. It can prove that an application endpoint or dependency check succeeded from that namespace at that moment. It cannot by itself prove DNS, firewall, load-balancer, TLS, upstream dependency, regional, or user-facing availability. Production evidence therefore keeps the health state and an external/request-path check separate.
9. Restart policy is not orchestration
Engine restart policy reacts to the container process exiting. It
does not reschedule work to another host, fix a full disk, repair a
failed database, or restart a merely unhealthy process.
On a single Compose host it is a useful local recovery mechanism;
broader high availability requires an orchestrator or another
explicit host-level availability design.
10. Resources and logs must be bounded
Containers have no CPU or memory limit by default. A production
contract should choose limits from measurement, not from folklore,
and then retain stats/cgroup evidence to show whether
throttling or OOM caused a failure. Logging also needs a retention
boundary. The per-service local driver is a convenient
lab choice because it rotates by default and supports explicit
max-size/max-file without changing the
daemon globally.
11. Companion/sidecar-like services need a narrow purpose
A companion container can be useful for a local proxy, telemetry forwarder, file converter, or health observer. Its contract must state why it exists, what network/mount/secret access it receives, how it behaves when the main app is absent, and whether the two lifecycles can change independently. Sharing the Docker socket, host namespaces, broad writable volumes, or all application credentials is not a sidecar pattern—it is a privilege expansion.
12. Compose production boundary
Current Docker documentation supports Compose for production deployments on a single host. That makes it appropriate for this chapter’s local operating-contract lab. Compose does not turn one Engine into a multi-host scheduler; Chapter 39 covers Swarm service semantics and later courses cover other orchestrators. Treat single-host replacement and restart as the scope of this chapter.
13. Read-only preflight
docker context show
docker version
docker compose version
docker buildx version
docker info --format 'os={{.OperatingSystem}} logging={{.LoggingDriver}} driver={{.Driver}} cgroup={{.CgroupVersion}}'
docker ps --format 'table {{.ID}} {{.Image}} {{.Status}} {{.Names}}'
docker volume ls
docker network ls
Inspection is intentionally broad only in visibility, not mutation. Before touching a production-like project, narrow all later actions by exact project name, labels, IDs, and digest references.
14. Production evidence packet
At minimum retain: source/release identity, resolved image digest, rendered runtime configuration, runtime user/security posture, mount and volume identities, network/publication evidence, health history, external request result, resource limits plus a stats sample, logging driver/rotation settings, shutdown evidence, restart count/events, companion-service grants, and backup/recovery assumptions.
Knowledge check
Is a container that reports healthy necessarily
externally available?
No. Health is a local container state; external DNS, firewall, TLS, routing and dependencies can still fail.
What happens to business data written only to the container writable layer when the container is replaced?
It is lost with that container layer. Durable state must be externalized to a volume or external state service.
Does restart: unless-stopped create high
availability across hosts?
No. It restarts a container on the same Engine; it does not reschedule to another host.
Why pin the deployed release by digest even if humans use a release tag?
A tag can move. The digest identifies the exact immutable content selected for deployment.
What makes a companion service acceptable?
A clear purpose, minimal network/mount/secret authority, explicit failure/lifecycle behavior, and independent observable identity.
Official references and version notes
2026-09-22. Course baseline: Docker Engine/CLI 29.8.1, Compose 5.5.1, Buildx 0.37.1, BuildKit 0.33.0. Packaged containerd/runc versions vary by installation, so executable labs always record the learner’s actual versions. Compose production guidance in this lesson is explicitly single-host.
- Docker Docs — Use Compose in production — single-host production use, production-specific overrides, and service recreation.
- Docker Docs — Why use Compose? — current single-host deployment boundary and application-model use cases.
- Docker Docs — Compose services reference — healthcheck, restart, read-only rootfs, resource limits, logging, configs, secrets, stop signal and grace period.
- Docker Docs — Compose Deploy Specification — resource limits/reservations and deployment-oriented service controls.
- Docker Docs — Start containers automatically — restart-policy semantics and the successful-start threshold.
- Docker Docs — Resource constraints — CPU/memory governance and OOM implications.
- Docker Docs — Volumes — persistent data lifecycle independent of containers.
- Docker Docs — Storage — writable-layer versus volume/tmpfs durability boundaries.
- Docker Docs — Manage secrets securely in Compose — file-mounted secret access and environment-variable risk.
- Docker Docs — Compose secrets reference — top-level secret sources and service grants.
- Docker Docs — Configure logging drivers — default json-file behavior and recommendation for bounded local logging.
- Docker Docs — Local logging driver — automatic rotation, max-size/max-file, and daemon-owned log files.
- Docker Engine 29 release notes — current Engine-era baseline; 29.8.1 released 2026-09-15.
- Docker Compose releases — current Compose 5.5.1 baseline used for version-sensitive examples.
- Docker Buildx releases — current Buildx 0.37.1 baseline.
- BuildKit releases — current BuildKit 0.33.0 baseline.
- Docker Official Image — registry — current Distribution Registry 3.1.1 local-registry image used in the optional digest-pinning lab path.
Keep the academy open
Support free, practical DevOps education.
Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0Send only Ethereum/ERC-20 compatible assets to this
address.