Containers, Virtual Machines, Linux Isolation, OCI Standards, and Docker Architecture: Configuration, Design Choices, and Tradeoffs
Choose container boundaries deliberately: compare containers with VMs, Docker Engine with lower-level runtime APIs, rootful with rootless operation, native Linux with Docker Desktop, and convenience with stronger isolation for untrusted workloads.
Learning objectives
- Select containers or VMs based on kernel, threat-model, portability, density, and failure-domain requirements.
- Explain why Docker Engine, containerd, and OCI runtimes are different lifecycle/control boundaries.
- Distinguish rootful Docker, userns-remap, rootless Docker, and non-root application users.
- Identify host-assumption differences between native Linux Engine and Docker Desktop.
- Document trust, reproducibility, isolation, portability, and recovery tradeoffs with observable evidence.
1. Architecture choices begin with the boundary you need
“Use containers” is not a complete design decision. You still need to decide where the daemon runs, how much privilege it has, whether workloads are mutually trusted, whether VM separation is required, how portable OCI artifacts must be, and which component owns lifecycle. This lesson compares those choices without pretending one mode is universally superior.
2. Container versus VM: optimize for isolation requirements, not slogans
Containers are excellent for packaging userspace and creating fast, repeatable process environments. VMs are useful when you need a separate guest kernel, stronger kernel fault separation, or a fundamentally different operating-system boundary. Many production platforms use both: containers for application packaging inside VM or bare-metal worker nodes.
| Requirement | Container-first fit | When a VM/stronger sandbox deserves consideration |
|---|---|---|
| Fast immutable application packaging | Strong fit | VM may add unnecessary guest-OS overhead. |
| Different kernel family/version per workload | Ordinary Linux containers cannot provide this | VM is the natural boundary. |
| Mutually trusted internal services | Common container use case with layered hardening | VM still useful for node/tenant failure domains. |
| Hostile multi-tenant arbitrary code | Ordinary containers alone may be insufficient | Consider VM-backed/sandboxed runtimes and stronger tenant isolation. |
| Desktop development | Docker Desktop supplies container UX | Desktop itself may use a VM under the hood. |
The decision is about threat model, kernel requirements, operational cost, startup/density goals, and recovery—not about whether one technology is “modern.”
3. Docker Engine versus lower-level runtime APIs
Docker Engine provides a cohesive API and UX for images, containers, networks, volumes, logs, build integration, and administration. containerd and OCI runtimes expose lower-level boundaries used by platforms and engines. Application teams should not bypass Docker's supported Engine API merely because they discover an internal containerd socket.
Current Docker documentation explicitly warns that its managed/embedded containerd endpoint is daemon-owned state and not a general-purpose integration endpoint. Mutating it with external clients can conflict with Docker's expectations. The design principle is simple: use the highest supported API that owns the lifecycle you intend to manage.
Use when Docker owns the application container object and associated networks, volumes, logs, and policy.
Use when you intentionally operate a containerd-native platform and own its namespaces/content/runtime model.
Low-level execution boundary; not a replacement for Docker's image distribution, networks, volumes, or higher-level lifecycle.
4. Rootful and rootless Docker reduce different operational risks
Traditional Docker Engine runs the daemon with root privileges and
exposes its Unix socket to authorized clients. Docker's rootless
mode runs both daemon and containers without root privileges inside
a user namespace, reducing the impact of daemon/runtime
vulnerabilities. This is different from merely putting
USER 1000 in an image, and it is different from
userns-remap, where the daemon remains rootful while
container UIDs/GIDs are remapped.
| Mode | Daemon privilege | Container identity mapping | Main tradeoff |
|---|---|---|---|
| Rootful default | Root | Container root has namespaced/restricted privileges but daemon is powerful | Broad compatibility; socket access is root-equivalent privilege. |
userns-remap |
Root | Container IDs map to subordinate host IDs | Reduces host identity impact but introduces filesystem/device compatibility considerations. |
| Rootless | Non-root | Daemon and containers operate inside a user namespace | Stronger privilege reduction with documented networking/storage/resource limitations. |
Non-root app USER |
Depends on daemon mode | Application process uses non-root UID inside container | Important least-privilege control, but not equivalent to rootless daemon operation. |
For this chapter, rootless is a conceptual choice only; do not migrate an existing host as a lab. Chapter 26 gives it a controlled setup and compatibility workflow.
5. Native Linux Engine and Docker Desktop have different host boundaries
On native Linux Engine, the daemon host is the Linux machine you administer. Host PID, namespace, cgroup, storage, firewall, and socket evidence can therefore be inspected directly. Docker Desktop inserts a managed Linux VM boundary. That improves desktop portability but changes path resolution, networking, filesystem sharing, process visibility, and performance interpretation.
/proc/<pid>, /sys/fs/cgroup,
/var/lib/docker, systemd, or host iptables exists on
the client machine is a Linux-host administration command—not a
portable Docker CLI operation. Mark it as such.
6. Convenience versus stronger isolation for untrusted workloads
Giving a workload --privileged, broad devices, host
PID/network namespaces, sensitive bind mounts, or the Docker socket
collapses parts of the isolation model. Those settings can be
legitimate for narrowly reviewed system workloads, but they are not
debugging conveniences.
When code is untrusted, increase isolation rather than relaxing it: use separate hosts/VMs or dedicated runner pools, rootless/user-namespace strategies where compatible, minimal capabilities, default seccomp/LSM enforcement, read-only filesystems, bounded mounts, and no daemon socket. The specific control depends on the threat model; the direction should be toward fewer privileges.
7. Worked decision table
| Scenario | Preferred starting point | Evidence/prerequisites | Why |
|---|---|---|---|
| Developer builds trusted service locally on Windows | Docker Desktop Linux containers | Desktop version, active context, VM resources, image digest | Portable developer UX; VM boundary is explicit. |
| Single-team Linux CI host, trusted repositories | Dedicated Linux Engine host or ephemeral VM runner | Daemon/socket owner, runner trust, cleanup, cgroup/security state | Native observability with a bounded trust domain. |
| Public fork code in CI | Ephemeral isolated builders without production daemon/socket credentials | Tenant boundary, credential policy, image/cache trust | Repository code must not inherit host-equivalent Docker control. |
| Need a different kernel per workload | VM or VM-backed sandbox | Guest kernel/version and hypervisor boundary | Ordinary containers share the daemon host kernel. |
| Personal lab wants reduced daemon privilege | Rootless Docker if host prerequisites fit | User namespaces, subordinate IDs, networking/storage limitations | Reduces daemon/container root privilege exposure. |
8. Evaluate five operational dimensions
For any architecture choice, document at least these dimensions:
- Trust: who can submit container configuration or code, and what host/registry/secret privileges follow?
- Reproducibility: can you identify the exact image digest, daemon/runtime baseline, and configuration?
- Failure isolation: does a kernel/runtime/daemon failure affect one container, one host, one VM, or many tenants?
- Portability: which assumptions are Linux-host-specific, Desktop-specific, architecture-specific, or runtime-specific?
- Recovery: which state must be rebuilt from source, restored from backup, or reconciled externally?
This turns a “Docker preference” into an auditable engineering choice.
9. Design lab — classify your current environment without changing it
Run only read-only inspection, then complete the worksheet below.
docker context show
docker version
docker info
docker info --format '{{json .SecurityOptions}}' 2>/dev/null || true
docker info --format 'cgroup-driver={{.CgroupDriver}} cgroup-version={{.CgroupVersion}}' 2>/dev/null || true
- Native Linux Engine, Docker Desktop, rootless Engine, or remote context?
- Who is authorized to access the daemon?
- Are workloads mutually trusted?
- What is the failure domain if the daemon host is compromised?
- Which evidence proves your answer rather than merely assuming it?
10. Common wrong approaches
- “Root inside a container means root on the host.” Not automatically; namespaces/capabilities/user mappings matter. But a rootful daemon or dangerous mounts/capabilities can still create severe host risk.
- “Rootless makes every container safe.” No. It reduces daemon/container privilege but does not remove application vulnerabilities, secret exposure, network risk, or resource abuse.
- “Docker Desktop is just Docker Engine installed directly on my laptop OS.” Not for Linux containers; Desktop uses a managed Linux environment/VM boundary.
- “If OCI defines containers, all engines have identical behavior.” OCI defines key portable contracts, not every networking, storage, build, security, or management behavior.
- “I can manage Docker's internal containerd directly because it has a socket.” Docker documents that daemon-owned endpoint as an implementation/debugging surface, not a general-purpose integration contract.
Knowledge check
A workload needs a custom Linux kernel module and a kernel version different from the host. Is an ordinary Linux container the natural boundary?
No. Ordinary Linux containers share the daemon host kernel. A VM or other environment with its own guest kernel is a more appropriate starting point.
What is the key difference between rootless Docker and
userns-remap?
In rootless mode the Docker daemon itself runs without root
privileges inside a user namespace. With
userns-remap, the daemon remains rootful while
container user IDs are remapped.
Why should application automation prefer the Docker Engine API over Docker's managed containerd socket?
Because Docker owns the lifecycle and state behind that internal containerd integration. The Engine API is the supported Docker control boundary; external mutations of daemon-owned containerd state can conflict with Docker.
Does Docker Desktop invalidate the statement that Linux containers share a kernel?
No. They share the kernel of the Docker Desktop Linux VM. The VM simply means that kernel is not the physical desktop OS kernel.
What is wrong with choosing --privileged simply
because a command returned “operation not permitted”?
It bypasses multiple isolation controls without proving which privilege was required. Diagnose the denied operation and grant the narrowest justified capability or redesign the workload instead.
Summary
Container architecture is a set of boundary choices. Containers and VMs isolate at different layers; Docker Engine and low-level runtimes own different lifecycle contracts; rootful, userns-remap, rootless, and non-root application users solve different privilege problems; Desktop introduces a VM boundary; and untrusted workloads deserve stronger isolation rather than convenient privilege escalation. A sound design states its trust model, platform assumptions, observable evidence, and recovery boundary.
Official references and version notes
Version-sensitive statements in this chapter were rechecked against primary documentation on 2026-09-20. Docker Engine, Docker Desktop, containerd, runc, OCI specifications, security defaults, and platform integration continue to evolve. Record the versions actually reported by your own environment before treating an example as production policy.
- Docker documentation — product documentation and current operating guidance.
- Docker Engine — Engine architecture, daemon, objects, and administration.
- Docker Engine 29 release notes — current 29.x changes; 29.8.1 was released 2026-09-15.
- Docker Engine API — API negotiation and current client/server compatibility guidance.
- Docker Engine security — namespaces, cgroups, daemon attack surface, and kernel hardening.
- Seccomp security profiles — default syscall filtering and why disabling it is not a routine fix.
- Runtime metrics — current cgroup v2 behavior and container cgroup inspection.
- Rootless mode — daemon/container privilege reduction and user-namespace model.
-
Docker Desktop on Linux
— the VM-backed Desktop execution boundary and
desktop-linuxcontext. - Open Container Initiative — Image, Runtime, and Distribution specifications.
- OCI Runtime Specification — runtime configuration, execution environment, and lifecycle; OCI Runtime Spec v1.3.0 was released in November 2025.
Docker Engine 29.8.1 is the current Engine 29 patch release as of
this lesson's verification date. The current Engine API page
contains an example for 29.8.1 reporting API 1.56 while its
version matrix lists Docker 29.8 with maximum API 1.55. This
chapter therefore treats the output of
docker version on the actual client/daemon pair as
authoritative lab evidence and does not hard-code an expected
negotiated API number.
Keep the academy open
Support free, practical DevOps education.
Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0
Send only Ethereum/ERC-20 compatible assets to this
address.