Docker Engine API, SDKs, Remote Daemons, TLS Authentication, Contexts, and Automation Clients: Guided Hands-On Workflow and Core Operations

1. Lab boundaries and prerequisites

The mutation path targets the same local disposable Engine you already use for coursework; it creates one labeled container and one context that points to the existing local endpoint. It does not alter dockerd listeners, firewall rules, TLS configuration, or production contexts.

2. Capture versions, context, and image identity

set -eu
LAB=dca34-lab
mkdir -p "$LAB/evidence"

date -u +%Y-%m-%dT%H:%M:%SZ | tee "$LAB/evidence/time.txt"
docker context show | tee "$LAB/evidence/context-before.txt"
docker context inspect "$(docker context show)" > "$LAB/evidence/context-before.json"
docker version | tee "$LAB/evidence/docker-version.txt"
docker compose version 2>&1 | tee "$LAB/evidence/compose-version.txt" || true
docker buildx version 2>&1 | tee "$LAB/evidence/buildx-version.txt" || true

docker pull alpine:3.22
docker image inspect alpine:3.22 > "$LAB/evidence/alpine-image.json"
docker image inspect --format 'id={{.Id}} repoDigests={{json .RepoDigests}}' alpine:3.22   | tee "$LAB/evidence/alpine-identity.txt"

The human-readable tag is convenient for the lab; the evidence packet records the resolved local image ID and registry digest(s) so later discussion is not about an ambiguous mutable tag.

3. Create a named local context safely

docker context inspect default >/dev/null

docker context rm -f dca34-local >/dev/null 2>&1 || true
docker context create   --description 'DevOps Academy Chapter 34 local API lab'   --docker host=unix:///var/run/docker.sock   dca34-local

docker context inspect dca34-local | tee "$LAB/evidence/context-lab.json"
docker --context dca34-local version | tee "$LAB/evidence/version-through-context.txt"

This does not change the sticky default context because every subsequent CLI command uses --context dca34-local. Docker Desktop users whose Engine is not exposed at this Linux path should create the context from their actual local endpoint instead, or use the CLI-only track with the existing context.

4. Read-only raw API preflight

SOCK=/var/run/docker.sock
curl --fail --silent --show-error --unix-socket "$SOCK"   http://localhost/_ping | tee "$LAB/evidence/api-ping.txt"

curl --fail --silent --show-error --unix-socket "$SOCK"   http://localhost/version   | tee "$LAB/evidence/api-version.json"   | python -m json.tool

APIVER="$(docker --context dca34-local version --format '{{.Server.APIVersion}}')"
printf 'api=%s
' "$APIVER" | tee "$LAB/evidence/api-selected.txt"

The raw REST URL is versioned in the next steps. Using the server’s advertised API version is appropriate for this tightly coupled local exercise; general-purpose clients should negotiate through the supported SDK/CLI mechanism instead of assuming a fixed server version.

5. Create one exact container through REST

cat > "$LAB/create.json" <<'EOF'
{
  "Image": "alpine:3.22",
  "Cmd": ["sh", "-c", "trap 'exit 0' TERM INT; while :; do sleep 2; done"],
  "Labels": {
    "devops-academy.lab": "chapter34",
    "devops-academy.owner": "student"
  }
}
EOF

curl --fail-with-body --silent --show-error   --unix-socket "$SOCK"   -H 'Content-Type: application/json'   -X POST   --data-binary @"$LAB/create.json"   "http://localhost/v$APIVER/containers/create?name=dca34-api"   | tee "$LAB/evidence/create-response.json"

CID="$(python -c 'import json; print(json.load(open("dca34-lab/evidence/create-response.json"))["Id"])')"
printf '%s
' "$CID" | tee "$LAB/evidence/container-id.txt"

The returned ID is now the authoritative identity for the rest of the workflow. The friendly name is retained for humans, but cleanup does not search for “the latest” container.

6. Start, inspect, and prove state through REST

curl --fail-with-body --silent --show-error   --unix-socket "$SOCK"   -X POST   "http://localhost/v$APIVER/containers/$CID/start"

curl --fail --silent --show-error   --unix-socket "$SOCK"   "http://localhost/v$APIVER/containers/$CID/json"   | tee "$LAB/evidence/container-inspect-api.json"   | python -m json.tool

docker --context dca34-local inspect   --format 'id={{.Id}} status={{.State.Status}} pid={{.State.Pid}} image={{.Image}} labels={{json .Config.Labels}}'   "$CID" | tee "$LAB/evidence/container-inspect-cli.txt"

Two interfaces now corroborate one daemon object. If the returned ID differs, the lab has a context/socket mismatch and should stop.

7. Observe logs and events without guessing readiness

docker --context dca34-local events   --since 10m   --until 0s   --filter container="$CID"   | tee "$LAB/evidence/events.txt" || true

docker --context dca34-local logs --timestamps "$CID"   | tee "$LAB/evidence/logs.txt" || true

This synthetic process has no application protocol or Docker healthcheck, so “running” is the intended runtime state. A web service would require an HTTP/TCP/application-level probe as separate evidence.

8. Optional Python SDK 7.2.0 inspection track

python -m venv "$LAB/.venv"
. "$LAB/.venv/bin/activate"
python -m pip install --upgrade pip
python -m pip install 'docker==7.2.0'
python -m pip freeze | tee "$LAB/evidence/python-packages.txt"

python - <<'PYSDK'
import docker
client = docker.from_env()
print('sdk=', docker.__version__)
print('server=', client.version().get('Version'))
print('api=', client.api.api_version)
obj = client.containers.get('dca34-api')
print('id=', obj.id)
print('status=', obj.status)
print('labels=', obj.labels)
PYSDK

deactivate

The SDK is optional because the raw API + CLI path is already complete. The pinned package makes the SDK example reproducible; learners should still confirm their project’s Python policy and supported SDK version before production use.

9. API version negotiation experiment

# Normal behavior: negotiated by Docker CLI.
docker --context dca34-local version --format 'client={{.Client.APIVersion}} server={{.Server.APIVersion}}'

# Debug-only: forcing a version disables negotiation for this one command.
DOCKER_API_VERSION=1.40 docker --context dca34-local version   | tee "$LAB/evidence/version-forced-1.40.txt"

The forced call is intentionally read-only. Do not make a fixed API version part of normal automation unless a documented compatibility requirement demands it.

10. SSH remote-context design — no real remote required

# DESIGN EXAMPLE ONLY — fake host/user.
docker context create \
  --description 'Example authorized remote Engine over SSH' \
  --docker host=ssh://docker-user@docker-lab.example.invalid   dca34-ssh-example

# Inspect metadata only; do not use the fake endpoint.
docker context inspect dca34-ssh-example

docker context rm dca34-ssh-example

In a real authorized environment, the SSH user must be able to access the Docker socket on the remote host. Public-key authentication and host-key verification belong to SSH; Docker forwards API traffic through that authenticated channel.

11. mTLS remote-daemon design

# DESIGN ONLY — do not expose a daemon for this lab.
# Server concept:
# dockerd --tlsverify --tlscacert=ca.pem --tlscert=server-cert.pem #         --tlskey=server-key.pem -H=tcp://trusted-interface:2376
#
# Client concept:
# docker --tlsverify --tlscacert=ca.pem --tlscert=cert.pem --tlskey=key.pem #        -H=tcp://docker-lab.example.invalid:2376 version

Never place private keys in lesson output, shell history, logs, Git, or build context. A client certificate accepted by the daemon can carry extremely broad host-control authority. Authentication answers “who may connect”; authorization policy may still be needed to limit what an authenticated principal may do.

12. Exact cleanup

CID="$(cat "$LAB/evidence/container-id.txt")"

# Verify label + ID one final time before deletion.
docker --context dca34-local inspect   --format 'id={{.Id}} lab={{index .Config.Labels "devops-academy.lab"}}'   "$CID"

docker --context dca34-local rm -f "$CID"
docker \
  --context dca34-local ps -a \
  --filter label=devops-academy.lab=chapter34 \
  --format '{{.ID}} {{.Names}} {{.Status}}'   | tee "$LAB/evidence/post-cleanup.txt"

docker context rm dca34-local

No prune command is used. The context is only client metadata, and the container is removed by the exact ID returned from its create response.

13. Mini challenge: choose the layer

You run a Python SDK script and receive “connection refused.” The same laptop’s docker ps works. Before rewriting API calls, identify the layer most likely at fault. Expected reasoning: compare the SDK’s endpoint environment (DOCKER_HOST/DOCKER_CONTEXT), Docker Desktop per-user socket if applicable, and selected daemon identity. This is a client/context/transport problem until evidence shows otherwise—not an image or container problem.

Knowledge check

Official references and version notes

Lab baseline: 2026-09-22. Docker SDK for Python 7.2.0 was released July 9, 2026. The raw Unix-socket path is native-Linux oriented; Docker Desktop users must use their actual endpoint and should not copy Linux host paths blindly.

• Docker Docs — Docker Engine API — versioned REST API, current version matrix, negotiation rules, and DOCKER_API_VERSION.
• Docker Docs — Develop with Docker Engine SDKs — supported SDK workflow, Go/Python clients, and version-selection guidance.
• Docker Docs — SDK and API examples — equivalent CLI, Go, Python, and raw HTTP operations.
• Docker Docs — Docker contexts — endpoint identity, TLS metadata, selection, and inspection.
• Docker CLI reference — --context, --host, DOCKER_CONTEXT, DOCKER_HOST, and TLS client options.
• Docker Docs — Protect the Docker daemon socket — SSH and mutual-TLS patterns and private-key authority warnings.
• Docker Docs — Configure remote access — remote endpoint risks, TCP configuration, and firewall considerations.
• Docker Docs — Docker Engine security — host-control implications of daemon access and secure remote transport.
• Docker Docs — docker version — negotiated API reporting and DOCKER_API_VERSION behavior.
• Docker Engine 29 release notes — Engine 29 behavior and compatibility baseline.
• PyPI — Docker SDK for Python — current Python package release identity used by the optional SDK lab.