Checkpoint Lab — Secrets and Configuration Patterns, Runtime Injection, Environment Risk, Build Secrets, and External Secret Stores

1. Checkpoint scenario and success criteria

You own a synthetic application that must authenticate to a private package source during build and to a service at runtime. There is no cloud account, production registry, or real credential. Success means the build fails without its required fake build token, succeeds with it, leaves the token out of the final image/history/logs, and the runtime container receives a separate fake credential through a file-based grant. You then rotate the runtime source from v1 to v2 and prove which lifecycle action activates it.

2. Record assumptions first

set -eu
LAB="$PWD/docker-ch28-checkpoint"
rm -rf "$LAB"
mkdir -p "$LAB/app" "$LAB/secrets" "$LAB/provider" "$LAB/evidence"
cd "$LAB"

docker version > evidence/docker-version.txt
docker info > evidence/docker-info.txt
docker context inspect "$(docker context show)" > evidence/context.json
docker buildx version > evidence/buildx-version.txt
docker buildx inspect --bootstrap > evidence/builder.txt
docker compose version > evidence/compose-version.txt
printf '%s
' "captured_at=$(date -u +%Y-%m-%dT%H:%M:%SZ)" > evidence/assumptions.txt
printf '%s
' 'mandatory path: local Docker/BuildKit/Compose; no paid provider required' >> evidence/assumptions.txt

3. Predict state changes

4. Declare source boundaries

printf '%s
' 'CHECKPOINT-BUILD-v1-FAKE' > secrets/build-token.txt
printf '%s
' 'CHECKPOINT-RUNTIME-v1-FAKE' > provider/runtime-token.txt
printf '%s
' 'v1' > provider/runtime-version.txt
chmod 600 secrets/build-token.txt provider/runtime-token.txt 2>/dev/null || true

cat > .dockerignore <<'EOF'
secrets/
provider/
evidence/
.env
*.key
*.pem
EOF

cat > app/app.sh <<'EOF'
#!/bin/sh
set -eu
test -s /run/secrets/runtime_token
# The version file is non-secret operational metadata.
echo "runtime-secret-present=yes version=${RUNTIME_SECRET_VERSION:-unknown}"
while :; do sleep 30; done
EOF
chmod +x app/app.sh

The version label is deliberately separate from the credential value. This is a useful production pattern: rotate/observe by version ID without logging the secret.

5. Build with an explicit required secret

cat > Dockerfile <<'EOF'
# syntax=docker/dockerfile:1
FROM alpine:3.22.1
RUN --mount=type=secret,id=build_token,required=true     test -s /run/secrets/build_token &&     printf '%s
' 'private-dependency-step=ok' > /build-evidence
RUN addgroup -S app && adduser -S -G app app
COPY --chown=app:app app/app.sh /usr/local/bin/app.sh
USER app
ENTRYPOINT ["/usr/local/bin/app.sh"]
EOF

# Negative control: expected failure because required secret is missing.
if docker buildx build --progress=plain --load -t da-ch28-check:missing . > evidence/build-missing.log 2>&1; then
  echo 'unexpected-success-without-build-secret' >&2
  exit 1
else
  echo 'required-build-secret-enforced=yes' | tee evidence/build-missing-result.txt
fi

docker buildx build --progress=plain   --secret id=build_token,src=secrets/build-token.txt   --load -t da-ch28-check:ok . 2>&1 | tee evidence/build-ok.log
IMG_ID="$(docker image inspect da-ch28-check:ok --format '{{.Id}}')"
printf '%s
' "image_id=$IMG_ID" | tee evidence/image-id.txt

6. Prove build-secret non-persistence across practical surfaces

docker image inspect da-ch28-check:ok > evidence/image-inspect.json
docker history --no-trunc da-ch28-check:ok > evidence/image-history.txt
docker run --rm --entrypoint sh da-ch28-check:ok -c   'cat /build-evidence; test ! -e /run/secrets/build_token'

if grep -R -F 'CHECKPOINT-BUILD-v1-FAKE'   evidence/image-inspect.json evidence/image-history.txt evidence/build-ok.log; then
  echo 'FAIL: fake build credential found in evidence' >&2
  exit 1
else
  echo 'build-secret-leak-check=pass' | tee evidence/build-secret-leak-check.txt
fi

7. Runtime injection with separate version metadata

cat > compose.yaml <<'EOF'
name: da-ch28-check
services:
  app:
    image: da-ch28-check:ok
    environment:
      RUNTIME_SECRET_VERSION: v1
    read_only: true
    tmpfs:
      - /tmp:size=8m,mode=1777
    secrets:
      - runtime_token
secrets:
  runtime_token:
    file: ./provider/runtime-token.txt
EOF

docker compose config > evidence/compose-v1.yaml
docker compose up -d
CID1="$(docker compose ps -q app)"
docker inspect "$CID1" > evidence/container-v1.json
docker compose logs --no-color --tail 20 app | tee evidence/app-v1.log

docker inspect "$CID1" --format 'Env={{json .Config.Env}} Mounts={{json .Mounts}}'   > evidence/runtime-surface-v1.txt

The environment contains only the non-secret version label. The credential itself remains a file grant. Do not mistake a version label for a secret value.

8. Rotate runtime v1 → v2

printf '%s
' 'CHECKPOINT-RUNTIME-v2-FAKE' > provider/runtime-token.txt
printf '%s
' 'v2' > provider/runtime-version.txt
python - <<'PY2'
p='compose.yaml'
s=open(p,encoding='utf-8').read().replace('RUNTIME_SECRET_VERSION: v1','RUNTIME_SECRET_VERSION: v2')
open(p,'w',encoding='utf-8').write(s)
PY2

docker compose config > evidence/compose-v2.yaml
docker compose up -d --force-recreate app
CID2="$(docker compose ps -q app)"
printf '%s
' "container_v1=$CID1" "container_v2=$CID2" > evidence/rotation-containers.txt
docker inspect "$CID2" > evidence/container-v2.json
docker compose logs --no-color --tail 20 app | tee evidence/app-v2.log

IMG_ID_AFTER="$(docker image inspect da-ch28-check:ok --format '{{.Id}}')"
printf '%s
' "image_id_after=$IMG_ID_AFTER" > evidence/image-id-after.txt
test "$IMG_ID" = "$IMG_ID_AFTER"

Image identity did not change because the runtime secret is external to the image. The local lab explicitly recreates the consumer; another provider/integration may use a different refresh model.

9. Evidence packet and limitations

cat >> evidence/assumptions.txt <<'EOF'
secret_values_are_fake=yes
build_secret_channel=BuildKit secret mount
runtime_secret_channel=local Compose per-service secret file
compose_local_secret_is_not_assumed_encrypted_at_rest=yes
external_provider=simulated local file boundary
logs_intentionally_contain_presence_and_version_only=yes
EOF

tar -czf ../docker-ch28-evidence.tgz evidence

Before sharing even this synthetic packet, review host paths, usernames, builder endpoints, and any other environment metadata. In production, the corresponding packet would reference provider audit IDs and secret versions without embedding credential values.

10. Verification checklist

• Required build secret absence causes the protected build step to fail.
• Successful build uses a BuildKit secret mount and no fake build-token marker appears in image config/history/build log.
• Runtime credential is not stored in the image and is not placed in Config.Env.
• Application logs show presence/version only, not the credential.
• Runtime rotation changes provider/consumer version evidence but leaves image identity unchanged.
• Cleanup targets only named lab containers/images/directories; no broad prune is required.

11. Cleanup and production bridge

docker compose down
docker image rm da-ch28-check:ok da-ch28-check:missing 2>/dev/null || true
cd ..
# Keep docker-ch28-evidence.tgz if desired. Remove only this lab directory after review.
# rm -rf "$LAB"

This chapter adds a secret lifecycle to the secure Docker operating model: owner, scope, phase, delivery path, consumer, rotation, revocation, and exposure evidence are all independent of image identity. Chapter 29 builds on that supply-chain mindset by examining vulnerability scanning, dependency hygiene, base-image maintenance, and remediation.

Knowledge check

Official references and version notes

• Docker Build secrets — secret mounts, SSH mounts, Git authentication secrets, file/environment sources, and build-time scope.
• Dockerfile reference — RUN --mount=type=secret, RUN --mount=type=ssh, required, target, ownership, and environment exposure options.
• SecretsUsedInArgOrEnv build check — why Dockerfile ARG/ENV are not secret channels.
• Manage secrets securely in Docker Compose — per-service grants and file-based runtime delivery.
• Compose secrets reference — top-level secret definitions and service consumption.
• Compose environment-variable best practices — configuration precedence and why sensitive data should use secrets.
• Swarm secrets — orchestrator-managed secret semantics, intentionally distinct from local Compose file mounts.
• Docker Engine 29 release notes — current Engine baseline.
• Buildx releases and BuildKit releases — current builder/frontend assumptions.