Chapter 30Lesson 05~200 minutes

Checkpoint Lab — SBOMs, SPDX/CycloneDX, in-toto Attestations, SLSA Provenance, Signatures, and Verification Workflows

Publish one disposable digest with SBOM, provenance, and signature evidence; verify all three against the same subject, then prove policy failure when digest or identity expectations are wrong.

CheckpointEvidence packetLocal registryPolicy failureReproducibility

Learning objectives

  • Predict and verify the exact image, registry, attestation, signature, and policy state changes before executing them.
  • Produce SBOM, provenance, signature, and subject-digest evidence for one disposable release.
  • Verify that all evidence resolves to the same subject and record tool/version/timestamp context.
  • Demonstrate policy failure for both a changed digest and an unexpected verification identity/key.
  • Clean up only checkpoint resources while retaining an auditable evidence packet and limitations note.

1. Scenario and success criteria

You are the release engineer for a synthetic service. Promotion is allowed only when one immutable digest has: (1) an SBOM, (2) provenance, and (3) a signature from the checkpoint key. You must then prove that the same policy rejects a changed digest and a mismatched verification key.

RPO is not the concept here; evidence continuity is. The checkpoint succeeds only if someone else can read your packet later and determine exactly which subject was built, which evidence existed, which signer was expected, which checks passed/failed, and what limitations remained.

2. Preflight and assumptions packet

set -eu
mkdir -p dca30-checkpoint && cd dca30-checkpoint
exec > >(tee -a checkpoint-transcript.log) 2>&1

date -u +%FT%TZ
docker version
docker info
docker context show
docker buildx version
docker buildx ls
docker compose version || true
cosign version || true

# Record host/runtime component evidence where available.
docker info --format '{{json .}}' > docker-info.json

Write ASSUMPTIONS.md before changing state. Include: Engine/CLI/Compose/Buildx versions; BuildKit worker version from buildx inspect; containerd/runc versions if exposed by docker info/docker version; OS/platform; port 5006 availability; Cosign version; and the fact that this is an HTTP loopback registry with no production trust properties.

cat > ASSUMPTIONS.md <<'EOF'
# Chapter 30 checkpoint assumptions
- Date baseline: 2026-09-22
- Mandatory path: local, disposable, synthetic data only
- Registry: localhost:5006, HTTP, loopback-only, registry:3.1.1
- Builder: isolated docker-container Buildx builder
- Signing: local throwaway Cosign key; no public transparency log
- Evidence policy: exact subject digest + SBOM + provenance + matching local public key
- Limitation: this lab proves mechanics, not production-grade key custody or CI identity
EOF

3. Predict state changes before execution

Prediction Layer Independent verification
Starting registry creates one loopback listener and one labeled container. Host/network/container docker ps, docker port, inspect labels.
Build pushes a new image index plus attestation manifests/referrers. Build/image/registry imagetools inspect --raw and SBOM/provenance templates.
Signing adds signature evidence for the exact digest without changing image bytes. Registry/security/identity Subject digest remains identical; Cosign verify succeeds.
Changing source creates a different digest. Source/build/image Compare SUBJECT and SUBJECT2.
Old signature policy rejects the unsigned second digest. Verification/policy Non-zero verifier exit and preserved stderr.

Write your own predicted values or qualitative outcomes in PREDICTIONS.md before running the build. The purpose is to separate expected causality from observations made after the fact.

4. Create the disposable registry and builder

docker pull registry:3.1.1
docker run -d --name dca30-checkpoint-registry   -p 127.0.0.1:5006:5000   --label devops.academy.chapter=30   registry:3.1.1

docker buildx create --name dca30-checkpoint-builder   --driver docker-container --use
docker buildx inspect --bootstrap | tee builder-inspect.txt

docker ps --filter name=dca30-checkpoint-registry
docker port dca30-checkpoint-registry

Do not configure an insecure registry daemon-wide. The build output and Cosign commands will explicitly opt into HTTP for this loopback test endpoint.

5. Build release v1 with evidence

cat > app.sh <<'EOF'
#!/bin/sh
printf '%s
' 'chapter30 checkpoint release v1'
EOF
chmod +x app.sh

cat > Dockerfile <<'EOF'
# syntax=docker/dockerfile:1
FROM alpine:3.22.1
COPY --chmod=0555 app.sh /usr/local/bin/checkpoint-app
USER 65532:65532
ENTRYPOINT ["/usr/local/bin/checkpoint-app"]
EOF

sha256sum Dockerfile app.sh | tee source-v1.sha256
IMAGE=localhost:5006/dca30/checkpoint:1

docker buildx build --builder dca30-checkpoint-builder   --progress=plain   --sbom=true   --attest type=provenance,mode=max,version=v1   --output type=registry,registry.insecure=true   --metadata-file build-v1-metadata.json   -t "$IMAGE" . | tee build-v1.log

DIGEST=$(docker buildx imagetools inspect "$IMAGE" --format '{{.Manifest.Digest}}')
SUBJECT="localhost:5006/dca30/checkpoint@${DIGEST}"
printf '%s
' "$SUBJECT" | tee subject-v1.txt

This checkpoint explicitly requests SLSA provenance v1 rather than relying on BuildKit’s default provenance schema. That makes the policy expectation observable.

6. Extract and bind SBOM/provenance to the subject

docker buildx imagetools inspect "$SUBJECT" --raw > subject-v1-index.json
docker buildx imagetools inspect "$SUBJECT" --format '{{json .SBOM}}' > sbom-v1.json
docker buildx imagetools inspect "$SUBJECT" --format '{{json .Provenance}}' > provenance-v1.json

sha256sum subject-v1-index.json sbom-v1.json provenance-v1.json   | tee evidence-v1.sha256

# Optional structured checks when jq is present.
jq '.SPDX.SPDXID' sbom-v1.json || true
jq '.SLSA.buildType, .SLSA.builder, .SLSA.materials' provenance-v1.json || true

Record the attestation/referrer descriptors visible in subject-v1-index.json. If the registry/tool represents attestations through compatibility annotations rather than an OCI 1.1 referrers response, record what you actually observed rather than rewriting it to match a diagram.

7. Sign and verify the exact v1 digest

mkdir -m 700 keys
(
  cd keys
  cosign generate-key-pair
)

cosign sign --yes --key keys/cosign.key   --allow-http-registry --tlog-upload=false   "$SUBJECT" | tee sign-v1.txt

cosign verify --key keys/cosign.pub   --allow-http-registry --insecure-ignore-tlog=true   "$SUBJECT" | tee verify-v1.json

# Re-resolve subject after signing: image digest must not change.
DIGEST_AFTER_SIGN=$(docker buildx imagetools inspect "$IMAGE" --format '{{.Manifest.Digest}}')
printf 'before=%s
after=%s
' "$DIGEST" "$DIGEST_AFTER_SIGN"   | tee digest-after-sign.txt
test "$DIGEST" = "$DIGEST_AFTER_SIGN"

A signature should add associated evidence, not mutate the image manifest bytes. The equality check is therefore part of the checkpoint.

8. Intentional policy failure #1: subject changed

printf '%s
' '# subject-changing edit' >> app.sh
sha256sum Dockerfile app.sh | tee source-v2.sha256
IMAGE2=localhost:5006/dca30/checkpoint:2

docker buildx build --builder dca30-checkpoint-builder   --sbom=true --attest type=provenance,mode=max,version=v1   --output type=registry,registry.insecure=true   -t "$IMAGE2" . | tee build-v2.log

DIGEST2=$(docker buildx imagetools inspect "$IMAGE2" --format '{{.Manifest.Digest}}')
SUBJECT2="localhost:5006/dca30/checkpoint@${DIGEST2}"
printf 'v1=%s
v2=%s
' "$SUBJECT" "$SUBJECT2" | tee subject-diff.txt
test "$DIGEST" != "$DIGEST2"

set +e
cosign verify --key keys/cosign.pub --allow-http-registry   --insecure-ignore-tlog=true "$SUBJECT2"   > verify-v2-expected-fail.txt 2>&1
RC1=$?
set -e
printf 'digest_mismatch_policy_exit=%s
' "$RC1" | tee policy-failure-1.txt
test "$RC1" -ne 0

The second build may have its own SBOM and provenance, but it has not satisfied the checkpoint signature requirement. Evidence does not transfer from v1 merely because the repository name is the same.

9. Intentional policy failure #2: wrong verification identity

mkdir -m 700 wrong-keys
(
  cd wrong-keys
  cosign generate-key-pair
)

set +e
cosign verify --key wrong-keys/cosign.pub --allow-http-registry   --insecure-ignore-tlog=true "$SUBJECT"   > wrong-key-expected-fail.txt 2>&1
RC2=$?
set -e
printf 'wrong_identity_policy_exit=%s
' "$RC2" | tee policy-failure-2.txt
test "$RC2" -ne 0

This failure separates subject correctness from identity correctness. The image is still v1, but the verifier’s trust anchor is wrong.

10. Evidence packet and limitations note

Your final packet should include at least:

Evidence Minimum contents
Environment UTC timestamp, Docker version/info/context, Buildx/Compose, builder inspect, Cosign version.
Source/build Dockerfile + source checksums, build logs, metadata file, builder identity/worker details.
Subject v1 digest/platform/raw index and v2 comparison digest.
SBOM raw extracted JSON, generator/format observations, checksum.
Provenance raw extracted JSON, predicate/build type/materials/builder observations, checksum.
Signature public key, sign output, successful verification output; never archive the private key in the evidence packet.
Policy failures changed-digest verifier failure and wrong-key verifier failure with exit codes.
Assumptions/limits HTTP local registry, local key custody, no transparency log, no production admission controller.
cat > LIMITATIONS.md <<'EOF'
# Limitations
- Local HTTP registry proves mechanics only; production registries require authenticated TLS and retention policy.
- Local file key proves public-key verification but not production key custody, KMS/HSM, or OIDC workload identity.
- Transparency logging is disabled in this exercise; no Rekor inclusion claim is made.
- BuildKit-generated SBOM/provenance content quality depends on generator/builder capabilities and inputs.
- Successful verification proves the configured checks, not vulnerability-free or malicious-code-free software.
EOF

sha256sum *.json *.txt *.md *.sha256 2>/dev/null | sort > packet.sha256

11. Verification checklist

Checkpoint passes only if:
  • v1 has a recorded immutable subject digest.
  • SBOM and provenance can be extracted for v1 and retained as evidence.
  • Provenance uses the requested v1 predicate path or the discrepancy is documented.
  • Cosign verification with the intended public key succeeds for v1.
  • Signing did not change the v1 image digest.
  • v2 has a different digest and fails the old signature policy.
  • v1 fails verification with the wrong public key.
  • No private key, production credential, Docker socket, or broad host privilege enters the packet.
  • Cleanup targets only named checkpoint resources.

12. Cleanup and rollback

docker buildx use default || true
docker buildx rm dca30-checkpoint-builder
docker rm -f dca30-checkpoint-registry

# Destroy disposable private keys; retain public key + verification evidence if desired.
rm -f keys/cosign.key wrong-keys/cosign.key wrong-keys/cosign.pub
# Optional after archiving evidence: rm -rf dca30-checkpoint

No docker system prune, daemon restart, security-control disablement, or production registry mutation is required.

13. What Chapter 30 adds to the operating model

Earlier chapters made builds reproducible, images immutable, runtime state inspectable, secrets scoped, and vulnerability findings digest-bound. Chapter 30 adds machine-readable evidence with explicit subject and identity verification. A production operating model can now say not just “this digest passed a scan,” but “this exact digest carries an inventory, a build-origin statement, and an authenticated release identity that were checked under policy version X at time Y.”

Chapter 31

Containerd image store, snapshotters, runtime internals, OCI specs, and Engine evolution

Next we move beneath the evidence layer and inspect how Docker Engine’s modern containerd-backed storage/runtime architecture represents and evolves image content.

Knowledge check

Why must the checkpoint re-resolve the digest after signing?

v2 has an SBOM and provenance but fails Cosign verification. Is that a build failure?

Why is the private Cosign key excluded from the evidence packet?

What does the wrong-key failure prove?

Does a successful SBOM + provenance + signature checkpoint prove the image is secure?

Official references and version notes

Checkpoint baseline: Engine 29.8.1; Buildx 0.37.1; BuildKit 0.33.0; Compose 5.5.1; containerd 2.3.5; runc 1.5.1; Cosign 3.1.3; registry 3.1.1; OCI artifact/referrers model 1.1; SPDX 3.0.1; CycloneDX 1.7; SLSA 1.2. Record actual installed/package versions because Docker Desktop and distribution packages can differ.

Keep the academy open

Support free, practical DevOps education.

Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.

Ethereum / ERC-20
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0Send only Ethereum/ERC-20 compatible assets to this address.