Checkpoint Lab — Jenkins Configuration as Code, YAML Bundles, Secrets, Reloading, Validation, and Git-Managed Controller State

1. Mission

You are the Jenkins platform operator for a disposable controller. The accepted controller release is defined by a pinned Docker image/plugin set, a Git commit containing JCasC YAML and a separately mounted fake secret. Your task is to reproduce that state, inject a bad supplementary config commit, capture the failure, roll back to the previous commit and prove that live state is again correct.

2. Record assumptions before setup

3. Setup the repository and secret source

mkdir -p ch26-cp/jcasc ch26-cp/secrets ch26-cp/evidence
cd ch26-cp
git init
printf 'secrets/\nevidence/\n' > .gitignore
printf '%s' 'CH26_CP_FAKE_ONLY_f46d' > secrets/lab_admin_password
chmod 600 secrets/lab_admin_password

Create the same pinned Dockerfile from Lesson 2. Create jcasc/00-core.yaml:

jenkins:
  systemMessage: "CH26 checkpoint accepted state"
  numExecutors: 0
  mode: EXCLUSIVE
  securityRealm:
    local:
      allowsSignup: false
      users:
        - id: "lab-admin"
          password: "${lab_admin_password}"
  authorizationStrategy:
    loggedInUsersCanDoAnything:
      allowAnonymousRead: false

git add .gitignore Dockerfile jcasc/00-core.yaml
git commit -m 'checkpoint: accepted jcasc baseline'
ACCEPTED=$(git rev-parse HEAD)
printf '%s\n' "$ACCEPTED" > evidence/accepted-commit.txt
sha256sum jcasc/*.yaml > evidence/accepted-yaml-sha256.txt

4. Predict state changes before execution

Write predictions in evidence/predictions.md before running the controller:

1. Prediction A: startup will create live controller configuration matching the accepted YAML: zero built-in executors, local lab user, accepted system message. It will not create build/workspace evidence because no job is run.
2. Prediction B: adding a second YAML that also sets jenkins.systemMessage will make the next JCasC application fail with a conflict; the previously working runtime state should remain the recovery reference.
3. Prediction C: reverting the bad Git commit and reloading/restarting will restore accepted live state without changing the plugin image.

5. Build and start the accepted controller

docker build -t ch26-cp-jenkins:baseline .
docker image inspect ch26-cp-jenkins:baseline --format '{{.Id}}' \
  > evidence/image-id.txt

docker volume create ch26-cp-home
docker run -d --name ch26-cp-jenkins \
  -p 127.0.0.1:8086:8080 \
  -e JAVA_OPTS='-Djenkins.install.runSetupWizard=false' \
  -e CASC_JENKINS_CONFIG=/var/jenkins_home/casc \
  -e CASC_STRICT_SECRET_RESOLUTION=true \
  -e SECRETS=/run/ch26-secrets \
  -v ch26-cp-home:/var/jenkins_home \
  -v "$PWD/jcasc:/var/jenkins_home/casc:ro" \
  -v "$PWD/secrets:/run/ch26-secrets:ro" \
  ch26-cp-jenkins:baseline

docker logs ch26-cp-jenkins 2>&1 | tee evidence/startup-accepted.log

Verify the controller is ready, log in with the fake lab user, inspect the system message, plugin versions, JCasC source and executor count. Record results without recording the password.

6. Capture accepted-state evidence

Use the Configuration as Code page to download/view an export and save a reviewed/sanitized copy under evidence/accepted-export.yaml. Capture:

• Jenkins/Java versions;
• JCasC/Credentials exact versions;
• Git commit and YAML hashes;
• JCasC source directory;
• system message and executor count;
• startup timestamp/log excerpt;
• secret source type/path and file permission—not the value.

7. Inject the conflict as a Git commit

Create jcasc/99-conflict.yaml:

jenkins:
  systemMessage: "CH26 conflicting state"

git add jcasc/99-conflict.yaml
git commit -m 'checkpoint: inject deliberate jcasc conflict'
BAD=$(git rev-parse HEAD)
printf '%s\n' "$BAD" > evidence/bad-commit.txt
sha256sum jcasc/*.yaml > evidence/bad-yaml-sha256.txt

Before applying it, state the expected failure in the evidence file. Do not modify the accepted YAML.

8. Trigger one controlled reload and preserve the first failure

In Manage Jenkins → Configuration as Code, choose Reload existing configuration once. Expected result: conflict/ConfiguratorException. Immediately preserve the relevant controller log:

docker logs ch26-cp-jenkins 2>&1 > evidence/reload-conflict.log

Do not repeatedly retry. Confirm the error references a conflicting configuration source/attribute. The queue and agents are irrelevant to this controller-global failure.

9. Recover by reverting the source, not by deleting state

git revert --no-edit "$BAD"
RECOVERY=$(git rev-parse HEAD)
printf '%s\n' "$RECOVERY" > evidence/recovery-commit.txt
sha256sum jcasc/*.yaml > evidence/recovery-yaml-sha256.txt

The revert removes the conflicting file in a traceable commit. Reload once from the UI. If reload is not possible because of a runtime/plugin condition, stop and restart the disposable controller using the same volume/image and restored source; record that stronger recovery boundary.

10. Verify recovery independently

Check each prediction rather than trusting a green banner:

• system message equals CH26 checkpoint accepted state;
• built-in executors remain 0;
• lab authentication still works;
• JCasC and Credentials plugin versions did not change;
• the source directory contains no conflicting fragment;
• a new reviewed export/live inspection matches the intended fields;
• controller logs show the successful post-revert application.

Record accepted commit → bad commit → recovery commit as the change chain.

11. Prove secret hygiene

# Never print the secret. Verify only repository/evidence absence.
git grep -n 'CH26_CP_FAKE_ONLY' || true
grep -R -n 'CH26_CP_FAKE_ONLY' evidence jcasc 2>/dev/null || true
stat -c '%a %n' secrets/lab_admin_password 2>/dev/null || \
  ls -l secrets/lab_admin_password

Expected: no secret value in tracked source or evidence, and restrictive local file permissions. A successful login proves the resolver supplied the value.

12. Required evidence packet

13. Cleanup and rollback

docker rm -f ch26-cp-jenkins 2>/dev/null || true
docker volume rm ch26-cp-home 2>/dev/null || true
# Retain the local Git/evidence directory only if you need the training record.

Delete the fake secret file when the lab is complete. Do not use wildcard Docker volume cleanup.

14. What Chapter 26 adds to the production operating model

Jenkins controller configuration is now treated as a release artifact: a protected Git commit bound to an exact core/plugin baseline, with secret references instead of committed values, explicit validation/reload/restart semantics, drift detection and a tested rollback path. That makes controller recovery and review far more deterministic than UI-only administration.

Knowledge check

Answer before revealing the explanation.

Official references and version notes

• Jenkins — Configuration as Code — handbook overview and operating model.
• Configuration as Code plugin — current release, installation, CASC_JENKINS_CONFIG, supplementary-source rules and plugin prerequisites.
• JCasC upstream repository — canonical feature documentation, examples and compatibility notes.
• JCasC — Handling Secrets — SecretSource behavior, file/Docker/Kubernetes/Vault options and strict secret resolution.
• JCasC — Triggering Configuration Reload — UI, authenticated API and CLI reload paths and their privilege requirements.
• JCasC — JSON Schema — instance-specific schema support; current implementation remains beta and should not be the only validation gate.
• Credentials plugin — current credentials API baseline used by the disposable controller image.
• Jenkins LTS changelog and Java support policy — current core/runtime assumptions.