JENKINS_HOME, Filesystem Layout, Controller Configuration, System Settings, Tools, and Global Properties: Guided Hands-On Workflow and Core Operations

1. Preflight: create a chapter-specific disposable controller

This lab is independent of Chapter 02 so it remains runnable even if that controller was removed. It binds Jenkins to loopback and uses a chapter-specific named volume. The setup wizard still requires a local browser step; do not expose it publicly.

docker --version
docker pull jenkins/jenkins:2.568.3-jdk21

docker volume create jenkins-ch03-home
docker run -d \
  --name jenkins-ch03-controller \
  --restart=no \
  -p 127.0.0.1:18083:8080 \
  -v jenkins-ch03-home:/var/jenkins_home \
  jenkins/jenkins:2.568.3-jdk21

docker logs --tail 80 jenkins-ch03-controller

Open http://127.0.0.1:18083 and complete the local setup wizard with synthetic training values. Retrieve the initial administrator password locally only when the wizard requires it. Do not paste that password into evidence, source control, screenshots, or chat.

2. Capture a metadata-only baseline

mkdir -p jenkins-ch03-evidence

docker exec jenkins-ch03-controller sh -lc '
  printf "JENKINS_HOME=%s\n" "$JENKINS_HOME"
  id
  java -version 2>&1 | head -4
  stat -c "owner=%U:%G mode=%a path=%n" "$JENKINS_HOME"
  printf "top-level entries:\n"
  find "$JENKINS_HOME" -maxdepth 1 -mindepth 1 -printf "%f\n" | sort
' | tee jenkins-ch03-evidence/baseline.txt

docker inspect --format='image={{.Config.Image}} mount={{range .Mounts}}{{.Name}}:{{.Destination}}{{end}}' \
  jenkins-ch03-controller | tee -a jenkins-ch03-evidence/baseline.txt

The inventory intentionally records names rather than file contents. If you need version evidence, use Manage Jenkins → About/System Information or an authenticated API request after setup; do not infer the running core version solely from a tag.

3. Make controlled changes through supported UI surfaces

In the disposable controller, open Manage Jenkins → System. Set the System Message to:

DevOps Academy — Chapter 03 disposable controller

On the same page, under Global properties, enable Environment variables and add:

ACADEMY_CHAPTER = 03

Save. These values are synthetic and safe to find in persisted configuration. Do not use this exercise to add credentials, tokens, SMTP passwords, cloud keys, or other real secrets.

4. Map persisted files before and after without dumping them

docker exec jenkins-ch03-controller sh -lc '
  stat -c "%y %s %n" "$JENKINS_HOME/config.xml"
  grep -n -E "systemMessage|ACADEMY_CHAPTER" "$JENKINS_HOME/config.xml" || true
' | tee jenkins-ch03-evidence/config-safe-excerpt.txt

docker exec jenkins-ch03-controller sh -lc '
  find "$JENKINS_HOME" -maxdepth 2 -type f \
    ! -path "*/secrets/*" \
    -printf "%TY-%Tm-%Td %TH:%TM:%TS %10s %p\n" 2>/dev/null \
    | sort | tail -80
' > jenkins-ch03-evidence/recent-files.txt

We grep only the two synthetic values we created. A production config.xml or plugin XML may contain sensitive or security-relevant values, so “cat every XML file” is not an acceptable evidence strategy.

5. Restart proves persistence—not recoverability

docker restart jenkins-ch03-controller
docker logs --tail 60 jenkins-ch03-controller

docker exec jenkins-ch03-controller sh -lc '
  grep -n -E "systemMessage|ACADEMY_CHAPTER" "$JENKINS_HOME/config.xml" || true
'

Reload the UI and confirm the System Message is still present. A restart demonstrates that the value is on durable controller storage. It does not prove that the volume can be restored after deletion, that plugin versions are recoverable, or that external integrations still work.

6. Prove that container-local temporary state is different

docker exec jenkins-ch03-controller sh -lc '
  printf "temporary chapter03 marker\n" > /tmp/jenkins-ch03-ephemeral.txt
  printf "durable chapter03 marker\n" > "$JENKINS_HOME/ch03-durable-marker.txt"
  ls -l /tmp/jenkins-ch03-ephemeral.txt "$JENKINS_HOME/ch03-durable-marker.txt"
'

The /tmp marker lives in the container layer. The home marker lives on the named volume. The checkpoint lab will recreate a controller from backup and verify that the durable marker returns while the container-local marker does not. Neither marker represents an agent workspace; agent execution state belongs to the agent machine/container/pod and is covered in later chapters.

7. Guarded snapshot: stop first, separate ordinary data from secret keys

For this training controller, we make a consistent stopped-controller snapshot. Production systems may use storage snapshots and coordinated backup tooling, but the same trust boundary applies.

docker stop jenkins-ch03-controller

docker volume create jenkins-ch03-config-backup
docker volume create jenkins-ch03-secret-backup

# Ordinary controller backup: intentionally excludes the secrets directory.
docker run --rm --entrypoint /bin/sh \
  -v jenkins-ch03-home:/source:ro \
  -v jenkins-ch03-config-backup:/backup \
  jenkins/jenkins:2.568.3-jdk21 -c '
    cd /source &&
    tar --exclude=./secrets -czf /backup/jenkins-home-nonsecrets.tgz . &&
    sha256sum /backup/jenkins-home-nonsecrets.tgz
  '

# Protected recovery material: kept in a separate volume.
docker run --rm --entrypoint /bin/sh \
  -v jenkins-ch03-home:/source:ro \
  -v jenkins-ch03-secret-backup:/backup \
  jenkins/jenkins:2.568.3-jdk21 -c '
    cd /source &&
    tar -czf /backup/jenkins-secrets.tgz secrets &&
    chmod 600 /backup/jenkins-secrets.tgz &&
    ls -l /backup/jenkins-secrets.tgz
  '

docker start jenkins-ch03-controller

8. Controller home is not an agent workspace

Current Jenkins controller-isolation guidance is to run builds on agents instead of the built-in node. When later chapters create agents, their workspaces may exist on entirely different filesystems and disappear when ephemeral agents terminate. Therefore:

• Do not put release truth in an agent workspace.
• Do not copy arbitrary workspaces into controller backups.
• Archive or publish intended evidence explicitly.
• Record source SHA, build number, artifact digest, and external repository identity instead of relying on “the file was still in workspace.”

9. Mini challenge: classify the state before choosing a fix

For each item, decide whether it belongs to controller durable state, separately protected recovery material, agent execution state, or an external system:

1. A global System Message.
2. The encryption keys under JENKINS_HOME/secrets.
3. A Maven target/ directory on a Kubernetes agent.
4. A promoted package in Nexus.
5. A job’s retained build record and archived artifact.

Then state what evidence proves recovery for each. The correct answer is not “back up everything together.”

10. Cleanup choices

If you are continuing to Lesson 5, keep the controller and three volumes. Otherwise remove only the chapter resources you created:

docker rm -f jenkins-ch03-controller
# Delete these only if you intentionally discard the lab and its recovery exercise:
# docker volume rm jenkins-ch03-home jenkins-ch03-config-backup jenkins-ch03-secret-backup
rm -rf jenkins-ch03-evidence

11. Summary

• Supported UI changes become durable controller state under JENKINS_HOME.
• Metadata-only inspection is safer than dumping controller configuration.
• A restart proves persistence, not disaster recovery.
• Container-local files, controller-home files, agent workspaces, and external systems are distinct layers.
• Consistent backups and separately protected secret-key recovery material are both required for a credible restore.

Knowledge check

Official references and version notes

• Configuring the System — current Jenkins home-directory and global system-configuration guidance.
• Managing Jenkins — current administrative surfaces for System, Tools, Plugins, status, and troubleshooting.
• System Information — controller system properties, environment variables, plugins, memory information, and diagnostics.
• Managing Tools — built-in tool-provider concepts and global tool configuration.
• Backing-up/Restoring Jenkins — backup scope, controller-key separation, and restore validation.
• Credentials security — why Jenkins secret-key material needs protection and separation from ordinary backups.
• Storing Secrets — technical description of Jenkins encryption keys under $JENKINS_HOME/secrets.
• Controller Isolation — current guidance for keeping routine builds off the built-in controller node.
• Jenkins LTS changelog and Java Support Policy — version/runtime assumptions for this chapter.