LTS Upgrades, Java Runtime Transitions, Plugin Upgrade Strategy, Compatibility Testing, and Rollback Planning: Guided Hands-On Workflow and Core Operations

1. Disposable lab scenario

The lab models a controller on Jenkins 2.541.3 with Java 17 moving to 2.568.3 with Java 21. That path intentionally crosses the 2.555.1 Java boundary. Keep the controller loopback-only and synthetic because the old baseline is no longer the security target.

2. Preflight and pull exact lab images

set -euo pipefail
LAB='/tmp/jenkins-upgrade-lab'
mkdir -p "$LAB/evidence"
docker pull jenkins/jenkins:2.541.3-lts-jdk17
docker pull jenkins/jenkins:2.568.3-lts-jdk21
docker image inspect jenkins/jenkins:2.541.3-lts-jdk17 --format '{{index .RepoDigests 0}}' | tee "$LAB/evidence/old-image.txt"
docker image inspect jenkins/jenkins:2.568.3-lts-jdk21 --format '{{index .RepoDigests 0}}' | tee "$LAB/evidence/target-image.txt"

3. Start the old disposable controller and create representative state

set -euo pipefail
docker volume create jenkins-upgrade-home-old >/dev/null
docker run -d --name jenkins-upgrade-old \
  -p 127.0.0.1:18080:8080 \
  -v jenkins-upgrade-home-old:/var/jenkins_home \
  jenkins/jenkins:2.541.3-lts-jdk17
printf 'Open http://127.0.0.1:18080 and complete only disposable setup.\n'

Create the smallest synthetic Pipeline job and one fake credential metadata entry if you want to test credential restoration. Never use a real secret. Record the job full name, one successful build number, source/Jenkinsfile revision, and any installed plugins you deliberately added.

pipeline {
  agent any
  stages {
    stage('Identity') {
      steps {
        sh 'java -version 2>&1; uname -a; printf "build=%s\\n" "$BUILD_NUMBER"'
      }
    }
    stage('Smoke') {
      steps {
        sh 'printf "upgrade-lab-ok\\n" > smoke.txt; sha256sum smoke.txt'
        archiveArtifacts artifacts: 'smoke.txt', fingerprint: true
      }
    }
  }
}

4. Capture read-only inventory

Use Manage Jenkins → System Information, Manage Jenkins → Plugins → Installed, node System Information, and job/build pages. Capture only non-secret fields.

Inventory record
----------------
controller_url=http://127.0.0.1:18080
core=2.541.3
controller_java=17.x
install_method=official Docker image
job=upgrade-lab/smoke
last_known_good_build=<number>
plugin_manifest=<short-name:version list>
JCasC_revision=<none or commit>
agents=<name / Java / labels / launcher>
external_integrations=none (mandatory lab)

5. Read every crossed LTS guide and create a change ledger

For this path read the 2.555.x and 2.568.x guides in order. The critical 2.555.1 requirement is Java 21/25 for controller and agents. The 2.568.x guide adds its own platform/image notes. Also review the current security advisory immediately before the rehearsal.

6. Stop and create a recoverable pre-upgrade snapshot

For a filesystem-level Docker volume backup, stop the old controller first so the copy is consistent.

set -euo pipefail
LAB='/tmp/jenkins-upgrade-lab'
docker stop jenkins-upgrade-old >/dev/null
docker run --rm \
  -v jenkins-upgrade-home-old:/source:ro \
  -v "$LAB":/backup \
  alpine:3.22 sh -c 'cd /source && tar -czf /backup/preupgrade-home.tgz .'
sha256sum "$LAB/preupgrade-home.tgz" | tee "$LAB/evidence/preupgrade-home.sha256"
ls -lh "$LAB/preupgrade-home.tgz"

Do not call this a validated rollback yet. It becomes validated only after you restore it into a separate volume and prove the old controller can start from it.

7. Restore the snapshot into a target volume

set -euo pipefail
LAB='/tmp/jenkins-upgrade-lab'
docker volume create jenkins-upgrade-home-target >/dev/null
docker run --rm \
  -v jenkins-upgrade-home-target:/target \
  -v "$LAB":/backup:ro \
  alpine:3.22 sh -c 'cd /target && tar -xzf /backup/preupgrade-home.tgz'

The old volume is now preserved. The target upgrade mutates only the cloned volume, which keeps rollback evidence clean.

8. Start the cloned state on the target core/runtime

set -euo pipefail
docker run -d --name jenkins-upgrade-target \
  -p 127.0.0.1:18081:8080 \
  -v jenkins-upgrade-home-target:/var/jenkins_home \
  jenkins/jenkins:2.568.3-lts-jdk21
sleep 8
docker logs --tail 200 jenkins-upgrade-target | tee /tmp/jenkins-upgrade-lab/evidence/target-startup.log

Preserve the first startup log before installing/updating anything. Look for migration warnings, failed plugin loads, missing dependencies, Java errors and configuration failures.

9. Reconcile plugins after core startup

From Manage Jenkins → Plugins, review installed plugin status, dependency warnings, security warnings, minimum core and restart requirements. Do not click “update all” simply because the target started.

For this dated lab, if installed, the following must meet the 16 September 2026 fixes:

Update a small dependency-coherent batch, restart if required, then re-run smoke checks before the next batch. Keep a before/after manifest.

10. Run acceptance tests in increasing scope

1. Controller health and UI/API reachability.
2. Authentication/authorization behavior with disposable identities.
3. Plugin load/dependency health.
4. upgrade-lab/smoke using the same Pipeline revision.
5. Artifact archive/fingerprint retrieval.
6. Representative Shared Library or multibranch job if present.
7. Agent pools: Java, Remoting connection, labels and tools.
8. External integrations in a real staging environment.
9. Queue/build/controller performance against Chapter 38 baseline.

11. Intentionally expose one compatibility failure: Java 17 agent

Keep one disposable inbound agent image/runtime at Java 17. After the controller is on 2.568.3, attempt to connect it. Preserve the agent log showing the unsupported runtime rather than repeatedly restarting it. Then rebuild/relaunch that lab agent on Java 21 and verify the same node/labels/toolchain return online.

12. Rehearse rollback correctly: restore, do not downgrade mutated data

Stop the target. Start a separate recovery controller from the untouched old volume or a freshly restored copy of the pre-upgrade archive using the old core/runtime. This proves the rollback source independently.

set -euo pipefail
docker stop jenkins-upgrade-target >/dev/null || true
docker run -d --name jenkins-upgrade-rollback \
  -p 127.0.0.1:18082:8080 \
  -v jenkins-upgrade-home-old:/var/jenkins_home \
  jenkins/jenkins:2.541.3-lts-jdk17

Verify the old controller/job/build identity and then stop it. Never point the old image at the already-upgraded target volume as your “rollback.”

13. Cleanup

set -euo pipefail
for c in jenkins-upgrade-old jenkins-upgrade-target jenkins-upgrade-rollback; do
  docker rm -f "$c" 2>/dev/null || true
done
for v in jenkins-upgrade-home-old jenkins-upgrade-home-target; do
  docker volume rm "$v" 2>/dev/null || true
done
LAB='/tmp/jenkins-upgrade-lab'
case "$LAB" in /tmp/jenkins-upgrade-lab) rm -rf -- "$LAB" ;; *) exit 70 ;; esac

Knowledge check

Answer before revealing the explanation.

Official references and version notes

Upgrade behavior is version-specific. Always read every skipped LTS guide and the current security advisories for the exact day you plan an upgrade.

• Jenkins LTS Upgrade Guide
• Upgrading to Jenkins LTS 2.555.x
• Upgrading to Jenkins LTS 2.568.x
• Jenkins Java Support Policy
• Upgrade Jenkins runtime to Java 21
• Managing Jenkins plugins
• Using Jenkins agents
• Backing up Jenkins
• Jenkins LTS changelog
• Jenkins Security Advisories
• Jenkins Security Advisory 2026-09-16
• Jenkins Plugin Index