Checkpoint Lab — LTS Upgrades, Java Runtime Transitions, Plugin Upgrade Strategy, Compatibility Testing, and Rollback Planning

1. Mission

Use only disposable local resources. Start from a synthetic Jenkins 2.541.3 / Java 17 controller, migrate an isolated copy to 2.568.3 / Java 21, reconcile security-sensitive plugins, verify representative behavior, intentionally expose one Java-17 agent incompatibility, and prove you can return to the known-good pre-upgrade controller state.

2. Write the change record before execution

Jenkins upgrade checkpoint
==========================
Current core/runtime: 2.541.3 / Java 17
Target core/runtime:  2.568.3 / Java 21
Crossed LTS guides:   2.555.x, 2.568.x
Current plugin manifest: <path>
Target security-sensitive plugin minimums:
  script-security >= 1422.v06869826dd9b_
  pipeline-groovy-lib >= 806.v408277b_33d1d
  workflow-multibranch >= 842.v3a_b_59b_57b_e6e
Backup ID/checksum: <fill after snapshot>
Representative jobs: upgrade-lab/smoke + optional library/multibranch
Agent pools: lab-java17 -> lab-java21
Acceptance owner: <name/role>
Rollback trigger: <explicit criteria>
Rollback method: restore pre-upgrade state; never downgrade mutated target volume

3. Predict state changes before you perform them

Prediction A — Java boundary
The 2.568.3 controller will run on Java 21; any Jenkins agent process still on Java 17
will be rejected/offline until its runtime is upgraded. Application build JDK selection
is independent and must be tested separately.

Prediction B — persisted controller state
The target clone may migrate plugin/core configuration in its own volume. Therefore the
old controller must never be pointed at that upgraded volume during rollback; recovery
must use the preserved pre-upgrade volume/archive.

Prediction C — plugin security baseline
If Script Security / Groovy Libraries / Multibranch are installed below the 16 Sep fixed
versions, the compatibility gate remains failed even when smoke jobs execute successfully.

4. Execute the staged rehearsal

Follow Lesson 2 exactly: start loopback-only old controller, create representative synthetic state, capture inventory, read guides/advisories, stop, snapshot and checksum, restore to a separate target volume, then start 2.568.3/JDK21 on the clone.

Preserve these identifiers:

5. Trigger one controlled compatibility failure

Create or retain one disposable inbound agent whose Jenkins agent JVM is Java 17. Attempt connection after the target controller is running. Capture the node/agent log and classify the failure as an agent runtime compatibility problem. Then rebuild/relaunch the same disposable agent on Java 21 and verify it reconnects.

Do not use a production node, do not weaken controller security, and do not include the inbound secret in the evidence packet.

6. Acceptance suite

7. Demonstrate rollback/recovery

1. Stop the upgraded target controller.
2. Do not attach the old image to the target volume.
3. Start the old controller from the untouched old volume or a fresh restore of the pre-upgrade archive.
4. Verify core/Java/job/build identity.
5. Record the recovery timestamp and whether external state would require reconciliation in a real incident.
6. Stop the recovery controller and retain evidence until the checkpoint report is complete.

8. Make an explicit decision

Decision: ACCEPT / ROLLBACK / FORWARD-FIX / INCONCLUSIVE

Acceptance evidence:
- core/runtime:
- plugins/security advisories:
- configuration:
- pipelines/libraries:
- agents/remoting:
- external integrations:
- performance comparison:
- recovery proof:

Compatibility failure injected:
Root cause layer:
Repair evidence:

Residual risks / unsupported assumptions:
Rollback source retained until:

9. Required evidence packet

• Current and target core/Java versions plus installation/image identity.
• Before/after plugin manifests with security advisory note.
• Skipped LTS guide checklist.
• Backup archive ID/checksum and restore-test result.
• JCasC/configuration source revision or UI ownership note.
• Representative job/build/source/library revisions.
• Controller startup and acceptance logs, redacted.
• Agent names, JVM/launcher/labels and failure/recovery timestamps.
• Artifact/report digest from smoke test.
• Matched Chapter 38 performance snapshot.
• Rollback/recovery proof and residual external-state note.

10. Cleanup

After grading/review, remove only the exact disposable containers/volumes and guarded evidence directory. Keep a copy of the written upgrade report if it is your learning artifact, but strip secrets and ephemeral connection material.

set -euo pipefail
for c in jenkins-upgrade-old jenkins-upgrade-target jenkins-upgrade-rollback; do
  docker rm -f "$c" 2>/dev/null || true
done
for v in jenkins-upgrade-home-old jenkins-upgrade-home-target; do
  docker volume rm "$v" 2>/dev/null || true
done
LAB='/tmp/jenkins-upgrade-lab'
case "$LAB" in /tmp/jenkins-upgrade-lab) rm -rf -- "$LAB" ;; *) echo 'refusing cleanup' >&2; exit 70 ;; esac

11. What Chapter 39 adds to a production Jenkins operating model

You now have a repeatable upgrade discipline that treats Jenkins LTS, Java, plugins, agent runtimes, Pipeline code, external integrations, security advisories and recovery state as one governed compatibility system. A production upgrade becomes a rehearsed change with measurable gates rather than a maintenance-window experiment.

Chapter 40 uses the same evidence discipline during live incidents: thread dumps, support bundles, stuck builds, agent failures and production playbooks must preserve first-failure evidence before corrective action.

Knowledge check

Answer before revealing the explanation.

Official references and version notes

Upgrade behavior is version-specific. Always read every skipped LTS guide and the current security advisories for the exact day you plan an upgrade.

• Jenkins LTS Upgrade Guide
• Upgrading to Jenkins LTS 2.555.x
• Upgrading to Jenkins LTS 2.568.x
• Jenkins Java Support Policy
• Upgrade Jenkins runtime to Java 21
• Managing Jenkins plugins
• Using Jenkins agents
• Backing up Jenkins
• Jenkins LTS changelog
• Jenkins Security Advisories
• Jenkins Security Advisory 2026-09-16
• Jenkins Plugin Index