Pipeline Fundamentals, Jenkinsfile, Pipeline Engine, Durable Execution, Nodes, Workspaces, and Stages: Guided Hands-On Workflow and Core Operations

1. Lab topology and safety boundary

Use the same disposable local Jenkins pattern from earlier chapters: one controller with persistent JENKINS_HOME, one non-controller agent labeled lab-linux, and a synthetic public/local Git repository containing only training files. The controller is reachable only on loopback or another explicitly private lab interface.

Record Jenkins core, Java, Pipeline/Declarative/Groovy/Job/Nodes-and-Processes versions before the experiment. Do not upgrade plugins as part of this lab.

2. Create the synthetic repository

Create a tiny repository with a text payload and a Jenkinsfile. Commit it and record the immutable SHA. If your lab uses a local Git service, keep it on the disposable network; a public throwaway repository also works when no credential is required.

mkdir jenkins-ch09-demo && cd jenkins-ch09-demo
git init
git config user.name "Jenkins Chapter 09 Lab"
git config user.email "jenkins-ch09@example.invalid"
printf 'chapter09-payload\n' > payload.txt
git add payload.txt
git commit -m 'ch09: add synthetic payload'
git rev-parse HEAD

The commit SHA is evidence input. Later, the Pipeline must report the same SHA after checkout.

3. Add a minimal durable Pipeline

The Pipeline uses a top-level lab agent for simplicity. It records identity, creates one small artifact, pauses with the Pipeline sleep step, then verifies state after the pause.

pipeline {
  agent { label 'lab-linux' }
  options { timestamps() }
  stages {
    stage('Observe') {
      steps {
        sh '''
          set -eu
          mkdir -p out
          {
            printf 'job=%s\n' "$JOB_NAME"
            printf 'build=%s\n' "$BUILD_NUMBER"
            printf 'build_url=%s\n' "$BUILD_URL"
            printf 'node=%s\n' "$NODE_NAME"
            printf 'workspace=%s\n' "$WORKSPACE"
            printf 'source_sha=%s\n' "$(git rev-parse HEAD)"
          } | tee out/before-restart.txt
        '''
      }
    }
    stage('Durable pause') {
      steps {
        echo 'Controller may be restarted during this safe pause.'
        sleep time: 3, unit: 'MINUTES'
      }
    }
    stage('Verify') {
      steps {
        sh '''
          set -eu
          test -f out/before-restart.txt
          printf 'resumed_build=%s\n' "$BUILD_NUMBER" | tee out/after-restart.txt
          sha256sum payload.txt > out/payload.sha256
        '''
      }
    }
  }
  post {
    always {
      archiveArtifacts artifacts: 'out/*', fingerprint: true
    }
  }
}

sleep is a Pipeline step, not a shell process. It is a deliberately safe pause for learning restart semantics. The later optional experiment can use a durable external process, but do not begin there.

4. Create “Pipeline script from SCM”

Create a disposable Pipeline job named ch09-durable-pipeline. Choose Pipeline script from SCM, select Git, set the synthetic repository URL and branch/ref, and leave the script path as Jenkinsfile. Use no credential for a public/local anonymous repository.

This configuration persists the location of the Pipeline definition; the run still needs the exact source revision as evidence. A branch name such as main is not immutable identity.

5. Start the run and capture pre-restart evidence

Trigger the job manually. In the console log and UI, confirm:

• one build number was created;
• the expected source SHA was checked out;
• lab-linux was allocated;
• the workspace path is visible in bounded evidence;
• the Observe stage completed;
• the run is currently in Durable pause.

Save the build URL/number before touching the controller. The test is about continuation of this exact run, not whether another run could succeed.

6. Restart only the disposable controller

While the Pipeline is in the sleep step, perform a controlled restart of the disposable controller. If it runs as the named lab container used in earlier chapters, the conceptual sequence is:

docker stop jenkins-ch09-controller
docker start jenkins-ch09-controller

Your actual container name may differ; use only the exact disposable controller identity you verified in preflight. Do not remove its persistent JENKINS_HOME volume, and do not restart unrelated Docker resources.

7. Verify the resumed run

Return to the original build URL. Confirm the run continues into Verify, and then inspect the archived evidence. Compare before-restart.txt and after-restart.txt: the build number must match. The workspace path may remain the same in this static-agent lab, but that is not the property being tested.

Also confirm the artifact archive survives independently of later workspace cleanup.

8. Prove workspace and archived evidence are different layers

After the successful build, copy/download the archived evidence from the Jenkins build record. Then, only on the disposable agent, remove the job workspace or use an authorized workspace cleanup operation. Re-open the build record and prove the archived files remain available.

9. Optional: durable external process observation

Once the safe pause is understood, you may replace the pause with a bounded external command such as sh 'sleep 180' on the disposable agent and repeat a controller restart while leaving the agent alive. Pipeline: Nodes and Processes is designed to monitor external processes across controller interruption/reconnection. Treat this as an observation, not a guarantee against agent/process loss.

10. Small challenge: choose the correct layer

Suppose the run resumes after controller restart but payload.txt is missing because the ephemeral agent disappeared. Which layer failed?

The controller Pipeline program survived; the execution workspace did not. A correct design would recreate source/workspace state on a replacement agent or restore needed run-scoped files from a durable mechanism. Restarting the controller again is not the repair.

11. Cleanup

Preserve the evidence packet, then delete only the disposable Chapter 09 job/repository/agent resources you created. Keep shared Jenkins controller/plugin/tool configuration intact. If you created a chapter-specific controller, stop and remove it only after preserving the evidence and after confirming the persistent volume is no longer needed.

Knowledge check

Official references and version notes

• Jenkins LTS changelog — current LTS line and tested Java configurations.
• Jenkins Pipeline handbook — Pipeline concepts, Jenkinsfile, development tools, shared libraries, and execution model.
• Getting Started with Pipelines — durability, pausing, and the reasons Pipeline differs from Freestyle automation.
• Pipeline Syntax — Declarative pipeline, agent, stages, steps, options, and restart-related directives.
• Scaling Pipelines — Pipeline durability modes, persistence tradeoffs, and restart implications.
• Pipeline plugin — the aggregator and current Pipeline plugin suite.
• Pipeline: Declarative — Declarative Pipeline implementation and compatibility.
• Pipeline: Groovy — CPS execution model and controller-side Groovy interpretation.
• Pipeline: Job — persisted Pipeline run/job implementation.
• Pipeline: Nodes and Processes — node/workspace allocation and durable external process steps.
• Pipeline: Stage View — optional visualization; not the source of Pipeline execution truth.