Scripted Pipeline, Groovy Control Flow, node Blocks, Dynamic Logic, and When to Use Scripted Pipelines: Diagnostics, Failure Modes, Security, and Performance

1. Evidence-first diagnostic ladder

Scripted Pipeline is flexible enough that many failures look superficially similar. Preserve evidence before changing anything, then classify the causal layer.

1. Record job full name, build number/URL, cause, exact Jenkinsfile/source SHA, and first failure timestamp.
2. Record Jenkins core/Java and Pipeline: Groovy, Script Security, and Nodes/Processes versions.
3. Inspect queue item and node/label/executor eligibility if a node step is waiting.
4. Record agent/Remoting/workspace/toolchain state if work reached an agent.
5. Inspect the Pipeline stack trace and identify whether it is CPS/serialization, sandbox, step, shell/tool, credential, or external-system failure.
6. Preserve artifacts/logs/external IDs before retry, replay, restart, or cleanup.
7. Change only the smallest causal layer and rerun the smallest safe scope.

2. Failure: heavy computation on the controller

Pipeline: Groovy runs orchestration code in the controller process. A Jenkinsfile that parses a huge dataset, performs CPU-heavy sorting, or holds giant object graphs can compete with scheduling and other Pipeline work.

// Poor design: large project computation inside Pipeline Groovy.
def rows = readFile('huge.csv').readLines()
def ranked = rows.collect { parseComplexRecord(it) }.sort { a, b -> b.score <=> a.score }
echo "top=${ranked.take(100)}"

The repair is not “add more controller heap” first. Move the computation to an agent-side program:

node('lab-linux') {
  checkout scm
  sh './tools/rank-data huge.csv > ranked.json'
  archiveArtifacts artifacts: 'ranked.json'
}

Measure controller CPU/heap/GC and Pipeline latency before and after if performance is the incident.

3. Intentionally broken example: non-serializable state across a suspension

A regex Matcher is a classic example of an object you should not retain across Pipeline suspension.

def matcher = ('release-42' =~ /release-(\d+)/)
sleep 2
echo "release=${matcher[0][1]}"

At or around the suspension/persistence point, Jenkins may report a serialization failure such as java.io.NotSerializableException: java.util.regex.Matcher. Preserve the stack trace and exact Jenkinsfile SHA.

Repair the data model, not the symptom:

def releaseId = ('release-42' =~ /release-(\d+)/)[0][1].toString()
sleep 2
echo "release=${releaseId}"

Only the serializable string survives. Do not mark the entire Pipeline @NonCPS; that would violate Pipeline step semantics.

4. Failure: CPS method mismatch

A common mistake is calling Pipeline steps from @NonCPS code:

@NonCPS
def wrongHelper() {
  echo 'This is a Pipeline step and does not belong here.'
}
wrongHelper()

The fix is to return data from the pure helper and call the Pipeline step from CPS-transformed code:

@NonCPS
def messageText() {
  return 'computed safely'
}
echo messageText()

5. Failure: dynamic logic hides stage identity

If every component is processed inside one generic stage, a failure log may say only “Build failed” with no component boundary. Improve evidence by using validated, bounded stage names and recording component identity in artifacts/logs.

components.each { component ->
  stage("Test ${component}") {
    node('lab-linux') {
      writeFile file: "evidence/${component}.txt", text: "component=${component}\n"
      sh "./test-component '${component}'"
    }
  }
}

6. Failure: sandbox rejection

Suppose a Jenkinsfile tries to call an internal Jenkins singleton or filesystem API and Script Security rejects it. The unsafe response is to approve whatever signature appears until the build turns green. The safe response is to ask why build code needs that authority.

7. Failure: node context mistaken for controller context

stage('Broken') {
  sh 'pwd'
}

In Scripted Pipeline, sh normally requires a node/workspace context. If no node is allocated, Jenkins reports a missing context rather than secretly running the shell on the controller. Repair by placing the external process inside the correct node block.

8. Failure: external side-effect identity lost across restart/rerun

Imagine a Scripted Pipeline creates synthetic release ID R-104, then waits or the controller restarts. A rerun that creates R-105 without checking existing state duplicates the side effect. Persist the release ID/digest as build evidence and reconcile the external target before repeating.

Pipeline durability preserves Jenkins program state; it cannot make an external API idempotent for you.

9. Failure taxonomy

10. Performance guardrails

• Keep Groovy “glue” small; run build logic in agent processes.
• Do not create unbounded loops, closures, dynamic stages, or parallel maps from untrusted input.
• Release node executors during long waits when workspace continuity is unnecessary.
• Archive only bounded evidence; do not keep massive object graphs in Pipeline variables.
• Measure queue wait, controller CPU/heap/GC, step duration, and artifact I/O before tuning.

11. Minimal incident packet

Capture: job/build/source identity; cause; queue/node/workspace evidence; first stack trace; controller and agent timestamps; installed Pipeline/Script Security versions; sandbox rejection if relevant; dynamic stage/component identity; archived artifacts; and any external side-effect ID. Preserve this before replay or repair.

Knowledge check

Official references and version notes

• Jenkins LTS changelog — current LTS and tested Java configurations.
• Jenkins Pipeline handbook — Scripted Pipeline fundamentals, node, stages, and Jenkinsfile concepts.
• Pipeline Syntax — Scripted control-flow examples and Pipeline syntax reference.
• Pipeline Best Practices — controller-side Groovy, serialization, and Pipeline scalability guidance.
• Pipeline CPS Method Mismatches — CPS transformation, @NonCPS, serialization, and mismatch failure modes.
• In-process Script Approval — Groovy Sandbox and script/signature approval security model.
• Pipeline: Groovy — current CPS execution engine implementation and compatibility.
• Script Security — current sandbox/approval plugin baseline and security history.
• Pipeline: Nodes and Processes — node, workspace, and durable external-process support.