Groovy CPS, Serialization, @NonCPS, Pipeline State, Restarts, and Common Pipeline Programming Pitfalls: Configuration, Design Choices, and Tradeoffs
Choose durable Pipeline data structures and execution boundaries deliberately: CPS state versus native helpers, Jenkins steps versus ordinary libraries, @NonCPS versus external tools, and controller computation versus agent processes.
Learning objectives
- Select CPS-safe state representations instead of retaining complex runtime objects.
- Decide when logic belongs in a Pipeline step, a pure @NonCPS helper, or an external tool executed on an agent.
- Compare restartability, controller resource cost, auditability, and testability for each approach.
- Explain how durability settings trade persistence frequency for performance without correcting invalid program state.
- Use a decision table to justify a maintainable implementation with observable evidence.
1. Design for the execution model you actually have
The easiest CPS problems to fix are the ones you avoid by design. A Pipeline should carry identifiers and small decisions—not live service clients, parser iterators, giant object graphs, or mutable handles. Prefer durable values such as strings, numbers, booleans, and compact maps/lists. Convert native library results into that durable shape before the next suspension.
2. CPS-safe data versus complex objects
| Need | Prefer | Avoid retaining | Evidence |
|---|---|---|---|
| Build identity | String SHA / build number | SCM client/session object | SHA in artifact/log |
| Deployment intent | Map of immutable IDs | HTTP client / response stream | Operation ID + target query |
| Parsed report summary | Small numeric/string map | Parser tree / iterator | Archived raw report + compact summary |
| Regex extraction | Extracted String | Matcher | Input text hash + returned value |
3. Pipeline step versus Java/Groovy library call
Use a Pipeline step when Jenkins must orchestrate Jenkins-aware behavior: allocate a node, run an agent process, wait, archive evidence, request input, or interact through a plugin contract. Use an ordinary library call for tiny deterministic calculations that do not need Jenkins state. The mismatch risk appears when native code calls back into CPS-transformed closures or methods.
If a library API accepts a closure/comparator/callback, verify that the combination is supported by current Pipeline CPS semantics. Do not assume every Groovy idiom behaves like standalone Groovy.
4. @NonCPS helper versus external tool
| Criterion | Small @NonCPS helper | Agent/external tool |
|---|---|---|
| Work size | Small, bounded transformation | Large/CPU-heavy/data-heavy work |
| Pipeline steps inside | Not allowed | Called through sh/bat/tool step |
| Restart mid-computation | Method restarts from beginning if re-entered | Depends on process/step contract |
| Controller CPU | Consumes controller CPU | Consumes agent/tool resources |
| Non-serializable locals | Allowed locally if they do not escape | Local to external process |
| Best use | Compact conversion/sort/parse of small values | Build, test, large parsing, network client logic |
5. Controller-side computation versus agent-side script
Even a native @NonCPS method still runs in the
controller process. It may be faster than CPS-transformed Groovy,
but it is not free. If the task is CPU-heavy, memory-heavy, or
scales with repository size, offload it.
node('lab-linux') {
sh './scripts/compute-summary.sh raw-results.json > summary.env'
def summary = readProperties file: 'summary.env'
echo "failed=${summary.failed} total=${summary.total}"
}
The controller retains only two small values; the large raw data remains a workspace/artifact concern.
6. Durability mode: performance versus survivability
Pipeline durability controls how frequently transient Pipeline state is persisted. Maximum survivability writes more often; performance-optimized modes reduce I/O and increase the risk that a dirty shutdown loses recent running-Pipeline state. This is an operational policy decision, especially important for critical deployments.
7. Worked scenario: summarize a large test report
Requirement: a 150 MB JSON report must produce
passed, failed, and duration.
The run must survive controller restart and preserve the raw report.
| Approach | Tradeoff | Decision |
|---|---|---|
readFile + parse huge map in CPS Groovy |
Large controller state, serialization pressure | Reject |
Huge @NonCPS parser on controller |
No CPS serialization inside helper, but controller CPU/heap cost remains | Reject for this size |
| Agent-side parser + compact summary | Requires tool on agent; strong isolation | Preferred |
Evidence should include parser tool/version, source SHA, raw report artifact digest, summary file, and Pipeline build URL.
8. Sandbox and approval are part of architecture
Do not disable the Groovy sandbox to “make code work.” Unsandboxed Pipeline code can invoke powerful controller APIs and requires administrative approval. Prefer supported Pipeline steps and sandbox-safe code. If an organization intentionally approves a script, treat that approval like privileged code change: review source/revision, scope, owner, and rollback.
9. Decision checklist
- Can the value be represented as a small serializable primitive/list/map?
- Does the operation need a Pipeline step or Jenkins context?
- Does a native helper call back into CPS-transformed code?
- How much controller CPU/heap does the work consume?
- What happens if the controller restarts halfway through?
- What external state exists outside Jenkins durability?
- What evidence proves the chosen boundary behaved as predicted?
Knowledge check
When is @NonCPS preferable to an external
tool?
For small bounded native Groovy/Java transformations that need no Pipeline steps and can return a compact serializable result.
Why is a 150 MB parsed map a poor Pipeline variable?
It increases controller heap and persistence/serialization cost and makes restart state large and fragile.
Does @NonCPS move computation to an agent?
No. It changes CPS transformation semantics but still executes in the controller process.
What does a lower durability mode change?
How frequently running Pipeline state is persisted and therefore the performance/survivability trade-off; it does not fix invalid state.
Why is disabling the sandbox a design/security decision?
Unsandboxed scripts can access powerful controller APIs, so approval expands trust and must be reviewed like privileged code.
Official references and version notes
- Jenkins LTS changelog — current Jenkins LTS and tested Java configurations.
-
Pipeline CPS Method Mismatches
— CPS transformation boundaries,
@NonCPS, constructors, closures, and mismatch diagnostics. -
Pipeline: Groovy plugin
— Pipeline execution engine, persistence model,
program.dat, rehydration, and@NonCPScontract. -
Pipeline Best Practices
— controller/agent boundaries, serializable state, and safe use of
@NonCPS. - Scaling Pipelines — durability modes, persistence trade-offs, controller CPU/memory, and restart implications.
- Script Security plugin — Groovy sandbox and script-approval security boundary.
- Jenkins Pipeline handbook — durable, pausable Pipeline fundamentals.
Rechecked on 2026-09-16. Examples assume Jenkins 2.568.3 LTS (tested with Java 21 and 25), Pipeline: Groovy 4380.v6eb_8378b_9647, Pipeline: Supporting APIs 1015.v785e5a_b_b_8b_22, and Script Security 1422.v06869826dd9b_. The mandatory path is local/disposable, uses synthetic state and fake identities, and requires no commercial service. Plugin releases are independent of Jenkins core, so record the versions actually installed on your controller before applying these lessons.
Keep the academy open
Support free, practical DevOps education.
Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0Send only Ethereum/ERC-20 compatible assets to this
address.