Pipeline Fundamentals, Jenkinsfile, Pipeline Engine, Durable Execution, Nodes, Workspaces, and Stages: Configuration, Design Choices, and Tradeoffs

1. Pipeline definition: controller UI or source control?

An inline Pipeline script stored in a job is fast for a disposable experiment, but it lives in controller configuration and is harder to review alongside application changes. A Jenkinsfile in SCM provides commit history, pull-request review, branching, and a direct source identity for the automation definition.

For production delivery automation, Pipeline-as-Code is usually the stronger ownership model, but “stored in Git” does not by itself make a Jenkinsfile trusted. Later multibranch chapters handle untrusted pull-request boundaries in depth.

2. One global agent versus stage-level agents

A top-level agent is simple and gives stages one continuous workspace, but it can hold an executor during pauses or stages that do not need the same machine. agent none with per-stage agents can reduce idle capacity and select specialized environments, at the cost of explicit file transfer and potentially more queueing.

3. Workspace, stash, archive, and external artifact repositories

These mechanisms solve different lifetimes. Treating them as interchangeable is a common Pipeline design error.

4. Durability setting is a risk decision

Pipeline can trade disk I/O for survivability. Jenkins documentation describes maximum-survivability, survivable-nonatomic, and performance-optimized modes. A performance-oriented setting may be reasonable for repeatable build/test jobs; a deployment or infrastructure-changing Pipeline may justify stronger persistence because re-running after a crash can have more serious consequences.

Do not change durability only because a controller is slow. Measure controller I/O, Pipeline count, storage behavior, and actual restart/recovery requirements first. A durability setting affects the next applicable run, so document which run was created under which setting.

5. disableResume is an explicit semantic choice

Declarative Pipeline supports options { disableResume() }. That option means a controller restart should not resume the Pipeline. Use it only when rerunning from the beginning is the intended and safe recovery model. It is inappropriate to assume this is harmless for jobs with non-idempotent external side effects.

6. Stages are communication and control boundaries

Stages improve observability and can structure agent/tool/security boundaries, but a stage name is not an isolation mechanism by itself. If Build and Deploy both run on the same privileged agent with the same credentials, naming them separate stages does not establish least privilege.

Use stages to expose meaningful transitions—checkout, build, test, package, approval, publish—not to wrap every shell command in visual noise.

7. Worked scenario: build → test → approval → simulated promotion

This design avoids holding one executor through the approval and prevents “rebuild during release” from silently changing artifact identity.

8. Rollback is about definitions and side effects separately

Reverting a Jenkinsfile changes future runs. It does not undo already-published artifacts or external deployments. Keep a rollback plan for the Pipeline definition and a recovery/rollback plan for each external system the Pipeline mutates.

9. Decision checklist

• Where is the Pipeline definition versioned and reviewed?
• Which stages need executors/workspaces, and which can remain controller-only coordination?
• Which files are scratch, run-scoped transfer, retained build evidence, or released packages?
• What happens if the controller restarts during each stage?
• What happens if the agent disappears?
• What external state could already have changed?
• Which build/source/artifact identifiers prove what happened?

Knowledge check

Official references and version notes

• Jenkins LTS changelog — current LTS line and tested Java configurations.
• Jenkins Pipeline handbook — Pipeline concepts, Jenkinsfile, development tools, shared libraries, and execution model.
• Getting Started with Pipelines — durability, pausing, and the reasons Pipeline differs from Freestyle automation.
• Pipeline Syntax — Declarative pipeline, agent, stages, steps, options, and restart-related directives.
• Scaling Pipelines — Pipeline durability modes, persistence tradeoffs, and restart implications.
• Pipeline plugin — the aggregator and current Pipeline plugin suite.
• Pipeline: Declarative — Declarative Pipeline implementation and compatibility.
• Pipeline: Groovy — CPS execution model and controller-side Groovy interpretation.
• Pipeline: Job — persisted Pipeline run/job implementation.
• Pipeline: Nodes and Processes — node/workspace allocation and durable external process steps.
• Pipeline: Stage View — optional visualization; not the source of Pipeline execution truth.