workflow:rules, Job rules, if Expressions, changes, exists, Pipeline Sources, and Conditional Execution: Diagnostics, Failure Modes, Security, and Performance
Most conditional-pipeline failures are not YAML failures: GitLab accepted the configuration but evaluated it differently from the author's mental model. This lesson diagnoses duplicate branch and merge-request pipelines, broad final rules, unvalidated user-controlled variables, wrong pipeline-source assumptions, and changes semantics that surprise scheduled/manual/tag pipelines.
Learning objectives
- Diagnose rule failures from preserved pipeline identity and compiled configuration before retrying or editing.
- Recognize duplicate branch/MR pipelines, over-broad final rules, user-variable trust mistakes, wrong source matches, and changes edge cases.
- Separate rule/configuration problems from queue, runner, script, artifact, deployment, API, and policy layers.
- Repair the narrow causal rule without hiding the original evidence or creating a new uncontrolled pipeline class.
- Measure rule-driven cost/performance issues such as duplicated work and expensive jobs triggered by overly broad diffs.
1. Evidence-first diagnostic sequence
Do not start by editing the rule that “looks wrong.” First preserve the pipeline and configuration evidence so you can distinguish a rule defect from a runner/script failure:
- Preserve pipeline ID(s), job IDs, timestamps, source/ref/SHA, and the first unexpected graph.
- Confirm the merged/compiled configuration for that revision.
-
Confirm
workflow:rulespipeline-creation behavior and job first-match behavior using effective non-secret inputs. - Only if the job exists, inspect queue/runner/executor/image state.
- Then inspect script/tool/network, reports/artifacts/caches, and any environment/deployment/external state.
- Apply the smallest rule correction and create a fresh pipeline in a controlled context. Preserve the original failure evidence.
2. Failure mode: duplicate branch + merge-request pipelines
Symptom: one push to an MR source branch yields two
pipeline IDs for the same commit SHA. One source is
push; the other is merge_request_event.
Jobs may run twice and external statuses can become ambiguous.
Broken configuration:
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_PIPELINE_SOURCE == "push"'
verify:
script: ./ci/verify.sh
rules:
- if: '$CI_COMMIT_BRANCH'
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
Both pipeline classes are allowed, and the job can also be included in both. Preserve both pipeline IDs before repair. Then suppress redundant push pipelines only when the branch has an open MR:
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS && $CI_PIPELINE_SOURCE == "push"'
when: never
- if: '$CI_PIPELINE_SOURCE == "push"'
- when: never
3. Failure mode: final catch-all creates unwanted jobs
Symptom: a job intended for MRs and schedules
appears in manual, API, or other admitted pipeline classes. The
common cause is a final rule such as - when: always. A
rule with only when always matches when reached.
# Broken
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_PIPELINE_SOURCE == "schedule"'
- when: always
# Repaired explicit behavior
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_PIPELINE_SOURCE == "schedule"'
- when: never
The repaired final rule documents exclusion. If the pipeline itself
should never exist for the other sources, move the policy up into
workflow:rules instead of repeating exclusion on every
job.
4. Failure mode: trusting an unvalidated user variable
Symptom: a manually/API-supplied variable causes a privileged job to appear. Pipeline variables are intent data, not proof of authorization. The job may still have access to protected runner/network/secret state if the surrounding trust controls are weak.
# Unsafe as the only gate for a privileged action
production_deploy:
script: ./deploy.sh
rules:
- if: '$RUN_PRODUCTION == "true"'
5. Failure mode: rule matches the wrong pipeline source
Symptom: a downstream job expected in a child
pipeline never appears because the author checked for
push. In a child pipeline the source is
parent_pipeline; in a multi-project downstream pipeline
it is pipeline. A trigger-token pipeline reports
trigger, while Pipelines API reports api.
Repair by recording the actual source first. Do not broaden the rule to “any source” just to make the job appear.
6. Failure mode: changed-files assumptions fail
Symptom: an expensive job runs in a
scheduled/manual/tag/new-branch context even though no relevant file
“changed.” With no ordinary push event, unqualified
rules:changes can evaluate true. Another failure is
comparing against an unintended moving baseline.
# Broken for broad multi-source use
expensive_scan:
script: ./scan.sh
rules:
- changes:
- services/**/*
# Source-aware repair
expensive_scan:
script: ./scan.sh
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
changes:
paths:
- services/**/*
- if: '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH'
changes:
compare_to: 'refs/heads/main'
paths:
- services/**/*
- when: never
7. Failure mode: using exists for runtime evidence
Symptom: a job rule checks for
dist/report.json expecting a prior build job to create
it, but the job is always omitted.
rules:exists examines repository state during
configuration evaluation, not job artifacts. The causal repair is to
include the downstream job through configuration and use
needs/artifacts or script-level file validation at
runtime. Chapter 09 and the artifact chapters develop that dataflow.
8. Controlled broken example: interpret before repair
Use only a disposable branch. Introduce this deliberately flawed policy:
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- when: always
heavy_check:
script: echo "synthetic heavy check"
rules:
- changes:
- src/**/*
- when: always
Now create an MR and push once. Preserve all pipeline IDs for the SHA. Explain two independent problems:
- Pipeline layer: the workflow catch-all admits both the MR pipeline and the push pipeline.
-
Job layer: the job catch-all includes
heavy_checkeven whensrc/**/*does not match.
Repair each layer separately, then create a new commit/pipeline. Do not delete the original pipelines; they are your evidence that the repair changed the intended state.
9. Security and disruptive-action guardrails
| Shortcut | Why it is unsafe | Safer response |
|---|---|---|
| Add a broad final allow rule until the job appears | Widens sources/trust/cost without explaining cause | Record actual source and add the narrow intended condition |
| Print every variable to debug an if expression | Can expose tokens/secrets/protected data | Print only safe predefined metadata and inspect settings without values |
| Use a broad PAT to call API and inspect pipelines | Long-lived identity with excessive permissions | Use UI/read-only authorized tooling or narrow job/API identity when taught |
| Retry all pipelines after a rule fix | Can repeat deployments/releases/external side effects | Create the smallest safe fresh pipeline or rerun only controlled idempotent scope |
| Disable protected controls to make a rule work | Conflates inclusion with authorization | Fix conditional inclusion and keep authorization boundary intact |
10. Conditional execution is a performance control
Rules can reduce cost only if they are correct. Measure
duplicate-pipeline count per SHA, runner queue time, expensive-job
invocation frequency, diff size, and critical path before
optimizing. A complicated changes policy that
occasionally admits every expensive job can cost more than a simple
deterministic pipeline.
Current GitLab docs impose bounded checks for patterned
changes and exists; at large scales
patterns can assume a match after limits are exceeded. Treat massive
monorepo rule sets as architecture requiring measurement and
version-aware testing, not as free filtering.
11. Compact diagnostic playbook
Inspect workflow, source/ref/SHA, CI Lint and pipeline-creation evidence.
Inspect first-match job rules, changed/existing paths, effective non-secret inputs.
Rules already succeeded; move to runner tags/scope/status.
Rules and routing succeeded; inspect shell/tool/network/identity.
Compare source values for same SHA and repair workflow boundary.
Preserve source + diff baseline + matched-rule hypothesis before narrowing.
Knowledge check
Two pipeline IDs share the same SHA, sources push and merge_request_event. What is the first repair layer?
workflow:rules, because the defect is duplicate pipeline creation.
A job with a final - when: always appears in an API pipeline. Why?
That rule always matches if reached, so it acts as a catch-all for any admitted pipeline source.
Why is RUN_PRODUCTION=true not sufficient authorization?
It is user/control input; authorization must come from trusted identity/ref/environment policy, not a freely supplied value.
A child-pipeline job checks CI_PIPELINE_SOURCE == push. What source should you investigate?
parent_pipeline.
Why preserve the original duplicate pipelines after fixing YAML?
They are first-failure evidence proving the old configuration created the undesired state and allow comparison with the repaired pipeline.
Version and compatibility note
GitLab and GitLab Runner evolve continuously. Treat version-sensitive YAML, runner/executor behavior, APIs, security features, policy controls, tiers, and deprecations as assumptions to verify against the current official GitLab documentation before production use. Preserve the exact project/ref/SHA, compiled configuration, pipeline/job IDs, runner/tool versions, and external target evidence used for any reproducible lab or incident record.
Official references and version notes
Documentation verification date: 2026-09-11. GitLab CI/CD rule semantics evolve; self-managed installations should confirm their deployed GitLab version before relying on newer syntax.
-
workflowkeyword — pipeline-creation rules, duplicate-pipeline avoidance, branch-to-MR switching, and current source examples. -
Specify when jobs run with
rules— first-match evaluation,CI_PIPELINE_SOURCEvalues, expressions, schedules,changes, and duplicate-pipeline guidance. -
CI/CD YAML syntax reference
— authoritative
rules,rules:changes,compare_to,rules:exists,when, and workflow syntax. - Predefined CI/CD variables — variable availability phases and safe source/ref/SHA metadata.
-
Debugging CI/CD pipelines
— current guidance for unexpected
changesbehavior and duplicate pipelines. - Troubleshooting merge request pipelines — evidence and repair guidance for branch + MR duplicates.
Keep the academy open
Support free, practical DevOps education.
Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0
Send only Ethereum/ERC-20 compatible assets to this
address.