Chapter 08Lesson 04~165 minutes

workflow:rules, Job rules, if Expressions, changes, exists, Pipeline Sources, and Conditional Execution: Diagnostics, Failure Modes, Security, and Performance

Most conditional-pipeline failures are not YAML failures: GitLab accepted the configuration but evaluated it differently from the author's mental model. This lesson diagnoses duplicate branch and merge-request pipelines, broad final rules, unvalidated user-controlled variables, wrong pipeline-source assumptions, and changes semantics that surprise scheduled/manual/tag pipelines.

DiagnosticsFailure modesSecurityRule orderPerformance

Learning objectives

  • Diagnose rule failures from preserved pipeline identity and compiled configuration before retrying or editing.
  • Recognize duplicate branch/MR pipelines, over-broad final rules, user-variable trust mistakes, wrong source matches, and changes edge cases.
  • Separate rule/configuration problems from queue, runner, script, artifact, deployment, API, and policy layers.
  • Repair the narrow causal rule without hiding the original evidence or creating a new uncontrolled pipeline class.
  • Measure rule-driven cost/performance issues such as duplicated work and expensive jobs triggered by overly broad diffs.

1. Evidence-first diagnostic sequence

Do not start by editing the rule that “looks wrong.” First preserve the pipeline and configuration evidence so you can distinguish a rule defect from a runner/script failure:

  1. Preserve pipeline ID(s), job IDs, timestamps, source/ref/SHA, and the first unexpected graph.
  2. Confirm the merged/compiled configuration for that revision.
  3. Confirm workflow:rules pipeline-creation behavior and job first-match behavior using effective non-secret inputs.
  4. Only if the job exists, inspect queue/runner/executor/image state.
  5. Then inspect script/tool/network, reports/artifacts/caches, and any environment/deployment/external state.
  6. Apply the smallest rule correction and create a fresh pipeline in a controlled context. Preserve the original failure evidence.

2. Failure mode: duplicate branch + merge-request pipelines

Symptom: one push to an MR source branch yields two pipeline IDs for the same commit SHA. One source is push; the other is merge_request_event. Jobs may run twice and external statuses can become ambiguous.

Broken configuration:

workflow:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_PIPELINE_SOURCE == "push"'

verify:
  script: ./ci/verify.sh
  rules:
    - if: '$CI_COMMIT_BRANCH'
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'

Both pipeline classes are allowed, and the job can also be included in both. Preserve both pipeline IDs before repair. Then suppress redundant push pipelines only when the branch has an open MR:

workflow:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS && $CI_PIPELINE_SOURCE == "push"'
      when: never
    - if: '$CI_PIPELINE_SOURCE == "push"'
    - when: never

3. Failure mode: final catch-all creates unwanted jobs

Symptom: a job intended for MRs and schedules appears in manual, API, or other admitted pipeline classes. The common cause is a final rule such as - when: always. A rule with only when always matches when reached.

# Broken
rules:
  - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
  - if: '$CI_PIPELINE_SOURCE == "schedule"'
  - when: always

# Repaired explicit behavior
rules:
  - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
  - if: '$CI_PIPELINE_SOURCE == "schedule"'
  - when: never

The repaired final rule documents exclusion. If the pipeline itself should never exist for the other sources, move the policy up into workflow:rules instead of repeating exclusion on every job.

4. Failure mode: trusting an unvalidated user variable

Symptom: a manually/API-supplied variable causes a privileged job to appear. Pipeline variables are intent data, not proof of authorization. The job may still have access to protected runner/network/secret state if the surrounding trust controls are weak.

# Unsafe as the only gate for a privileged action
production_deploy:
  script: ./deploy.sh
  rules:
    - if: '$RUN_PRODUCTION == "true"'
Repair: combine intent with trusted source/ref/environment authorization. For example, restrict inclusion to the protected default branch and then rely on protected-environment/user permissions for the deployment authorization itself. Do not encode or print secret credentials in rules.

5. Failure mode: rule matches the wrong pipeline source

Symptom: a downstream job expected in a child pipeline never appears because the author checked for push. In a child pipeline the source is parent_pipeline; in a multi-project downstream pipeline it is pipeline. A trigger-token pipeline reports trigger, while Pipelines API reports api.

Repair by recording the actual source first. Do not broaden the rule to “any source” just to make the job appear.

6. Failure mode: changed-files assumptions fail

Symptom: an expensive job runs in a scheduled/manual/tag/new-branch context even though no relevant file “changed.” With no ordinary push event, unqualified rules:changes can evaluate true. Another failure is comparing against an unintended moving baseline.

# Broken for broad multi-source use
expensive_scan:
  script: ./scan.sh
  rules:
    - changes:
        - services/**/*

# Source-aware repair
expensive_scan:
  script: ./scan.sh
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
      changes:
        paths:
          - services/**/*
    - if: '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH'
      changes:
        compare_to: 'refs/heads/main'
        paths:
          - services/**/*
    - when: never

7. Failure mode: using exists for runtime evidence

Symptom: a job rule checks for dist/report.json expecting a prior build job to create it, but the job is always omitted. rules:exists examines repository state during configuration evaluation, not job artifacts. The causal repair is to include the downstream job through configuration and use needs/artifacts or script-level file validation at runtime. Chapter 09 and the artifact chapters develop that dataflow.

8. Controlled broken example: interpret before repair

Use only a disposable branch. Introduce this deliberately flawed policy:

workflow:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - when: always

heavy_check:
  script: echo "synthetic heavy check"
  rules:
    - changes:
        - src/**/*
    - when: always

Now create an MR and push once. Preserve all pipeline IDs for the SHA. Explain two independent problems:

  1. Pipeline layer: the workflow catch-all admits both the MR pipeline and the push pipeline.
  2. Job layer: the job catch-all includes heavy_check even when src/**/* does not match.

Repair each layer separately, then create a new commit/pipeline. Do not delete the original pipelines; they are your evidence that the repair changed the intended state.

9. Security and disruptive-action guardrails

Shortcut Why it is unsafe Safer response
Add a broad final allow rule until the job appears Widens sources/trust/cost without explaining cause Record actual source and add the narrow intended condition
Print every variable to debug an if expression Can expose tokens/secrets/protected data Print only safe predefined metadata and inspect settings without values
Use a broad PAT to call API and inspect pipelines Long-lived identity with excessive permissions Use UI/read-only authorized tooling or narrow job/API identity when taught
Retry all pipelines after a rule fix Can repeat deployments/releases/external side effects Create the smallest safe fresh pipeline or rerun only controlled idempotent scope
Disable protected controls to make a rule work Conflates inclusion with authorization Fix conditional inclusion and keep authorization boundary intact

10. Conditional execution is a performance control

Rules can reduce cost only if they are correct. Measure duplicate-pipeline count per SHA, runner queue time, expensive-job invocation frequency, diff size, and critical path before optimizing. A complicated changes policy that occasionally admits every expensive job can cost more than a simple deterministic pipeline.

Current GitLab docs impose bounded checks for patterned changes and exists; at large scales patterns can assume a match after limits are exceeded. Treat massive monorepo rule sets as architecture requiring measurement and version-aware testing, not as free filtering.

11. Compact diagnostic playbook

No pipeline

Inspect workflow, source/ref/SHA, CI Lint and pipeline-creation evidence.

Pipeline, no job

Inspect first-match job rules, changed/existing paths, effective non-secret inputs.

Job pending

Rules already succeeded; move to runner tags/scope/status.

Job failed

Rules and routing succeeded; inspect shell/tool/network/identity.

Duplicate IDs

Compare source values for same SHA and repair workflow boundary.

Unexpected expensive job

Preserve source + diff baseline + matched-rule hypothesis before narrowing.

Knowledge check

Two pipeline IDs share the same SHA, sources push and merge_request_event. What is the first repair layer?

A job with a final - when: always appears in an API pipeline. Why?

Why is RUN_PRODUCTION=true not sufficient authorization?

A child-pipeline job checks CI_PIPELINE_SOURCE == push. What source should you investigate?

Why preserve the original duplicate pipelines after fixing YAML?

Version and compatibility note

GitLab and GitLab Runner evolve continuously. Treat version-sensitive YAML, runner/executor behavior, APIs, security features, policy controls, tiers, and deprecations as assumptions to verify against the current official GitLab documentation before production use. Preserve the exact project/ref/SHA, compiled configuration, pipeline/job IDs, runner/tool versions, and external target evidence used for any reproducible lab or incident record.

Official references and version notes

Documentation verification date: 2026-09-11. GitLab CI/CD rule semantics evolve; self-managed installations should confirm their deployed GitLab version before relying on newer syntax.

Next lesson

Checkpoint Lab — workflow:rules, Job rules, if Expressions, changes, exists, Pipeline Sources, and Conditional Execution

Build a checkpoint ruleset, prove branch/MR/schedule behavior, inject a duplicate-pipeline defect, and produce a reviewable evidence packet.

Keep the academy open

Support free, practical DevOps education.

Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.

Ethereum / ERC-20
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0 Send only Ethereum/ERC-20 compatible assets to this address.