rules, workflow, Pipeline Sources, Changes, Conditions, and Dynamic Pipeline Creation: Diagnostics, Failure Modes, Security, and Performance

1. Diagnostic sequence: preserve → scope → evaluate → repair → verify

1. Preserve evidence: configuration SHA, pipeline IDs, source/ref/SHA, MR state, and lint output.
2. Identify scope: did no pipeline exist, did the pipeline exist with a missing job, or did a job exist but fail/pending?
3. Inspect resolved configuration: includes and workflow can change behavior outside the local snippet.
4. Evaluate variables and rule order: use only documented creation-time variables for that source.
5. Inspect comparison context: target branch, previous commit, explicit compare_to, or no push event.
6. Choose the least destructive correction: do not add broad when: always, allow_failure, or runner privileges to hide a creation defect.
7. Verify independently: reproduce with the same source category and compare source/SHA/job set.

2. First classify the missing object

3. Broken example: branch and MR pipelines both run

A common anti-pattern is relying on job rules with a broad fallback while allowing both pipeline types to exist:

test:
  script: echo test
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - when: always

A push to a branch with an open MR can yield a branch pipeline and an MR pipeline. Preserve both pipeline IDs and sources. Repair the whole-pipeline policy with workflow:rules that switches from branch to MR pipelines, rather than adding increasingly complex exclusions to every job.

4. Broken example: using a variable unavailable in this source

mr_only:
  script: echo "MR"
  rules:
    - if: '$CI_COMMIT_BRANCH == "feature/demo"' 

If the intent is an MR job, this predicate is conceptually wrong because CI_COMMIT_BRANCH is not available in MR pipelines. Repair the source identity first:

rules:
  - if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == "feature/demo"' 

5. Broken example: changes assumed to mean “since last human edit”

A scheduled or manually started pipeline is configured with only:

nightly_backend:
  script: ./test-backend.sh
  rules:
    - changes:
        - backend/**/*

The author expects the job to skip because no one pushed backend code immediately before the schedule. That is not a defined comparison model for this source. For pipelines without an associated push, changes without compare_to can evaluate true. Repair by making the schedule intent explicit (often run it unconditionally) or specify a deliberate comparison base.

6. Broken example: the later rule can never win

deploy_review:
  script: echo deploy
  rules:
    - if: '$CI_COMMIT_BRANCH'
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
      when: never

On the default branch, the first rule already matches, so the deny rule is unreachable. Move the specific deny above the broad branch allow, or express the intended condition in one rule.

7. Broken example: generated child configuration is invalid

The generator job succeeds and uploads generated-child.yml, but the trigger job fails because the artifact is not valid GitLab CI configuration. Preserve the artifact rather than regenerating it immediately. Validate its YAML and GitLab CI schema exactly as you would a committed configuration.

# Intentionally invalid generated artifact fixture:
child_probe:
  scripts:                # wrong key; GitLab expects script
    - echo "hello"

Repair the generator template to emit script, regenerate under a new commit/pipeline, and retain the failed artifact/pipeline ID as evidence of the original cause.

8. Security failure: generated YAML trusts unreviewed input

A more serious defect is syntactically valid generated YAML derived from untrusted input that can select images, runner tags, includes, or commands. Do not “sanitize by regex and hope.” Define an allowlisted data model (for example component names from a committed manifest), generate only known YAML structures, and ensure untrusted pipelines cannot access protected variables/runners/environments.

9. Evidence bundle for a rule incident

• Commit SHA containing the CI configuration.
• Pipeline ID(s), source, ref, SHA, status, and creation timestamp.
• MR IID/source/target branch when relevant.
• CI Lint or merged-configuration result.
• The exact ordered rules involved, with the first matching rule identified.
• For changes, the intended and actual comparison base.
• For dynamic children, generator job ID, generated YAML artifact checksum/content, trigger job state, and child pipeline ID if created.

10. Performance and cost are causal, not generic warnings

Duplicate pipelines multiply jobs immediately. Broad changes/exists patterns also consume configuration-evaluation work; GitLab documents limits and fail-open matching behavior for very large match workloads. Dynamic child pipelines can reduce a huge static graph, but generation and additional pipeline orchestration have cost. Measure pipeline count and job count per source before optimizing syntax.

11. Symptom → smallest repair

Knowledge check

Summary

Creation-time debugging starts by identifying which object is missing or duplicated, then reading source/ref/SHA, resolved configuration, variable availability, rule order, and comparison base. Dynamic pipeline failures add one more artifact—the generated configuration—which must be preserved and reviewed as code.

Official references

• GitLab Docs — workflow keyword
• GitLab Docs — Specify when jobs run with rules
• GitLab Docs — CI/CD YAML syntax reference
• GitLab Docs — Predefined CI/CD variables
• GitLab Docs — Where variables can be used
• GitLab Docs — Merge request pipelines
• GitLab Docs — Downstream pipelines
• GitLab Docs — Pipeline editor
• GitLab Docs — CI Lint
• GitLab Docs — CI/CD pipelines
• GitLab Docs — Pipelines API
• GitLab Docs — Use CI/CD configuration from other files