Chapter 16Lesson 04~180 minutes

Merge Request Pipelines, Merged Results Pipelines, Merge Trains, and Pre-Merge Validation: Diagnostics, Failure Modes, Security, and Performance

Failures in pre-merge CI are frequently identity or trust failures: the wrong ref was tested, two pipelines raced, an untrusted fork reached parent resources, merged-results assumptions were unavailable, or a flaky train failure was retried without preserving evidence. This lesson diagnoses those layers causally.

DiagnosticsWrong refDuplicate pipelinesProtected resourcesFlakiness

Learning objectives

  • Diagnose wrong-ref validation, protected-resource exposure, duplicate pipelines, unavailable merged-results behavior, and flaky merge-train failures.
  • Preserve original pipeline/job IDs, MR IID, source/target metadata, event type, and first-failure logs before retrying.
  • Separate rule/configuration failures from runner/tool failures, protected-resource authorization, mergeability, and integration-state failures.
  • Apply least-destructive repairs such as changing workflow rules or resource exposure rather than bypassing merge checks.
  • Recognize when throughput tuning, parallel train capacity, or retries can amplify cost or conceal nondeterministic tests.

1. Evidence-first diagnostic sequence

Use the same state ordering introduced earlier in the course, with MR identity inserted explicitly:

  1. Preserve MR IID, pipeline ID, job IDs, first-failure logs, timestamps, and reports.
  2. Confirm pipeline source, exact ref/SHA, MR source/target projects/branches, and event type.
  3. Inspect compiled configuration and the workflow/job rules that created/included work.
  4. Inspect graph/queue and runner/executor/image identity.
  5. Inspect the failing script/tool/network.
  6. Inspect reports/artifacts/caches.
  7. Inspect mergeability/protected-resource authorization and any external system touched.
  8. Apply the least destructive correction and rerun only the smallest safe scope.
Never start by rebasing or clicking retry repeatedly. Both actions can make the original candidate harder to reconstruct.

2. Failure mode: pipeline tests the wrong ref

Symptom: a developer says “the MR passed,” but the pipeline has source push and ref glci/ch16-feature; the intended MR-specific jobs are absent.

Evidence: pipeline source is push, MR IID/ref variables are absent, and compiled rules show the MR job was omitted. The pipeline may be healthy—it is simply the wrong evidence object.

Repair: configure root workflow:rules or job rules for merge_request_event and make the merge requirement depend on the intended MR pipeline/check rather than a same-named branch result.

3. Failure mode: untrusted fork code reaches parent resources

Symptom: a parent-project member runs a fork MR pipeline in the parent context and the CI diff includes commands that enumerate or transmit sensitive data.

Stop condition: do not run the pipeline merely to “see what happens.” Review CI changes first. Parent-context fork execution can use parent resources/settings and the triggering member’s permissions.

Repair: keep fork execution in the fork for ordinary checks, remove secret requirements from validation, restrict runner/network reachability, and require explicit trusted review before any parent-context execution.

Do not print or transform a masked value to test masking. Masking is a log-safety aid, not a data-loss-prevention boundary.

4. Failure mode: duplicate branch and MR pipelines

Symptom: one push to an open MR creates two pipelines with the same source commit: one push, one merge_request_event.

Causal layer: pipeline creation. Look for a broad workflow:rules fallthrough or job rules that admit both sources.

# Broken: the final rule admits many non-MR pipelines.
workflow:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - when: always

# Safer allowlist for this course scenario.
# workflow:
#   rules:
#     - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
#     - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
#     - when: never

Preserve both original pipeline IDs. The duplicate itself is useful evidence that the creation rule was too broad.

5. Failure mode: merged-results assumptions are unavailable

Symptom: documentation or a copied pipeline expects CI_MERGE_REQUEST_SOURCE_BRANCH_SHA/TARGET_BRANCH_SHA to be populated, but they are empty and the pipeline displays as a normal MR pipeline.

Diagnosis: inspect tier and project settings. Those SHAs are documented as populated in merged-results pipelines, not ordinary MR pipelines. On Free, the expected behavior is an ordinary source-only MR pipeline.

Repair: branch logic on the actual event type and feature availability; use the Free local integration simulation when teaching; do not fabricate missing values or downgrade security to obtain them.

6. Failure mode: merge train hides a flaky integration failure

Symptom: a candidate fails once on the train, passes after retry, and the team labels the first failure “transient” without preserving candidate identity or failure signature.

Diagnosis: compare pipeline SHA/event type, job trace, runner identity, test seed/shard data, and queue state. If the candidate changed because the train re-created the integration commit, the two runs are not equivalent evidence.

Repair: quarantine or fix nondeterministic tests with ownership and expiry, preserve first failure, and use bounded retries only for classified transient infrastructure errors.

7. Failure mode: green pipeline but MR still cannot merge

Do not debug the runner. Check merge conflicts, approvals, unresolved discussions, draft state, required checks, branch protection, and train status. A green pipeline is one input to the mergeability decision.

Conversely, if the UI offers merge but the policy expects a particular pre-merge pipeline, verify the requirement configuration rather than assuming GitLab inferred it from job names.

8. Intentionally broken example: wrong-source acceptance

The following configuration appears to “validate everything,” but it allows both branch push and MR pipelines. The release team then reads whichever green pipeline is easiest to find.

workflow:
  rules:
    - if: '$CI_COMMIT_BRANCH'
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'

pre_merge_test:
  image: alpine:3.22.1
  script:
    - test -f src/app.conf

The first rule matches branch pipelines before the MR rule is relevant in that context. The repair is not to rename the job; it is to define which pipeline sources are valid evidence and enforce that at creation time. Then the merge gate can depend on a stable check produced in that intended context.

9. Performance: optimize the correct state

Measure these independently:

Metric What it diagnoses Wrong “fix”
Pipeline duplication rate Rule/source design Adding runners
MR time to first failure Fast-check critical path Skipping required integration checks
Queue time Runner capacity/routing Increasing train size blindly
Train revalidation count Queue churn/target changes Blind retries
Flake retry rate Test nondeterminism Treating retries as success criteria
Merged-result duration Integration suite cost Moving privileged checks to branch pipelines

10. Security-sensitive actions checklist

  • Running fork code in a parent project: privileged trust decision.
  • Enabling protected variables/runners for MR pipelines: privileged resource-exposure decision.
  • Enabling merged results or merge trains: changes merge validation semantics.
  • Changing “pipelines must succeed,” required checks, or branch protection: governance change.
  • Skipping/bypassing a merge train: exceptional policy action, not a routine recovery technique.

Document owner, reason, scope, and rollback for each change.

11. Least-destructive recovery map

Failure Smallest safe correction
Wrong source pipeline Fix workflow/job source rules; create a new intended pipeline
Duplicate pipeline Narrow pipeline creation rules; preserve duplicates as evidence
Fork resource exposure risk Keep execution in fork or remove privileged resources; review CI before parent run
Merged-results unavailable Use ordinary MR pipeline + labeled local integration simulation, or upgrade/enable in authorized project
Flaky merge-train job Classify/fix flake; bounded retry only when justified; preserve first failure
Green-but-unmergeable MR Inspect merge policy/approvals/conflicts/train state rather than rerunning healthy CI

Knowledge check

A pipeline is green but CI_PIPELINE_SOURCE is push. Can it prove an MR-only gate passed?

Why is a parent-run fork pipeline security-sensitive even if the fork is public?

What should you do before retrying a merge-train failure?

Why do empty source/target branch SHA variables not automatically indicate a GitLab bug?

What layer owns duplicate pipeline creation?

Next lesson

Checkpoint lab

Build a minimal pre-merge contract, capture a wrong-source/duplicate failure, repair it, and prove the exact successful candidate without relying on paid features.

Version and compatibility note

GitLab and GitLab Runner evolve continuously. Treat version-sensitive YAML, runner/executor behavior, APIs, security features, policy controls, tiers, and deprecations as assumptions to verify against the current official GitLab documentation before production use. Preserve the exact project/ref/SHA, compiled configuration, pipeline/job IDs, runner/tool versions, and external target evidence used for any reproducible lab or incident record.

Official references and version notes

Documentation verification date: 2026-09-11. Merge-request pipeline behavior, protected-resource access, merged-results pipelines, merge trains, predefined variables, and merge checks are version-sensitive. Re-check the deployed GitLab version and project settings before using these patterns in production.

Keep the academy open

Support free, practical DevOps education.

Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.

Ethereum / ERC-20
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0 Send only Ethereum/ERC-20 compatible assets to this address.