Parent-Child Pipelines, Multi-Project Pipelines, Trigger Tokens, and Pipeline Composition: Diagnostics, Failure Modes, Security, and Performance

1. Diagnostic sequence: preserve the edge before changing it

1. Record upstream project, pipeline ID/source/ref/SHA/user and trigger-job ID/status.
2. Record downstream project/ref/pipeline ID/source/status if creation occurred.
3. Classify the edge: local child include, multi-project YAML trigger, CI_JOB_TOKEN API call, or trigger token API call.
4. Inspect strategy, target permissions, job-token allowlist/fine-grained permissions, accepted pipeline source rules, and only the variables/inputs needed to explain behavior.
5. Make the least destructive correction and create a new verification pipeline.
6. Preserve the failed pipeline; do not retry/delete it merely to make dashboards green.

2. Intentionally broken example: trigger job stays pending

Symptom: the trigger job sits pending, so an operator starts debugging runner tags.

Correction in mental model: trigger jobs do not use runners. Pending here means GitLab is trying to create the downstream pipeline and cannot complete that control-plane action.

UPSTREAM_PROJECT_ID="12345678"
PIPELINE_ID="9100"

glab api "projects/$UPSTREAM_PROJECT_ID/pipelines/$PIPELINE_ID/trigger_jobs"   --jq '.[] | {id,name,status,failure_reason,downstream_pipeline}'

Next inspect the target project/ref, CI configuration validity, user permission, and target workflow:rules. Do not register another runner.

3. Failure mode: cross-project job token receives 404/not-found authorization response

Current GitLab often intentionally returns 404 Not Found for unauthorized job-token resource/API access. Preserve the HTTP status/body and diagnose:

• Is the source project/group on the target project's inbound job-token allowlist?
• Is the pipeline user a member of the target and authorized for the requested action/ref?
• Did fine-grained permissions remove the trigger endpoint family?
• Is the target ref real and allowed by workflow:rules?

glab api "projects/$TARGET_PROJECT_ID/job_token_scope" --jq '{inbound_enabled}'
glab api "projects/$TARGET_PROJECT_ID/job_token_scope/allowlist" --paginate   --jq '.[] | {id,path_with_namespace}'

Least-privilege repair: add only the required source project (or narrow group when justified), retain the allowlist, ensure the actor already has required target permission, and grant only required fine-grained API families when that mode is enabled.

4. Allowlist repaired but trigger still fails: authorization is separate

An allowlist entry does not grant Developer/Reporter/Maintainer access. If the user who started the source pipeline cannot run pipelines in the target project or access the chosen protected ref, the job token still cannot perform the action.

Fix membership/ref authorization according to actual ownership policy. Do not raise the source user to Owner merely to clear an error; identify the minimum target role/ref permission required.

5. Failure mode: trigger token leaks into repository or logs

Containment first: revoke the exact trigger token in Settings > CI/CD > Pipeline trigger tokens. A revoked token cannot be restored. Only after revocation should you remove the value from Git/logs, assess jobs/pipelines created with it, and create a replacement if the integration still needs one.

Because a trigger token impersonates its creator's project access, review whether the leaked token could have created pipelines that received protected variables or deployment authority. A masked variable is not a reason to delay revocation.

6. Failure mode: parent succeeds while downstream fails

Symptom: upstream is green, downstream is red.

Cause: default trigger semantics mark the trigger job successful once the downstream pipeline is created. The system did exactly what was configured.

# Loose coupling: creation success only.
trigger_downstream:
  trigger:
    project: platform/downstream

# Correct when downstream result is part of upstream correctness.
trigger_downstream_mirrored:
  trigger:
    project: platform/downstream
    strategy: mirror

Do not add allow_failure to hide the mismatch. Decide whether coupling is actually required, then encode it.

7. Failure mode: legacy strategy: depend assumptions

Current GitLab explicitly recommends mirror rather than depend. If a pipeline inherited depend, reproduce its current manual/allowed-failure behavior before migrating. Then change one trigger edge at a time and compare status timelines.

8. Failure mode: sensitive variable crosses project boundary

Symptom: downstream logs reveal a value the upstream project treated as masked.

Cause: ordinary variable forwarding carries the value, not the upstream masking configuration. A downstream job can therefore expose it.

Response: if the value is a credential, revoke/rotate it first. Then remove the forwarding path, move credential ownership downstream, and replace broad variable forwarding with typed non-secret inputs.

9. Forwarded variable unexpectedly overrides downstream configuration

Values forwarded with trigger:forward are pipeline variables with high precedence. A downstream variable of the same name may be overridden. Inspect the trigger job's forward settings and rename configuration inputs to avoid ambient collisions.

10. Failure mode: recursive or duplicate triggers create a pipeline storm

Common causes include A→B→A trigger cycles, webhook responses to pipeline events, or rules that accept both push and downstream pipeline sources for the same mutation. Freeze the source of new triggers first by disabling the disposable trigger job/webhook or adding a narrow source guard; do not bulk-delete running pipelines before preserving IDs and causes.

# Example downstream guard: accept only the intended source.
workflow:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "pipeline"'
    - when: never

A trigger-token pipeline has source trigger, so this guard intentionally rejects it. Source policy must match the architecture you intend.

11. Failure mode: invalid or ambiguous downstream ref

A multi-project trigger can fail because the target branch/tag does not exist or because a tag and branch share the same name. Verify target refs read-only before editing orchestration. For trigger-token API calls, a nonexistent ref returns a 400-style error.

12. Failure mode: child job checks for merge_request_event

Inside a child pipeline, CI_PIPELINE_SOURCE remains parent_pipeline even when the parent was an MR pipeline. To specialize child jobs for MR context, use the inherited CI_MERGE_REQUEST_* variables such as CI_MERGE_REQUEST_ID, not CI_PIPELINE_SOURCE == "merge_request_event".

13. Failure mode: downstream cannot fetch upstream artifact

First verify tier: explicit upstream artifact retrieval with needs:pipeline:job/needs:project is Premium/Ultimate. Then verify producer job success, pipeline/ref identity, target/source job-token allowlist direction, and user access. For MR pipelines, use the MR ref path rather than assuming the branch pipeline artifact is correct.

14. Security and performance are causal here

Every downstream edge can multiply runner work and broaden where code/variables execute. A fan-out storm consumes runner capacity and may invoke external systems repeatedly. A cross-project trigger can expose private project name/status metadata upstream. A leaked persistent trigger token can create unscheduled pipelines. These are direct effects of composition, so limits, source rules, short-lived identities, and clear owners are reliability controls as well as security controls.

15. Composition incident runbook

1. Stop creation of NEW downstream pipelines without erasing existing evidence.
2. Capture upstream pipeline + trigger-job IDs, source/ref/SHA/user.
3. Capture downstream pipeline/project/ref/SHA/source/status if created.
4. Classify trigger identity: keyword | CI_JOB_TOKEN | trigger token.
5. If persistent credential leaked: revoke first.
6. Inspect target allowlist + user/ref permission + workflow source rules.
7. Inspect strategy and variable/input forwarding.
8. Repair the narrowest edge.
9. Run one verification pipeline and compare IDs/status.
10. Remove temporary allowlist/token/config changes by exact ID/path.

Knowledge check

Summary

Downstream diagnostics begin at the trigger edge, not the runner. Preserve source/ref/SHA/user/status, classify the trigger identity, inspect target allowlists and permissions separately, and understand strategy semantics before changing anything. Revoke leaked persistent credentials first, keep sensitive variables inside their owning project, and stop trigger loops at their source.

Official references

• GitLab Docs — Downstream pipelines
• GitLab Docs — Pipeline architectures
• GitLab Docs — CI/CD YAML trigger reference
• GitLab Docs — Trigger pipelines with the API
• GitLab Docs — CI/CD job token
• GitLab Docs — Job token scope API
• GitLab Docs — Fine-grained job-token permissions
• GitLab Docs — Pipelines API
• GitLab Docs — Jobs API / trigger jobs
• GitLab Docs — CI/CD inputs
• GitLab Docs — Troubleshooting downstream pipelines
• GitLab Docs — Job artifacts troubleshooting
• GitLab Docs — Pipeline security
• GitLab Docs — Roles and permissions
• GitLab 19.3 — What’s new