Parent-Child Pipelines, Dynamic Child Pipelines, Generated Configuration, and Monorepo Decomposition: Diagnostics, Failure Modes, Security, and Performance
Failures across a pipeline hierarchy are easy to misdiagnose because creation, status propagation, generated YAML, variable forwarding, artifacts, runners, and reports are separate layers. This lesson preserves parent/child evidence and diagnoses injection risk, wrong configuration identity, misleading green parents, cross-boundary artifact assumptions, and fan-out explosions.
Learning objectives
- Diagnose generated-YAML injection and malformed configuration without hiding the original generated artifact or validation error.
- Prove whether parent and child used the intended source SHA and intended child configuration.
- Explain why a green trigger job may only prove child creation unless status mirroring is configured.
- Separate cross-pipeline artifact transfer from report visibility and distinguish Free core orchestration from paid artifact-transfer features.
- Detect excessive fan-out, empty child pipelines, wrong rules, unsafe variable forwarding, and hierarchy-limit failures from evidence.
1. Evidence-first diagnostic sequence across a hierarchy
- Preserve parent pipeline ID, source/ref/SHA, expanded configuration, trigger job, and first failure.
- Preserve generator job ID/logs/artifact/digest when configuration is generated.
- Confirm trigger rules, include source, strategy, inputs, and variable-forwarding policy.
- Confirm whether GitLab created a child; record its ID, source, ref, SHA, and compiled configuration.
- Inspect child job graph/queue and then runner/executor/image/toolchain.
- Inspect failing scripts/network, then child reports/artifacts/caches.
- Inspect any external target only if the child actually performed external side effects.
- Apply the smallest causal repair and rerun only the generator, trigger, child, or parent scope required.
2. Failure: generated YAML injection
Broken pattern: a generator pastes an unvalidated
TARGET variable into job names and scripts. A malicious
or malformed value can change YAML structure or shell behavior even
though the final YAML still parses.
# BAD: do not do this.
target = os.environ["TARGET"]
print(f"""{target}-job:
script:
- ./test.sh {target}
""")
Repair: map external metadata to internal identifiers through an allowlist, emit from fixed templates, reject unexpected keys, cap output cardinality, validate the result, and retain the generated artifact. Do not “sanitize” arbitrary YAML text with ad-hoc replacements.
3. Failure: child appears to use the wrong code/configuration
For a true parent-child pipeline, child project/ref/SHA should match the parent. If the observed behavior looks stale, first compare those immutable identities. Then inspect the child configuration source: local child file at that SHA, external project include/ref, or generated artifact from a specific generator job.
Do not fix this by retrying repeatedly. A retry can recreate the child and obscure whether the original generated artifact or include resolution was wrong.
4. Failure: parent is green while child later fails
This is often expected default trigger behavior, not a GitLab bug.
The trigger job succeeded because GitLab created the child. If
organizational policy requires the parent gate to reflect child
completion, use strategy: mirror and preserve
before/after evidence.
If an older instance does not support mirror, record
the exact version and compatibility design rather than pretending
default trigger success means completion.
5. Failure: downstream pipeline cannot be created because it would be empty
A child can compile to zero runnable jobs when its
rules incorrectly expect
merge_request_event or when all conditions reject
parent_pipeline. GitLab reports that the resulting
downstream pipeline would be empty.
# BROKEN in a child:
child-check:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
script: echo "never included in a child"
# FIX:
child-check:
rules:
- if: '$CI_PIPELINE_SOURCE == "parent_pipeline"'
script: echo "child job"
6. Failure: artifact cannot be fetched across the boundary
First ask whether cross-boundary transfer is actually required. A child can often rebuild lightweight test fixtures from the same source SHA or produce its own outputs. If it must consume a parent artifact, define the supported transfer mechanism explicitly.
needs:pipeline:job currently requires Premium/Ultimate
and a successful producer job in the same hierarchy. It cannot be
used in the trigger job itself. Do not replace a failed authorized
transfer with a broad PAT, public artifact URL, or “rebuild whatever
was there” shortcut.
7. Failure: fan-out explosion
Symptoms include many downstream cards, high pipeline-creation rates, large runner queues, delayed feedback, artifact/log growth, or the hierarchy-size error. Preserve the parent SHA and generator artifact before changing anything.
Count intended subtrees versus emitted children. Then reduce granularity, cap generated targets, deduplicate metadata, and prefer jobs/DAG edges inside a child where another pipeline boundary adds no ownership value. Raising the hierarchy limit is an instance-admin operation and is not the first fix.
8. Failure: hidden variable forwarding changes child behavior
Forwarded variables become trigger variables with high precedence. A
child can therefore ignore its own default because an upstream
variable with the same name wins. Inspect non-secret effective
values and the trigger’s inherit:variables/forward
policy.
Repair by narrowing the forwarded contract, renaming contextual variables, or converting stable configuration to typed inputs. Never dump the entire environment to debug precedence.
9. Failure: report visibility is mistaken for artifact availability
A report appearing in a merge-request widget is GitLab report ingestion/UI state. It does not mean a parent job can open the child’s raw report artifact from its workspace. Conversely, a successfully transferred raw artifact does not guarantee GitLab ingested it as a typed report.
Preserve report metadata, artifact metadata, producer job ID, and parent/child IDs separately.
10. Security-sensitive actions and prohibited shortcuts
- Do not forward real secrets broadly to children.
- Do not generate jobs that request privileged runners from untrusted branch/MR metadata.
- Do not interpolate untrusted strings into generated scripts, images, includes, tags, environment names, or external targets.
- Do not use broad PATs to work around child artifact/permission issues.
- Do not disable TLS or project protections to make external includes work.
- Do not delete/recreate pipelines or artifacts before preserving first-failure IDs and generated configuration.
- Do not raise hierarchy/rate limits as a substitute for fixing runaway generation.
11. Intentionally broken example: one line, two causal layers
Suppose the generator emits valid YAML with an unsupported child rule and the trigger uses default strategy. The generator job is green, the trigger can fail because the child would be empty, and no child runner ever starts. That is a configuration/rule failure.
Now fix the child rule but leave default trigger strategy. The trigger becomes green as soon as the child is created; if a child job later fails, the parent trigger remains green. That second symptom is status-contract behavior. The two failures look similar in the parent graph but require different repairs.
12. Performance: measure hierarchy overhead before optimizing
Measure parent creation time, child creation time, runner queue time, child critical path, number of children/jobs, and artifact/report I/O. Child pipelines can reduce irrelevant jobs through sparse routing, but they do not make runner capacity infinite.
A faster-looking parent graph can hide longer end-to-end completion if children wait in queues or the parent does not mirror their status. Optimize user-visible feedback and correctness evidence, not just the parent’s duration.
Knowledge check
What should you preserve before fixing a malformed generated child pipeline?
Parent ID/SHA, generator job/logs, the generated YAML artifact and ideally its digest, trigger job configuration/status, and the exact child-creation validation error.
Why can a child pipeline be empty when the parent was an MR pipeline?
Child CI_PIPELINE_SOURCE is parent_pipeline, not merge_request_event. Rules written only for merge_request_event can omit every child job.
What does strategy: mirror repair?
The status contract between trigger job and downstream pipeline. It does not repair child YAML, runner failures, or artifacts.
Why is increasing pipeline_hierarchy_size a poor first response to fan-out?
It hides a decomposition/generation problem and increases instance resource risk. First bound and deduplicate child creation.
Can a report visible in the parent/MR UI be assumed to exist in a parent job workspace?
No. Report ingestion/UI and cross-pipeline artifact transfer are separate states.
Version and compatibility note
GitLab and GitLab Runner evolve continuously. Treat version-sensitive YAML, runner/executor behavior, APIs, security features, policy controls, tiers, and deprecations as assumptions to verify against the current official GitLab documentation before production use. Preserve the exact project/ref/SHA, compiled configuration, pipeline/job IDs, runner/tool versions, and external target evidence used for any reproducible lab or incident record.
Official references and version notes
Documentation verification date: 2026-09-11.
Parent/child pipeline syntax, downstream status behavior, report
visibility, hierarchy limits, inputs, and variable forwarding are
version-sensitive. Re-check the deployed GitLab version before
relying on newer syntax such as strategy: mirror.
- Downstream pipelines — parent-child semantics, dynamic child pipelines, nesting, pipeline source, reports, variables, inputs, and status behavior.
-
CI/CD YAML syntax reference
—
trigger,trigger:include,trigger:strategy,trigger:forward, andneeds:pipeline:job. - Use CI/CD configuration from other files — include rules and dynamic-child configuration limitations.
- Troubleshooting downstream pipelines — empty child pipelines, permission/configuration failures, and variable forwarding issues.
- CI/CD limits — downstream hierarchy sizing and resource-protection rationale.
-
Pipelines API
— listing pipelines and querying child pipelines with
source=parent_pipeline.
CI_PIPELINE_SOURCE=parent_pipeline. Nested child
pipelines are limited to two child levels; the default pipeline
hierarchy limit is 1000 downstream pipelines. One child trigger can
combine up to three child configuration files. Dynamic child
configuration can come from a generated artifact; the artifact path
is interpreted by the GitLab server, and CI/CD variables cannot be
used in an include section inside the dynamic child
configuration. strategy: mirror was introduced in
GitLab 18.2 and is the recommended status-coupling strategy; without
a strategy the trigger job succeeds once the child is created.
YAML-defined trigger variables forward by default, while pipeline
variables do not unless
trigger:forward:pipeline_variables: true is set.
needs:pipeline:job for cross-parent/child artifact
download is currently Premium/Ultimate, so mandatory labs do not
depend on it.
Keep the academy open
Support free, practical DevOps education.
Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0
Send only Ethereum/ERC-20 compatible assets to this
address.