Chapter 14Lesson 04~175 minutes

Parent-Child Pipelines, Dynamic Child Pipelines, Generated Configuration, and Monorepo Decomposition: Diagnostics, Failure Modes, Security, and Performance

Failures across a pipeline hierarchy are easy to misdiagnose because creation, status propagation, generated YAML, variable forwarding, artifacts, runners, and reports are separate layers. This lesson preserves parent/child evidence and diagnoses injection risk, wrong configuration identity, misleading green parents, cross-boundary artifact assumptions, and fan-out explosions.

Parent-child pipelinesDynamic configurationMonoreposGenerated YAMLPipeline orchestration

Learning objectives

  • Diagnose generated-YAML injection and malformed configuration without hiding the original generated artifact or validation error.
  • Prove whether parent and child used the intended source SHA and intended child configuration.
  • Explain why a green trigger job may only prove child creation unless status mirroring is configured.
  • Separate cross-pipeline artifact transfer from report visibility and distinguish Free core orchestration from paid artifact-transfer features.
  • Detect excessive fan-out, empty child pipelines, wrong rules, unsafe variable forwarding, and hierarchy-limit failures from evidence.

1. Evidence-first diagnostic sequence across a hierarchy

  1. Preserve parent pipeline ID, source/ref/SHA, expanded configuration, trigger job, and first failure.
  2. Preserve generator job ID/logs/artifact/digest when configuration is generated.
  3. Confirm trigger rules, include source, strategy, inputs, and variable-forwarding policy.
  4. Confirm whether GitLab created a child; record its ID, source, ref, SHA, and compiled configuration.
  5. Inspect child job graph/queue and then runner/executor/image/toolchain.
  6. Inspect failing scripts/network, then child reports/artifacts/caches.
  7. Inspect any external target only if the child actually performed external side effects.
  8. Apply the smallest causal repair and rerun only the generator, trigger, child, or parent scope required.

2. Failure: generated YAML injection

Broken pattern: a generator pastes an unvalidated TARGET variable into job names and scripts. A malicious or malformed value can change YAML structure or shell behavior even though the final YAML still parses.

# BAD: do not do this.
target = os.environ["TARGET"]
print(f"""{target}-job:
  script:
    - ./test.sh {target}
""")

Repair: map external metadata to internal identifiers through an allowlist, emit from fixed templates, reject unexpected keys, cap output cardinality, validate the result, and retain the generated artifact. Do not “sanitize” arbitrary YAML text with ad-hoc replacements.

3. Failure: child appears to use the wrong code/configuration

For a true parent-child pipeline, child project/ref/SHA should match the parent. If the observed behavior looks stale, first compare those immutable identities. Then inspect the child configuration source: local child file at that SHA, external project include/ref, or generated artifact from a specific generator job.

Do not fix this by retrying repeatedly. A retry can recreate the child and obscure whether the original generated artifact or include resolution was wrong.

4. Failure: parent is green while child later fails

This is often expected default trigger behavior, not a GitLab bug. The trigger job succeeded because GitLab created the child. If organizational policy requires the parent gate to reflect child completion, use strategy: mirror and preserve before/after evidence.

If an older instance does not support mirror, record the exact version and compatibility design rather than pretending default trigger success means completion.

5. Failure: downstream pipeline cannot be created because it would be empty

A child can compile to zero runnable jobs when its rules incorrectly expect merge_request_event or when all conditions reject parent_pipeline. GitLab reports that the resulting downstream pipeline would be empty.

# BROKEN in a child:
child-check:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
  script: echo "never included in a child"

# FIX:
child-check:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "parent_pipeline"'
  script: echo "child job"

6. Failure: artifact cannot be fetched across the boundary

First ask whether cross-boundary transfer is actually required. A child can often rebuild lightweight test fixtures from the same source SHA or produce its own outputs. If it must consume a parent artifact, define the supported transfer mechanism explicitly.

needs:pipeline:job currently requires Premium/Ultimate and a successful producer job in the same hierarchy. It cannot be used in the trigger job itself. Do not replace a failed authorized transfer with a broad PAT, public artifact URL, or “rebuild whatever was there” shortcut.

7. Failure: fan-out explosion

Symptoms include many downstream cards, high pipeline-creation rates, large runner queues, delayed feedback, artifact/log growth, or the hierarchy-size error. Preserve the parent SHA and generator artifact before changing anything.

Count intended subtrees versus emitted children. Then reduce granularity, cap generated targets, deduplicate metadata, and prefer jobs/DAG edges inside a child where another pipeline boundary adds no ownership value. Raising the hierarchy limit is an instance-admin operation and is not the first fix.

8. Failure: hidden variable forwarding changes child behavior

Forwarded variables become trigger variables with high precedence. A child can therefore ignore its own default because an upstream variable with the same name wins. Inspect non-secret effective values and the trigger’s inherit:variables/forward policy.

Repair by narrowing the forwarded contract, renaming contextual variables, or converting stable configuration to typed inputs. Never dump the entire environment to debug precedence.

9. Failure: report visibility is mistaken for artifact availability

A report appearing in a merge-request widget is GitLab report ingestion/UI state. It does not mean a parent job can open the child’s raw report artifact from its workspace. Conversely, a successfully transferred raw artifact does not guarantee GitLab ingested it as a typed report.

Preserve report metadata, artifact metadata, producer job ID, and parent/child IDs separately.

10. Security-sensitive actions and prohibited shortcuts

  • Do not forward real secrets broadly to children.
  • Do not generate jobs that request privileged runners from untrusted branch/MR metadata.
  • Do not interpolate untrusted strings into generated scripts, images, includes, tags, environment names, or external targets.
  • Do not use broad PATs to work around child artifact/permission issues.
  • Do not disable TLS or project protections to make external includes work.
  • Do not delete/recreate pipelines or artifacts before preserving first-failure IDs and generated configuration.
  • Do not raise hierarchy/rate limits as a substitute for fixing runaway generation.

11. Intentionally broken example: one line, two causal layers

Suppose the generator emits valid YAML with an unsupported child rule and the trigger uses default strategy. The generator job is green, the trigger can fail because the child would be empty, and no child runner ever starts. That is a configuration/rule failure.

Now fix the child rule but leave default trigger strategy. The trigger becomes green as soon as the child is created; if a child job later fails, the parent trigger remains green. That second symptom is status-contract behavior. The two failures look similar in the parent graph but require different repairs.

12. Performance: measure hierarchy overhead before optimizing

Measure parent creation time, child creation time, runner queue time, child critical path, number of children/jobs, and artifact/report I/O. Child pipelines can reduce irrelevant jobs through sparse routing, but they do not make runner capacity infinite.

A faster-looking parent graph can hide longer end-to-end completion if children wait in queues or the parent does not mirror their status. Optimize user-visible feedback and correctness evidence, not just the parent’s duration.

Knowledge check

What should you preserve before fixing a malformed generated child pipeline?

Why can a child pipeline be empty when the parent was an MR pipeline?

What does strategy: mirror repair?

Why is increasing pipeline_hierarchy_size a poor first response to fan-out?

Can a report visible in the parent/MR UI be assumed to exist in a parent job workspace?

Next lesson

Checkpoint lab

Generate one bounded child from validated metadata and prove source/status traceability in a toy monorepo.

Version and compatibility note

GitLab and GitLab Runner evolve continuously. Treat version-sensitive YAML, runner/executor behavior, APIs, security features, policy controls, tiers, and deprecations as assumptions to verify against the current official GitLab documentation before production use. Preserve the exact project/ref/SHA, compiled configuration, pipeline/job IDs, runner/tool versions, and external target evidence used for any reproducible lab or incident record.

Official references and version notes

Documentation verification date: 2026-09-11. Parent/child pipeline syntax, downstream status behavior, report visibility, hierarchy limits, inputs, and variable forwarding are version-sensitive. Re-check the deployed GitLab version before relying on newer syntax such as strategy: mirror.

Current behavior used by this chapter: parent-child pipelines are available on Free/Premium/Ultimate and GitLab.com/Self-Managed/Dedicated. Child pipelines run in the same project, ref, and commit SHA as the parent and report CI_PIPELINE_SOURCE=parent_pipeline. Nested child pipelines are limited to two child levels; the default pipeline hierarchy limit is 1000 downstream pipelines. One child trigger can combine up to three child configuration files. Dynamic child configuration can come from a generated artifact; the artifact path is interpreted by the GitLab server, and CI/CD variables cannot be used in an include section inside the dynamic child configuration. strategy: mirror was introduced in GitLab 18.2 and is the recommended status-coupling strategy; without a strategy the trigger job succeeds once the child is created. YAML-defined trigger variables forward by default, while pipeline variables do not unless trigger:forward:pipeline_variables: true is set. needs:pipeline:job for cross-parent/child artifact download is currently Premium/Ultimate, so mandatory labs do not depend on it.

Keep the academy open

Support free, practical DevOps education.

Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.

Ethereum / ERC-20
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0 Send only Ethereum/ERC-20 compatible assets to this address.