GitLab CI/CD Foundations: .gitlab-ci.yml, Pipelines, Jobs, Stages, and Execution Model: Diagnostics, Failure Modes, Security, and Performance

1. Diagnostic sequence: identify the failed phase first

Use the same disciplined sequence from earlier chapters, now expanded for CI/CD:

1. Preserve evidence: pipeline URL/ID, job ID, source, ref, SHA, status, log excerpt, runner ID/tags, and configuration commit.
2. Identify scope: GitLab offering/instance → namespace/project → ref/MR → pipeline source/ID → job/stage → runner/executor → artifact/token/resource.
3. Inspect the owning layer: CI Lint/config, pipeline graph, runner list/tags/protection, job log, API status/body, or job-token policy.
4. Choose the least destructive correction: repair one rule/tag/command/permission rather than disabling governance or adding broad credentials.
5. Verify: prove the new pipeline/job corresponds to the intended SHA and that the original cause no longer exists.

2. Symptom-to-layer map

3. Broken example A: YAML parses, GitLab rejects the CI model

This file is syntactically valid YAML, but the job names a stage that is not declared. Generic YAML validation can pass while GitLab CI Lint rejects it.

stages:
  - test

compile:
  stage: build
  script:
    - echo "compile"

stages:
  - build
  - test

compile:
  stage: build
  script:
    - echo "compile"

4. Broken example B: valid configuration deliberately creates no pipeline

A configuration can be valid and still suppress pipeline creation. This is a creation-time result, not a failed pipeline.

workflow:
  rules:
    - when: never

never_created:
  script:
    - echo "A job definition exists, but no pipeline is created."

5. Broken example C: no runner satisfies the job tags

The following job can create a pipeline successfully yet remain pending if no available runner carries every declared tag.

needs_impossible_runner:
  tags:
    - ch10-lab-no-such-runner
  script:
    - echo "This should not execute in the lab."

6. Broken assumption D: “green” for the wrong commit

Imagine pipeline P1 passed for SHA a1a1…. A new commit moves the branch to b2b2…. P1 remains a valid historical result for a1a1…, but it is not evidence for b2b2…. The repair is not “rerun P1 until green”; create/identify the pipeline whose recorded SHA equals the candidate commit.

BRANCH="feature/example"
git fetch origin "$BRANCH"
REMOTE_SHA="$(git rev-parse FETCH_HEAD)"

glab ci list --ref "$BRANCH" --output json --per-page 10   --jq '.[] | {id,status,source,ref,sha}'

printf 'remote_tip=%s\n' "$REMOTE_SHA"

7. Broken assumption E: CI_JOB_TOKEN should access everything the user can

GitLab documents a limited set of resources/endpoints for job-token authentication. The token also expires when the job ends. Cross-project access normally requires the target’s job-token allowlist plus the triggering user’s underlying permission. A syntactically correct JOB-TOKEN header can therefore receive 401 or 403 for a legitimate policy reason.

8. Runner security is causally relevant to token safety

GitLab explicitly warns that insecure runner configuration can let jobs steal tokens from other jobs. Reusing a shell executor host for mutually untrusted projects, or using privileged containers on a reused machine without a deliberate isolation design, expands the blast radius. This is why Chapter 14 treats runner executors/isolation as security architecture.

At this chapter’s level, the rule is simple: do not send untrusted CI configuration to a runner merely because the runner is powerful and available. Match the runner trust zone to the source/configuration trust level.

9. Performance and cost only after correctness

A pending job caused by no matching runner is not solved by buying faster compute. A pipeline never created does not consume job runtime. A slow job that is actually running may justify profiling command duration, image/setup time, network, or runner capacity. Keep these causal distinctions so cost optimization does not mask correctness failures.

Hosted-runner compute limits and billing are volatile. Record the current account/project policy when it materially affects a lab, but do not encode current minute/credit numbers into operational logic.

10. Minimal evidence pack for a CI incident

# Example read-only evidence collection; replace project/ref values.
REPO="GROUP/PROJECT"
REF="feature/example"
mkdir -p ci-evidence

git fetch origin "$REF"
git rev-parse FETCH_HEAD > ci-evidence/remote-sha.txt

glab ci list -R "$REPO" --ref "$REF" --output json --per-page 20   > ci-evidence/pipelines.json

glab runner list -R "$REPO" --output json   > ci-evidence/runners.json

# Copy only non-secret log excerpts/IDs needed for diagnosis.
# Never archive CI_JOB_TOKEN or runner authentication tokens.

Knowledge check

Summary

CI/CD diagnosis becomes tractable when you identify the phase: parse/validate → create pipeline → materialize job → schedule runner → execute checkout/script → authorize resource → upload evidence. Preserve IDs and SHAs, repair the owning layer, and resist broad-credential or governance-bypass shortcuts.

Official references

• GitLab Docs — CI/CD
• GitLab Docs — Get started with GitLab CI/CD
• GitLab Docs — CI/CD YAML syntax reference
• GitLab Docs — Validate CI/CD configuration
• GitLab Docs — Pipeline editor
• GitLab Docs — Pipelines
• GitLab Docs — CI/CD jobs
• GitLab Docs — Job execution flow
• GitLab Docs — Predefined CI/CD variables
• GitLab Docs — CI/CD variables
• GitLab Docs — CI/CD job token
• GitLab Docs — Runners
• GitLab Docs — Configure runners
• GitLab Docs — GitLab-hosted runners
• GitLab Docs — Pipelines API
• GitLab Docs — Jobs API
• GitLab Docs — CI Lint API
• GitLab Docs — Job rules and CI_PIPELINE_SOURCE
• GitLab CLI — ci
• GitLab CLI — ci list
• GitLab CLI — ci get
• GitLab CLI — ci status
• GitLab CLI — runner list