SBOM Generation, Dependency Lists, Vulnerability Reports, Policy Evaluation, and Supply-Chain Evidence: Diagnostics, Failure Modes, Security, and Performance

1. Evidence-first diagnostic sequence

1. Preserve pipeline/job IDs, CI_PIPELINE_SOURCE, CI_COMMIT_SHA, first failing logs, and existing artifacts.
2. Confirm compiled configuration and rule decisions: did the intended build/SBOM/policy jobs exist?
3. Confirm runner/executor/image/tool versions; do not assume the scanner/generator from the YAML comment is what ran.
4. Verify the build artifact digest before opening the SBOM.
5. Verify SBOM format/spec/generator/input scope and its own digest.
6. Verify vulnerability-data source/snapshot/time and component identity mapping.
7. Verify policy version, exception data, and decision.
8. Inspect GitLab ingestion/UI state only after raw evidence is understood.
9. Apply the smallest safe correction and rerun only the necessary scope.

2. Failure: SBOM from the wrong artifact

Suppose pipeline 410 builds artifact digest aaa…, but the SBOM metadata/evidence manifest references bbb… from an earlier run. The SBOM might be perfectly valid CycloneDX; the failure is lineage.

pipeline_id=410
job_id=9123
source_sha=8f2c...lab
artifact_sha256=aaa111...
sbom_sha256=4bd7...
evidence_manifest_artifact_sha256=bbb222...   <-- mismatch

Repair by regenerating only the SBOM/evidence for the already-built artifact if that operation is deterministic and authorized, or rebuild deliberately as a new lineage. Never silently edit the old manifest to make hashes agree.

3. Failure: policy uses stale severity data

Severity is not a timeless field. Advisory providers can revise severity, affected ranges, fixed versions, and identifiers. If policy stores only “HIGH → block,” you cannot later explain why a release was blocked or allowed.

Record the vulnerability database/source identifier, retrieval/snapshot time, scanner/correlator version, and the raw finding. If live re-correlation changes the result tomorrow, that is a new evaluation—not proof that yesterday's record was wrong.

4. Failure: report omitted because the job failed

A generator can fail after partially writing output. If artifacts upload only on success, the most useful forensic evidence disappears. For evidence-producing jobs, use bounded artifacts:when: always for non-secret diagnostic files and validate completeness explicitly.

sbom_evidence:
  stage: evidence
  script:
    - ./generate-sbom.sh
    - python tools/validate_evidence.py
  artifacts:
    when: always
    expire_in: 7 days
    paths:
      - evidence/

when: always is not permission to archive secrets, full environment dumps, private keys, or tokens. Evidence design still needs data minimization.

5. Failure: generated evidence is not tied to a digest

Filenames such as app-latest.tar.gz or sbom.json are mutable labels. An evidence bundle should contain a manifest that binds immutable identifiers.

{
  "pipeline_id": 410,
  "job_id": 9123,
  "pipeline_source": "merge_request_event",
  "source_sha": "8f2c000000000000000000000000000000000000",
  "artifact": {
    "path": "dist/app.txt",
    "sha256": "aaa1110000000000000000000000000000000000000000000000000000000000"
  },
  "sbom": {
    "path": "evidence/sbom.cdx.json",
    "sha256": "4bd7000000000000000000000000000000000000000000000000000000000000",
    "spec": "CycloneDX 1.6"
  },
  "policy": {
    "name": "chapter26-no-unexcepted-high-v1",
    "decision": "block"
  }
}

6. Failure: Dependency List interpreted as vulnerability proof

The Dependency List is an inventory-oriented view. A component appearing there is not proof it is vulnerable; a component lacking a visible vulnerability is not proof it is safe. Current GitLab docs can enrich dependency data with known vulnerabilities, but the evidence must still be traced back to the SBOM/scanner and advisory data.

7. Intentionally broken example: policy reads a mutable “latest” report

policy_gate_broken:
  stage: policy
  script:
    # WRONG: downloads mutable external report unrelated to this pipeline's source/artifact.
    - curl --fail --silent --show-error -o report.json https://example.invalid/reports/latest.json
    - python policy/evaluate.py report.json

The compiled YAML is syntactically valid, the runner can execute it, and the policy might produce a deterministic decision over the downloaded file. The causal failure is that the evidence input is not bound to this pipeline or artifact.

Repair: consume the report artifact produced by an explicit upstream job in the same evidence chain, verify its checksum, and store source/artifact identity inside the report/manifest.

policy_gate_fixed:
  stage: policy
  needs:
    - job: sbom_and_correlation
      artifacts: true
  script:
    - sha256sum --check evidence/evidence-SHA256SUMS
    - python policy/evaluate.py

8. Failure: “GitLab policy approved it, so scanner evidence must be authentic”

That conclusion is unsupported. GitLab's merge request approval policy documentation explicitly notes that policy evaluation does not check integrity/authenticity of scan results. Protect the configuration path: review/pin scanner components/images, restrict who can alter policy/scanner jobs, preserve raw reports, and consider stronger build/attestation controls where risk warrants them.

9. Performance and cost

• Generate the minimum inventories needed for each boundary; avoid scanning identical immutable bytes repeatedly.
• Cache tool downloads only when the cache cannot silently substitute unreviewed binaries.
• Prefer deterministic dependency resolution and reuse of approved artifacts.
• Split source and image SBOMs when that improves parallelism and evidence clarity.
• Retain raw evidence long enough for audit/reproduction, but do not keep unlimited giant artifacts by default.

10. Failure-layer matrix

Knowledge check

Official references and version notes

Documentation verification date: 2026-09-12. GitLab currently documents the Dependency List, artifacts:reports:cyclonedx, Vulnerability Report, and organization-wide security policies as Ultimate features. The mandatory course path therefore keeps the SBOM, vulnerability correlation, policy decision, hashes, and evidence bundle as ordinary artifacts and local scripts so it remains Free/disposable-compatible. GitLab's Dependency List can ingest CycloneDX 1.4, 1.5, or 1.6 documents from the latest default-branch pipeline. Security-policy evaluation trusts scanner artifact reports; policy evaluation does not itself prove the authenticity or integrity of the scanner that produced them. The lab pins cyclonedx-bom==7.3.1, released 2026-07-23, and requests CycloneDX 1.6 explicitly. Evidence retention should be treated as its own design decision. GitLab security findings, vulnerability records, job artifacts, and external audit storage have different lifecycles and permissions.

• Dependency list — official reference.
• Dependency scanning by using SBOM — official reference.
• Continuous dependency scanning — official reference.
• CI/CD artifacts report types — official reference.
• CI/CD YAML syntax — official reference.
• Security scanning results — official reference.
• Vulnerability report — official reference.
• Security policies — official reference.
• Merge request approval policies — official reference.
• Scan execution policies — official reference.
• CycloneDX Python SBOM generator — official reference.
• CycloneDX specification 1.6 — official reference.
• Anchore Syft — official reference.

Current assumptions used in this chapter: Version-sensitive YAML, Runner/executor behavior, APIs, security features, policy controls, tiers, and deprecations must be verified against the exact GitLab, GitLab Runner, tool, and external-system versions used in production. Preserve the exact project/ref/SHA, compiled configuration, pipeline/job IDs, runner/tool versions, and external target evidence used for reproducible work.