Checkpoint Lab — Test Reports, Coverage, JUnit, Code Quality, Browser Performance, Accessibility, and Pipeline Feedback

1. Checkpoint scenario

Use the disposable glci-ch24-feedback-lab project. The pipeline runs a deterministic standard-library test harness, emits JUnit XML and Cobertura XML, extracts a coverage percentage, and retains raw evidence. You then create one deliberate failure: either malformed JUnit XML or a misleading coverage regex. Diagnose it without hiding the original job/report.

2. Assumptions and preflight

set -eu
git status --short
git rev-parse HEAD
printf 'source=%s ref=%s sha=%s pipeline=%s\n' \
  "$CI_PIPELINE_SOURCE" "$CI_COMMIT_REF_NAME" "$CI_COMMIT_SHA" "$CI_PIPELINE_ID"

3. Predict before execution

4. Exact checkpoint pipeline

stages: [test, verify]

test_and_report:
  stage: test
  image: python:3.12-alpine
  script:
    - python tools/run_checks.py | tee reports/tool-output.txt
    - sha256sum reports/junit.xml reports/cobertura.xml reports/tool-output.txt | tee reports/SHA256SUMS
    - printf 'source=%s\nref=%s\nsha=%s\npipeline=%s\njob=%s\n' "$CI_PIPELINE_SOURCE" "$CI_COMMIT_REF_NAME" "$CI_COMMIT_SHA" "$CI_PIPELINE_ID" "$CI_JOB_ID" > reports/source.env
  coverage: '/TOTAL .* ([0-9]+(?:\.[0-9]+)?)%$/'
  artifacts:
    when: always
    expire_in: 7 days
    paths:
      - reports/
    reports:
      junit: reports/junit.xml
      coverage_report:
        coverage_format: cobertura
        path: reports/cobertura.xml

verify_raw_reports:
  stage: verify
  image: python:3.12-alpine
  needs:
    - job: test_and_report
      artifacts: true
  script:
    - sha256sum -c reports/SHA256SUMS
    - python -c 'import xml.etree.ElementTree as ET; ET.parse("reports/junit.xml"); ET.parse("reports/cobertura.xml")'
    - grep -F "sha=${CI_COMMIT_SHA}" reports/source.env
    - grep -F "pipeline=${CI_PIPELINE_ID}" reports/source.env

The verify_raw_reports job proves raw-file integrity independently of GitLab's report widgets. A production pipeline could use a stronger provenance/evidence manifest; the key concept is that parser presentation and raw verification are distinct checks.

5. Expected observations

6. Inject one controlled failure

Option A — malformed JUnit

On a disposable branch, after report generation but before artifact upload, append a single invalid fragment such as <broken> without a closing tag to reports/junit.xml. Record the new checksum. Keep the test tool exit result unchanged. Expect local XML validation to fail and GitLab's JUnit feedback to be missing/invalid. Preserve the original pipeline/job/report before correcting the generator step.

Option B — misleading coverage regex

Add an earlier synthetic line TOTAL warmup 1 covered 10.0% and intentionally loosen the regex so it matches the wrong percentage. Preserve the job output and extracted value. Repair the regex to match the intended final line, not the report contents.

Choose only one option. The purpose is to locate the failure layer, not to create a collection of broken states.

7. Required diagnostic sequence

1. Record pipeline/job IDs and first-failure screenshots/text evidence.
2. Confirm CI_PIPELINE_SOURCE, ref, CI_COMMIT_SHA, and compiled report paths.
3. Confirm tool exit code and exact script line that changed the report/output.
4. Checksum and locally validate the raw report.
5. Inspect GitLab Tests/coverage feedback without assuming it is the source of truth.
6. Apply only the causal correction.
7. Rerun the smallest safe scope and compare new evidence with the preserved first failure.

8. Required evidence packet

9. Verification checklist

• Report files correspond to the recorded source SHA and producer job.
• JUnit and Cobertura parse locally.
• Checksums verify in the downstream verification job.
• JUnit result and job status are interpreted separately.
• Coverage percentage and coverage visualization are interpreted separately.
• No report contains credentials or production data.
• The intentional failure has preserved first-failure evidence.
• The correction changes only the causal layer.
• Artifact retention/access and tier assumptions are documented.

10. Cleanup / rollback

Delete the disposable branch/project when the exercise is complete, or keep it until the seven-day artifact expiry for review. No external resources exist. If you change project artifact-access or retention settings for experimentation, restore the original values and record that rollback.

Knowledge check

11. What Chapter 24 adds to the production operating model

You can now treat quality feedback as a traceable evidence pipeline: exact source and configuration → tool/version and exit semantics → structured report → GitLab parser/UI → explicit gate/advisory policy → retained raw evidence. This prevents green jobs, dashboards, or percentages from becoming unexamined proxies for correctness.

Chapter 25 extends the same model to security scanners. Scanner findings introduce additional questions about scanner scope, template/version, false positives, policy gates, exceptions, and remediation evidence.

Official references and version notes

Documentation verification date: 2026-09-12. JUnit/unit-test reports, coverage percentage extraction, Cobertura/JaCoCo coverage visualization, Code Quality report import, and Accessibility reports are available on Free/Premium/Ultimate. Browser Performance reports are Premium/Ultimate. Code Quality's built-in CodeClimate-based template was deprecated in GitLab 17.3 and is planned for removal in GitLab 19.0; current guidance is to integrate a supported tool's report directly. The checkpoint remains fully Free-compatible. Browser Performance is taught as an optional Premium/Ultimate extension, while accessibility/report-import concepts remain available without requiring paid infrastructure.

• Unit test reports — official reference.
• Unit test report examples — official reference.
• Code coverage — official reference.
• Coverage reporting — official reference.
• Coverage visualization — official reference.
• Cobertura coverage visualization — official reference.
• Code Quality — official reference.
• Accessibility testing — official reference.
• Browser performance testing — official reference.
• Artifacts reports types — official reference.
• Job artifacts — official reference.
• CI/CD YAML syntax reference — official reference.

Current assumptions used in this chapter: Version-sensitive YAML, Runner/executor behavior, APIs, security features, policy controls, tiers, and deprecations must be verified against the exact GitLab, GitLab Runner, tool, and external-system versions used in production. Preserve the exact project/ref/SHA, compiled configuration, pipeline/job IDs, runner/tool versions, and external target evidence used for reproducible work.