Test Reports, Coverage, JUnit, Code Quality, Browser Performance, Accessibility, and Pipeline Feedback: Guided Hands-On Workflow and Core Operations

1. Disposable scenario and safety boundary

Create a throwaway project named glci-ch24-feedback-lab. The lab contains a two-function Python module and a deterministic report generator. It writes no secrets, makes no network requests, and changes no external environment. The required path is fully Free-compatible: JUnit, coverage percentage, Cobertura visualization, and Code Quality report import.

2. Preflight: prove source and runner context

git status --short
git rev-parse HEAD
printf 'source=%s ref=%s sha=%s pipeline=%s job=%s\n' \
  "$CI_PIPELINE_SOURCE" "$CI_COMMIT_REF_NAME" "$CI_COMMIT_SHA" \
  "$CI_PIPELINE_ID" "$CI_JOB_ID"
python --version

Record the GitLab version/offering, runner version/executor, and execution-image identity visible in the job page. The examples use python:3.12-alpine for readability; production pipelines should pin an organization-approved image digest where practical.

3. Create deterministic source and report generator

# app.py
def add(a, b):
    return a + b

def classify(n):
    return "positive" if n > 0 else "non-positive"

The test generator below performs three assertions, emits JUnit XML, emits minimal Cobertura XML, and prints one stable coverage line that GitLab can parse with a regex.

# tools/run_checks.py
from pathlib import Path
import hashlib, sys, xml.etree.ElementTree as ET
from app import add, classify

Path("reports").mkdir(exist_ok=True)
tests = [
    ("test_add", lambda: add(2, 3) == 5),
    ("test_positive", lambda: classify(4) == "positive"),
    ("test_zero", lambda: classify(0) == "non-positive"),
]
failures = []
for name, fn in tests:
    try:
        ok = fn()
    except Exception as exc:
        ok = False
        failures.append((name, repr(exc)))
    if not ok and all(name != x[0] for x in failures):
        failures.append((name, "assertion returned false"))

suite = ET.Element("testsuite", name="chapter24", tests=str(len(tests)), failures=str(len(failures)))
for name, _ in tests:
    case = ET.SubElement(suite, "testcase", classname="chapter24", name=name, time="0.001")
    for failed, message in failures:
        if failed == name:
            ET.SubElement(case, "failure", message=message).text = message
ET.ElementTree(suite).write("reports/junit.xml", encoding="utf-8", xml_declaration=True)

coverage = ET.Element("coverage", {"line-rate":"1.0", "branch-rate":"0", "version":"chapter24"})
sources = ET.SubElement(coverage, "sources")
ET.SubElement(sources, "source").text = "."
packages = ET.SubElement(coverage, "packages")
package = ET.SubElement(packages, "package", {"name":"chapter24", "line-rate":"1.0", "branch-rate":"0"})
classes = ET.SubElement(package, "classes")
klass = ET.SubElement(classes, "class", {"name":"app", "filename":"app.py", "line-rate":"1.0", "branch-rate":"0"})
lines = ET.SubElement(klass, "lines")
for number in (1,2,5,6):
    ET.SubElement(lines, "line", {"number":str(number), "hits":"1"})
ET.ElementTree(coverage).write("reports/cobertura.xml", encoding="utf-8", xml_declaration=True)

for p in ("reports/junit.xml", "reports/cobertura.xml"):
    data = Path(p).read_bytes()
    print(f"REPORT {p} sha256={hashlib.sha256(data).hexdigest()}")
print("TOTAL 4 statements 4 covered 100.0%")
sys.exit(1 if failures else 0)

4. Upload two structured reports and preserve raw evidence

stages: [test, quality]

test_reports:
  stage: test
  image: python:3.12-alpine
  script:
    - python tools/run_checks.py | tee reports/tool-output.txt
    - python -c 'import xml.etree.ElementTree as ET; ET.parse("reports/junit.xml"); ET.parse("reports/cobertura.xml")'
    - sha256sum reports/* | tee reports/SHA256SUMS
  coverage: '/TOTAL .* ([0-9]+(?:\.[0-9]+)?)%$/'
  artifacts:
    when: always
    expire_in: 7 days
    paths:
      - reports/
    reports:
      junit: reports/junit.xml
      coverage_report:
        coverage_format: cobertura
        path: reports/cobertura.xml

code_quality_demo:
  stage: quality
  image: python:3.12-alpine
  script:
    - mkdir -p reports
    - python tools/write_code_quality.py
    - python -m json.tool reports/gl-code-quality-report.json >/dev/null
  artifacts:
    expire_in: 7 days
    paths:
      - reports/gl-code-quality-report.json
    reports:
      codequality: reports/gl-code-quality-report.json

artifacts:reports enables GitLab parsing. artifacts:paths makes the same report files easy to browse as raw evidence. The explicit seven-day retention is long enough for this disposable exercise without turning the lab into indefinite storage.

5. Add a synthetic Code Quality report

The Code Quality schema is a JSON array. Each finding needs a human-readable description, a check name, a stable fingerprint, a repository-relative location, and a supported severity.

# tools/write_code_quality.py
import json
from pathlib import Path
Path("reports").mkdir(exist_ok=True)
report = [{
    "description": "Synthetic maintainability note for the Chapter 24 lab.",
    "check_name": "chapter24-demo",
    "fingerprint": "chapter24-demo-app-classify-line-5",
    "severity": "minor",
    "location": {"path": "app.py", "lines": {"begin": 5}}
}]
Path("reports/gl-code-quality-report.json").write_text(json.dumps(report, indent=2), encoding="utf-8")

This finding is intentionally advisory. If your policy requires a linter to block delivery, the linter process itself should return a blocking exit code under a documented threshold. Do not assume the existence of a Code Quality report automatically creates that policy.

6. Expected observations after the pipeline

7. Accessibility and Browser Performance: inspect before adopting

Accessibility report ingestion is Free-tier and GitLab documents a Pa11y-based template/report flow. For this lab, do not create an external URL just to exercise the feature. Instead, inspect the current template/documentation and record the report filename, target URL trust boundary, and whether accessibility findings are blocking in your intended policy.

Browser Performance is Premium/Ultimate. It relies on a sitespeed.io-generated browser-performance JSON artifact and merge-request comparison semantics; GitLab cannot combine multiple browser-performance reports. Treat it as an optional architecture extension. A Free-tier learner can faithfully simulate the decision by retaining a synthetic performance JSON plus a threshold-check exit code without claiming GitLab rendered the paid widget.

8. Before/after evidence table

9. Challenge: choose the layer, not a copy-paste fix

Your test job is green, GitLab shows three JUnit cases with one failure, and the raw JUnit checksum matches the current job. Which layer is wrong?

Do not edit the report to hide the failure. Inspect the tool/script exit semantics. If the command swallowed a non-zero result or generated a report independently from the process status, repair the execution-policy layer so the job status matches your intended blocking rule while keeping the original failing report as evidence.

10. Cleanup

The lab mutates only the disposable repository and GitLab job-artifact state. Delete the throwaway project when finished, or retain it until the seven-day artifact expiry completes the exercise. There is no cloud, registry, environment, or secret cleanup.

Knowledge check

Official references and version notes

Documentation verification date: 2026-09-12. JUnit/unit-test reports, coverage percentage extraction, Cobertura/JaCoCo coverage visualization, Code Quality report import, and Accessibility reports are available on Free/Premium/Ultimate. Browser Performance reports are Premium/Ultimate. Code Quality's built-in CodeClimate-based template was deprecated in GitLab 17.3 and is planned for removal in GitLab 19.0; current guidance is to integrate a supported tool's report directly. The mandatory lab intentionally uses direct report generation and direct Code Quality import rather than the deprecated built-in CodeClimate template. Accessibility and Browser Performance are separated by current tier/runner prerequisites.

• Unit test reports — official reference.
• Unit test report examples — official reference.
• Code coverage — official reference.
• Coverage reporting — official reference.
• Coverage visualization — official reference.
• Cobertura coverage visualization — official reference.
• Code Quality — official reference.
• Accessibility testing — official reference.
• Browser performance testing — official reference.
• Artifacts reports types — official reference.
• Job artifacts — official reference.
• CI/CD YAML syntax reference — official reference.

Current assumptions used in this chapter: Version-sensitive YAML, Runner/executor behavior, APIs, security features, policy controls, tiers, and deprecations must be verified against the exact GitLab, GitLab Runner, tool, and external-system versions used in production. Preserve the exact project/ref/SHA, compiled configuration, pipeline/job IDs, runner/tool versions, and external target evidence used for reproducible work.