Checkpoint Lab — Testing and Quality Pipelines: JUnit, Coverage, SonarQube, Selenium, JMeter, and Quality Gates

1. Scenario

You own jenkins-labs/quality-checkpoint. The job builds synthetic Python code on a disposable quality-lab agent. You will execute three separate Jenkins builds against the same committed source:

1. baseline — tests pass and reports ingest normally.
2. failing-test — one controlled assertion fails, but JUnit XML is still produced and retained.
3. missing-report — the test command succeeds but the configured JUnit XML path is intentionally absent, so ingestion fails.

An optional fourth coverage-gate run reaches a separate quality-policy failure after successful tests and report ingestion.

2. Predict before running

3. Preflight

• Disposable Jenkins LTS 2.568.3 controller with Java 21.
• A separate agent labeled quality-lab; no routine build on built-in node.
• JUnit and Coverage plugin versions recorded.
• Python 3.10+ and Git available on the agent.
• No real credentials are required for the mandatory lab.
• Workspace path and free disk recorded.
• Job full name, build number/URL/cause and source SHA captured for every scenario.

set -euo pipefail
printf 'source=%s\n' "$(git rev-parse HEAD)"
printf 'workspace=%s\n' "$PWD"
python3 --version
java -version 2>&1 | head -n 1
df -h . | tail -n 1

4. Checkpoint project files

set -euo pipefail
mkdir -p tests ci reports
cat > quality_math.py <<'PYAPP'
def divide(a, b):
    if b == 0:
        raise ValueError("b must not be zero")
    return a / b

def band(n):
    if n < 0:
        return "negative"
    if n == 0:
        return "zero"
    return "positive"
PYAPP
cat > tests/test_quality_math.py <<'PYTEST'
import os
import pytest
from quality_math import divide, band

def test_divide():
    assert divide(8, 2) == 4

def test_zero_guard():
    with pytest.raises(ValueError):
        divide(1, 0)

def test_band():
    assert band(0) == "zero"

def test_controlled_failure():
    if os.environ.get("QUALITY_SCENARIO") == "failing-test":
        assert band(3) == "negative"
PYTEST
cat > requirements-ci.txt <<'REQ'
pytest==9.1.1
coverage==7.16.1
REQ

5. Evidence-producing runner

The script deliberately never hides the test runner’s original status. In the missing-report scenario it changes only the configured report path after successful execution so the ingestion failure is cleanly attributable.

#!/usr/bin/env bash
set -u -o pipefail
SCENARIO="${QUALITY_SCENARIO:-baseline}"
mkdir -p reports
rm -f reports/junit.xml reports/junit.actual.xml reports/coverage.xml reports/*.txt .coverage

. .venv-quality/bin/activate
printf 'scenario=%s\n' "$SCENARIO" | tee reports/scenario.txt
git rev-parse HEAD | tee reports/source-sha.txt
python --version > reports/python-version.txt 2>&1
python -m pytest --version > reports/pytest-version.txt 2>&1
python -m coverage --version > reports/coverage-version.txt 2>&1

set +e
QUALITY_SCENARIO="$SCENARIO" \
  python -m coverage run --branch -m pytest -q --junitxml=reports/junit.actual.xml
test_rc=$?
set -e
printf '%s\n' "$test_rc" > reports/test-exit-code.txt

# Coverage generation is attempted independently from JUnit publication.
python -m coverage xml -o reports/coverage.xml || printf 'coverage_xml_failed\n' > reports/coverage-error.txt

case "$SCENARIO" in
  baseline|failing-test|coverage-gate)
    cp reports/junit.actual.xml reports/junit.xml
    ;;
  missing-report)
    # Intentionally leave reports/junit.xml absent.
    ;;
  *)
    echo "unknown scenario: $SCENARIO" >&2
    exit 64
    ;;
esac

find reports -maxdepth 1 -type f -printf '%f %s bytes\n' | sort > reports/file-manifest.txt
exit "$test_rc"

Commit this as ci/run-quality.sh and make it executable. The missing-report scenario is intentionally synthetic and must remain confined to this lab job.

6. Local policy simulator

This small script demonstrates a separate policy owner without pretending to be SonarQube.

# ci/local_quality_gate.py
import json
from pathlib import Path

scenario = Path('reports/scenario.txt').read_text().strip().split('=', 1)[1]
status = 'ERROR' if scenario == 'coverage-gate' else 'OK'
evidence = {
    'gateId': 'local-quality-gate-v1',
    'scenario': scenario,
    'status': status,
    'policy': {'lineCoverageMinimum': 80.0},
}
Path('reports/quality-gate.json').write_text(json.dumps(evidence, indent=2) + '\n')
print(json.dumps(evidence))
raise SystemExit(0 if status == 'OK' else 42)

7. Checkpoint Jenkinsfile

pipeline {
  agent { label 'quality-lab' }
  parameters {
    choice(name: 'QUALITY_SCENARIO',
           choices: ['baseline', 'failing-test', 'missing-report', 'coverage-gate'],
           description: 'Controlled checkpoint scenario')
  }
  options {
    timestamps()
    disableConcurrentBuilds()
    buildDiscarder(logRotator(numToKeepStr: '30'))
  }
  stages {
    stage('Preflight') {
      steps {
        checkout scm
        sh '''
          set -euo pipefail
          git rev-parse HEAD
          java -version
          python3 --version
          python3 -m venv .venv-quality
          . .venv-quality/bin/activate
          python -m pip install --requirement requirements-ci.txt
        '''
      }
    }

    stage('Execute quality tools') {
      steps {
        script {
          env.TEST_RC = sh(
            returnStatus: true,
            script: "QUALITY_SCENARIO='${params.QUALITY_SCENARIO}' bash ci/run-quality.sh"
          ).toString()
        }
      }
      post {
        always {
          archiveArtifacts artifacts: 'reports/**', allowEmptyArchive: true, fingerprint: true
        }
      }
    }

    stage('Ingest JUnit evidence') {
      steps {
        junit testResults: 'reports/junit.xml',
              allowEmptyResults: false,
              skipPublishingChecks: true
      }
    }

    stage('Ingest coverage evidence') {
      when { expression { fileExists('reports/coverage.xml') } }
      steps {
        recordCoverage(
          tools: [[parser: 'COBERTURA', pattern: 'reports/coverage.xml']],
          id: 'checkpoint-coverage', name: 'Checkpoint Coverage',
          sourceCodeRetention: 'MODIFIED'
        )
      }
    }

    stage('Local quality policy') {
      when {
        expression { params.QUALITY_SCENARIO != 'missing-report' && env.TEST_RC == '0' }
      }
      steps {
        sh '. .venv-quality/bin/activate && python ci/local_quality_gate.py'
      }
    }

    stage('Execution policy') {
      when { expression { params.QUALITY_SCENARIO != 'missing-report' } }
      steps {
        script {
          if (env.TEST_RC != '0') {
            error("Original test runner exit code was ${env.TEST_RC}")
          }
        }
      }
    }
  }
  post {
    always {
      echo "scenario=${params.QUALITY_SCENARIO} test_rc=${env.TEST_RC ?: 'not-set'}"
    }
  }
}

The stage order is intentional. In failing-test, JUnit gets a chance to parse the failed testcase before the final execution policy throws. In missing-report, the JUnit stage itself fails despite TEST_RC=0, proving a separate ingestion failure. The coverage-gate run can reach policy with valid tests and reports, then fail there.

8. Required run matrix

9. Required evidence packet

10. Verification checklist

• All builds run on quality-lab, not the built-in node.
• All required builds use the same committed source SHA unless the evidence packet explains otherwise.
• Baseline: original test exit is 0 and JUnit ingestion succeeds.
• Failing-test: original exit is non-zero and JUnit XML exists with a failed testcase.
• Missing-report: original exit is 0 and the JUnit publisher fails because the configured file is absent.
• Coverage XML is separately retained and, when present, parsed independently of JUnit.
• No || true or equivalent hides the original test failure.
• Raw evidence is archived before cleanup.
• No real token, private repository, production URL or uncontrolled load target appears anywhere.

11. Optional real SonarQube extension

Replace only ci/local_quality_gate.py with a disposable SonarQube integration. Keep the same evidence fields: source SHA, project key, analysis task ID, webhook delivery, gate status and timeout. Use SonarQube Scanner plugin 2.19.0 or a currently reviewed successor and the maintained waitForQualityGate flow.

12. Optional Selenium/JMeter extension

Add one Selenium smoke test and one low-volume JMeter CLI test against a service bound to 127.0.0.1. Publish Selenium JUnit XML under a separate check/report name and retain raw JTL/logs. Do not make either extension mandatory if the agent lacks a pinned browser or JMeter installation.

13. Cleanup / rollback

set -euo pipefail
case "$PWD" in /|/home|/var|/tmp) echo 'unsafe cleanup location' >&2; exit 70;; esac
rm -rf -- .venv-quality .coverage reports web
# Keep committed source and Jenkins archived evidence until review is complete.

Delete the disposable Jenkins job/agent only by exact name after confirming the evidence packet is exported or retained according to your lab policy. If optional SonarQube resources were created, delete only the exact lab project/token/webhook.

14. What the checkpoint proves—and does not prove

15. Production model and bridge to Chapter 34

You now have a quality evidence chain that keeps tool execution, raw reports, Jenkins ingestion, external analysis and policy outcome separate. Chapter 34 takes those exact build/source/policy states and publishes actionable feedback—SCM checks, commit statuses, pull-request comments, chat/email notifications and release communication—without leaking secrets or creating retry spam.

Knowledge check

Answer before revealing the explanation.

Official references and version notes

Test-report schemas, Jenkins plugins, SonarQube integration and browser/load-test tooling evolve. Prefer current primary documentation.

• Jenkins LTS changelog
• Jenkins Java Support Policy
• Jenkins Security Advisories
• JUnit plugin
• Coverage plugin
• SonarQube Scanner plugin
• SonarQube Server 2026.1 — Jenkins analysis integration
• SonarQube Server 2026.1 — waitForQualityGate / pipeline pause
• Selenium documentation
• Apache JMeter User Manual
• Jenkins Performance plugin
• pytest
• coverage.py