Coverage, Test Execution Data, Duplication, and External Analyzer Reports: Guided Hands-On Workflow

1. Disposable workflow contract

2. Create the tested Python fixture

Before running the scanner, create the disposable project academy-sq-ch14 with a lab administrator identity (UI or approved Web API), then generate a project-analysis token scoped to that project and place only its value in the SONAR_TOKEN environment variable. Do not use a system-administrator token for routine analysis.

mkdir -p academy-sq-ch14/src academy-sq-ch14/tests academy-sq-ch14/reports academy-sq-ch14/evidence
cd academy-sq-ch14
git init
git config user.name "Academy Learner"
git config user.email "academy@example.invalid"

cat > .gitignore <<'EOF'
.venv/
.scannerwork/
.coverage
__pycache__/
.pytest_cache/
reports/
evidence/
EOF

python -m venv .venv
# Linux/macOS:
. .venv/bin/activate
# Windows PowerShell: .\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install pytest pytest-cov coverage

cat > src/cart.py <<'PY'
def subtotal(lines):
    total = 0.0
    for quantity, unit_price in lines:
        if quantity < 0 or unit_price < 0:
            raise ValueError("negative input")
        total += quantity * unit_price
    return round(total, 2)


def shipping(total, expedited=False):
    if total >= 100:
        return 0.0
    if expedited:
        return 15.0
    return 5.0
PY

cat > tests/test_cart.py <<'PY'
import pytest
from src.cart import subtotal, shipping


def test_subtotal():
    assert subtotal([(2, 10.0), (1, 5.5)]) == 25.5


def test_negative_input():
    with pytest.raises(ValueError):
        subtotal([(-1, 10.0)])


def test_shipping():
    assert shipping(120.0) == 0.0
    assert shipping(20.0) == 5.0
PY

cat > .coveragerc <<'EOF'
[run]
relative_files = True
source = src

[report]
show_missing = True
EOF

cat > sonar-project.properties <<'EOF'
sonar.projectKey=academy-sq-ch14
sonar.projectName=Academy SonarQube Chapter 14
sonar.sources=src
sonar.tests=tests
sonar.test.inclusions=tests/**/*.py
sonar.python.version=3.11
sonar.python.coverage.reportPaths=reports/coverage.xml
sonar.python.xunit.reportPath=reports/junit.xml
sonar.sourceEncoding=UTF-8
EOF

git add .gitignore .coveragerc sonar-project.properties src tests
git commit -m "ch14 coverage fixture"
git rev-parse HEAD | tee evidence/revision.txt

3. Produce reports before scanning

python -m pytest -q \
  --cov=src \
  --cov-report=term-missing \
  --cov-report=xml:reports/coverage.xml \
  --junitxml=reports/junit.xml

ls -l reports
python - <<'PY'
from pathlib import Path
import xml.etree.ElementTree as ET
for name in ('coverage.xml','junit.xml'):
    p=Path('reports')/name
    print(name, p.stat().st_size, 'bytes')
    root=ET.parse(p).getroot()
    print(' root=', root.tag)
PY
sha256sum reports/coverage.xml reports/junit.xml | tee evidence/report-sha256.txt

Preserve the producer output. If pytest fails, stop here; do not scan stale XML from a prior successful run.

4. Run the intentionally broken coverage-path analysis

Keep the source revision and generated report fixed. Override only the coverage path so the scanner looks at a file that does not exist.

export SONAR_HOST_URL="http://localhost:9000"
# SONAR_TOKEN is already set securely in the shell environment.

sonar-scanner -X \
  -Dsonar.python.coverage.reportPaths=reports/does-not-exist.xml \
  2>&1 | tee evidence/broken-scan.log

cp .scannerwork/report-task.txt evidence/broken-report-task.txt
cat evidence/broken-report-task.txt

Expected evidence is release-sensitive in exact wording, but the Python coverage sensor should show that the configured path/pattern produced no usable coverage report. The scanner may still upload an analysis; therefore preserve report-task.txt and check the Compute Engine task separately.

5. Follow the broken run through Compute Engine

CE_TASK_ID=$(grep '^ceTaskId=' evidence/broken-report-task.txt | cut -d= -f2)
printf '%s\n' "$CE_TASK_ID" > evidence/broken-ce-task-id.txt

# Read-only API path if SONAR_API_TOKEN is available:
curl -fsS -H "Authorization: Bearer $SONAR_API_TOKEN" --get \
  --data-urlencode "id=$CE_TASK_ID" \
  "$SONAR_HOST_URL/api/ce/task" | tee evidence/broken-ce-task.json

curl -fsS -H "Authorization: Bearer $SONAR_API_TOKEN" --get \
  --data-urlencode "component=academy-sq-ch14" \
  --data-urlencode "metricKeys=coverage,line_coverage,lines_to_cover,uncovered_lines,tests,test_failures,test_errors,skipped_tests,duplicated_lines,duplicated_lines_density" \
  "$SONAR_HOST_URL/api/measures/component" | tee evidence/broken-measures.json

If the coverage measure is absent or unexpectedly low, do not infer “tests did not run.” The pytest artifact proves test execution; the scanner log proves import failure. Those are distinct states.

6. Correct only the import path and reanalyze the same revision

test -s reports/coverage.xml
test -s reports/junit.xml
git status --short
git rev-parse HEAD

# No command-line override: sonar-project.properties points to reports/coverage.xml.
sonar-scanner -X 2>&1 | tee evidence/correct-scan.log
cp .scannerwork/report-task.txt evidence/correct-report-task.txt

CE_TASK_ID=$(grep '^ceTaskId=' evidence/correct-report-task.txt | cut -d= -f2)
curl -fsS -H "Authorization: Bearer $SONAR_API_TOKEN" --get \
  --data-urlencode "id=$CE_TASK_ID" \
  "$SONAR_HOST_URL/api/ce/task" | tee evidence/correct-ce-task.json

curl -fsS -H "Authorization: Bearer $SONAR_API_TOKEN" --get \
  --data-urlencode "component=academy-sq-ch14" \
  --data-urlencode "metricKeys=coverage,line_coverage,lines_to_cover,uncovered_lines,tests,test_failures,test_errors,skipped_tests,test_execution_time,duplicated_lines,duplicated_lines_density" \
  "$SONAR_HOST_URL/api/measures/component" | tee evidence/correct-measures.json

The causal variable is the coverage-report path. The source SHA, test XML bytes, coverage XML bytes, scanner/server versions, profile, gate, and project key remain fixed. That makes the measure difference attributable to import alignment rather than code or policy manipulation.

7. Optional duplication fixture: prove CPD is not an imported report

If you want a visible duplication metric, add two intentionally repeated synthetic files in a new revision. Make the repeated block comfortably larger than the current non-Java threshold so the experiment does not depend on borderline tokenization.

cat > /tmp/repeated.py <<'PY'
def normalize_order(order):
    customer = order.get("customer", "unknown")
    country = order.get("country", "US")
    subtotal = float(order.get("subtotal", 0))
    tax = float(order.get("tax", 0))
    shipping = float(order.get("shipping", 0))
    discount = float(order.get("discount", 0))
    if subtotal < 0:
        subtotal = 0.0
    if tax < 0:
        tax = 0.0
    if shipping < 0:
        shipping = 0.0
    if discount < 0:
        discount = 0.0
    gross = subtotal + tax + shipping
    net = max(0.0, gross - discount)
    return {"customer": customer, "country": country, "gross": gross, "net": net,
            "tax": tax, "shipping": shipping, "discount": discount, "subtotal": subtotal}
PY
cp /tmp/repeated.py src/dup_a.py
cp /tmp/repeated.py src/dup_b.py
git add src/dup_a.py src/dup_b.py
git commit -m "add explicit CPD fixture"
sonar-scanner
# Then inspect duplicated_lines / duplicated_blocks / duplicated_lines_density.

Do not carry this intentionally duplicated fixture into real product code. It exists only to make the detection boundary observable.

8. Optional SARIF import in a separate disposable project

Keep external-issue experimentation separate from the coverage project so it does not change the coverage lab gate unexpectedly.

mkdir -p ../academy-sq-ch14-sarif/src ../academy-sq-ch14-sarif/reports
cd ../academy-sq-ch14-sarif
printf 'def demo():\n    return 1\n' > src/demo.py
cat > reports/demo.sarif <<'JSON'
{
  "version": "2.1.0",
  "$schema": "https://json.schemastore.org/sarif-2.1.0.json",
  "runs": [{
    "tool": {"driver": {"name": "AcademyDemoAnalyzer", "rules": [{
      "id": "ACADEMY001",
      "shortDescription": {"text": "Synthetic training finding"},
      "defaultConfiguration": {"level": "warning"}
    }]}},
    "results": [{
      "ruleId": "ACADEMY001",
      "level": "warning",
      "message": {"text": "Synthetic external finding for import provenance training."},
      "locations": [{"physicalLocation": {
        "artifactLocation": {"uri": "src/demo.py"},
        "region": {"startLine": 1, "startColumn": 1}
      }}]
    }]
  }]
}
JSON
cat > sonar-project.properties <<'EOF'
sonar.projectKey=academy-sq-ch14-sarif
sonar.sources=src
sonar.sarifReportPaths=reports/demo.sarif
sonar.python.version=3.11
EOF
# Use a separate project-analysis token in SONAR_TOKEN, then run sonar-scanner.

After import, verify the issue is external and that its rule is not a native profile rule. Preserve the SARIF producer name/rule ID with the issue evidence.

9. Challenge: choose the broken layer

A CI pipeline shows: pytest PASS; reports/coverage.xml exists in job A; Sonar job B reports no coverage. You may change only one thing first. Which layer should you inspect?

Answer: artifact/workspace transfer and scanner-visible report path. Do not change the gate, profile, source scope, or coverage exclusions before proving whether job B actually possesses the report at the configured path.

Knowledge check

Official references and version notes

• Test coverage overview — SonarQube consumes coverage reports generated by external build/test tools; it does not execute tests.
• Test coverage parameters — language-specific and generic coverage import keys including sonar.python.coverage.reportPaths and sonar.coverageReportPaths.
• Test execution parameters — execution-report keys including sonar.testExecutionReportPaths and sonar.python.xunit.reportPath; execution reports are branch-only.
• Generic test data — generic coverage and execution XML formats when a producer has no native integration.
• About external issues — supported external-analyzer integrations and how imported results participate in analysis.
• External analyzer reports — analyzer-specific report paths including Python Pylint/Bandit/Flake8/Mypy/Ruff integrations.
• Generic external-issue format — sonar.externalIssuesReportPaths and ownership semantics for third-party rules.
• SARIF reports — SARIF 2.1.0 requirements and sonar.sarifReportPaths.
• Scanner-only parameters — external report paths, report-task.txt, quality-gate wait behavior, and path resolution.
• Metric definitions — coverage/test metrics and current duplication/CPD thresholds and keys.
• SonarQube downloads — current Community Build release identity.
• SonarScanner CLI 8.1.0.6389 — standalone scanner baseline used in the local lab.