Coverage, Test Execution Data, Duplication, and External Analyzer Reports: Diagnostics, Failure Modes, and Production Practices

1. Evidence-first diagnostic sequence

1. Preserve first-failure evidence: producer logs, report bytes/hash, scanner logs, report-task.txt, CE task, API/UI result.
2. Confirm exact SonarQube edition/version, scanner, language/runtime, producer tool, and external analyzer versions.
3. Confirm source revision and sonar.projectBaseDir.
4. Prove the report was generated before the scan and belongs to the same revision/build.
5. Validate format and scanner-visible path.
6. Inspect the corresponding scanner sensor/import log.
7. Confirm report upload and Compute Engine task success.
8. Inspect the persisted coverage/test/duplication/external-issue result.
9. Only then inspect policy/gate effects.
10. Apply the least destructive correction and rerun the smallest equivalent scenario.

2. Failure: “SonarQube should have run my tests”

Symptom: no coverage, but no test command exists in the pipeline.

Cause: scanner executed without a producer artifact.

Repair: add the test/coverage command before analysis, fail the pipeline if tests or report generation fails, then prove report existence/hash before scanning.

Do not: lower the gate threshold, mark files excluded, or treat Zero Coverage Sensor output as proof that tests ran.

3. Failure: report generated after the scan

The 09:05 report belongs to no already-uploaded analysis. Reordering the pipeline is the fix. Retrying the server or Compute Engine cannot import a local file that was absent during scanner execution.

4. Failure: host path inside a scanner container

# Host producer creates:
/opt/ci/work/reports/coverage.xml

# Scanner container mounts repository at /workspace:
docker run --rm \
  -v "$PWD:/workspace" -w /workspace \
  ...scanner-image...

# Correct scanner-visible path:
reports/coverage.xml
# Wrong host-only path inside container:
/opt/ci/work/reports/coverage.xml

Inspect the actual mount and run ls -l reports/coverage.xml inside the scanner environment before changing Sonar properties.

5. Failure: stale report from another revision

A cached coverage.xml can exist and still be wrong. Compare build revision, report timestamps, artifact manifest, source paths in the XML, and CI job lineage. Prefer cleaning/recreating the report directory before the test run, then archive the new report with the source SHA.

6. Failure: incompatible report merge

Combining XML by string concatenation can create malformed documents or semantically invalid totals. Even valid XML can represent incompatible producer versions or path roots. Validate each input and use the producer’s supported merge operation or Sonar’s documented multiple-path support.

7. Failure: exclusions added to inflate coverage

Symptom: coverage rises sharply after sonar.coverage.exclusions=src/legacy/** without new tests.

Diagnosis: denominator changed, not test evidence. Preserve before/after lines_to_cover, uncovered_lines, source scope, exclusion setting, and change rationale.

Repair: restore the governed scope unless the excluded files are independently justified as generated/non-testable. Do not call the higher percentage an engineering improvement if it came only from scope removal.

8. Failure: expected duplication does not appear

Check:

• Are the files indexed and supported for duplication?
• Does the repeated region exceed current language-specific token/line thresholds?
• Is sonar.cpd.exclusions set globally/project/scanner-side?
• Do scanner logs show CPD calculating files?
• Did source scope change?

Do not invent a duplication report import. CPD is computed from source.

9. Failure: imported external finding is treated like a native rule

Symptom: team searches the quality profile for SARIF rule ACADEMY001 and cannot find it.

Cause: external rules remain external; generic/SARIF imported rule definitions are not native profile rules.

Repair: trace producer name/version/rule ID/report and manage the rule in the external analyzer. Use Sonar for visibility and policy integration, not as a false source of rule ownership.

10. Layer map for causal troubleshooting

11. Intentionally broken example: preserve and repair

# Preserve producer evidence.
cp reports/coverage.xml evidence/coverage-before-fix.xml
sha256sum reports/coverage.xml > evidence/coverage-before-fix.sha256

# Broken: scanner-visible file is reports/coverage.xml.
sonar-scanner -X -Dsonar.python.coverage.reportPaths=/host-only/work/coverage.xml \
  2>&1 | tee evidence/path-failure.log
cp .scannerwork/report-task.txt evidence/path-failure-report-task.txt

# Repair only the path; do not change source, exclusions, gate, or project key.
sonar-scanner -X -Dsonar.python.coverage.reportPaths=reports/coverage.xml \
  2>&1 | tee evidence/path-repaired.log

Compare both scanner sensor logs, both CE tasks, and the final coverage measure. The original failure remains in the packet rather than being erased.

12. Production anti-patterns to reject

• Blanket scanner retries without preserving the first report/import error.
• Deleting report directories/logs before copying evidence.
• Disabling TLS validation because a report import failed.
• Using admin tokens in test jobs that only need analysis.
• Direct database edits to “fix” coverage or issue state.
• Lowering gate thresholds after seeing missing-report results.
• Mass exclusions/suppressions to recover a green pipeline.
• Re-running analysis under a different project key to escape history.

Knowledge check

Official references and version notes

• Test coverage overview — SonarQube consumes coverage reports generated by external build/test tools; it does not execute tests.
• Test coverage parameters — language-specific and generic coverage import keys including sonar.python.coverage.reportPaths and sonar.coverageReportPaths.
• Test execution parameters — execution-report keys including sonar.testExecutionReportPaths and sonar.python.xunit.reportPath; execution reports are branch-only.
• Generic test data — generic coverage and execution XML formats when a producer has no native integration.
• About external issues — supported external-analyzer integrations and how imported results participate in analysis.
• External analyzer reports — analyzer-specific report paths including Python Pylint/Bandit/Flake8/Mypy/Ruff integrations.
• Generic external-issue format — sonar.externalIssuesReportPaths and ownership semantics for third-party rules.
• SARIF reports — SARIF 2.1.0 requirements and sonar.sarifReportPaths.
• Scanner-only parameters — external report paths, report-task.txt, quality-gate wait behavior, and path resolution.
• Metric definitions — coverage/test metrics and current duplication/CPD thresholds and keys.
• SonarQube downloads — current Community Build release identity.
• SonarScanner CLI 8.1.0.6389 — standalone scanner baseline used in the local lab.