DevOps AcademyLearn the delivery system
Chapter 14Lesson 03~120 minutes

Coverage, Test Execution Data, Duplication, and External Analyzer Reports: Configuration, Design Patterns, and Trade-Offs

Choose coverage versus test execution, native versus external findings, report-merging and path-mapping strategies, coverage/duplication exclusions, and local-versus-container layouts using explicit provenance and rollback criteria.

Path mappingExclusionsCI containersTrade-offsGovernance

Learning objectives

  • Choose language-native report parameters versus generic Sonar formats based on producer compatibility and long-term maintainability.
  • Keep coverage and test-execution data separate even when the same test run produces both artifacts.
  • Use coverage exclusions only for legitimate non-testable scope decisions and keep them distinct from source exclusions and CPD exclusions.
  • Design report paths that survive local, CI, container, and monorepo workspace changes without embedding host-only absolute paths.
  • Distinguish native Sonar rules from imported external rules whose lifecycle and configuration remain owned by the third-party analyzer.
  • Evaluate single/merged report strategies using provenance, compatibility, failure isolation, portability, and auditability.

1. Configuration is an evidence-supply-chain design

A report import setting is not “just another scanner flag.” It couples a producer, format, path, workspace, scanner version, language analyzer, server version, and downstream policy. Good design makes those dependencies explicit and reproducible.

2. Coverage versus test execution: keep separate contracts

Decision Coverage Test execution
Question What code/conditions were exercised? What tests ran and what happened?
Typical producer coverage.py, JaCoCo, LCOV producer pytest/JUnit/NUnit/xUnit/etc.
Failure symptom Missing/zero coverage or uncovered lines Missing test count/failure/time measures
PR support Depends on coverage/language path and PR feature support Current Community Build execution-report import is branch-only
Governance Often gate-relevant Usually diagnostic/operational evidence

One producer command may generate both, but retain separate files, hashes, parameters, and assertions. Combining semantic roles into one “test report” label makes diagnosis harder.

3. Native producer format versus generic conversion

Prefer a documented language-native parameter when SonarQube directly supports the producer format—for example Python Cobertura XML through sonar.python.coverage.reportPaths. Use generic coverage/test formats when the producer is unsupported and conversion is stable and tested.

Native path

Less custom transformation, clearer vendor documentation, but tied to language/tool integration.

Generic path

Portable across unsupported tools, but the conversion becomes your owned build artifact and must be versioned/tested.

Do not merge unlike XML documents just because they share an extension. “XML” is serialization, not a semantic format.

4. Single versus merged reports

Multiple test shards or modules can create multiple compatible coverage reports. Merge at the producer layer when the producer officially supports it, or use a Sonar property that accepts multiple paths where documented. Preserve per-shard artifacts as provenance even if a merged file is used for scanning.

shard reports → validated compatible merge → merged report hash → scanner import

Avoid blind concatenation. XML documents, LCOV records, JaCoCo reports, and Cobertura outputs have different merge semantics.

5. Source exclusions, coverage exclusions, and duplication exclusions are not interchangeable

Mechanism Effect Governance question
sonar.exclusions Removes files from source analysis scope Should these files be analyzed at all?
sonar.coverage.exclusions Keeps files analyzed but omits them from coverage calculation Are they legitimately non-testable/generated/configuration artifacts?
sonar.cpd.exclusions Omits files from duplication detection Is repeated generated/vendor content distorting CPD?

Changing any exclusion alters the measurement population. Record ownership, reason, affected paths, before/after measures, and rollback. Never add broad exclusions after seeing an unfavorable number simply to improve the dashboard.

6. Local, CI, and container path patterns

Pattern Strength Failure mode
Repo-relative reports/coverage.xml Portable and reviewable Producer/scanner use different working roots
Absolute host path Unambiguous on one machine Breaks on agents/containers/other OSes
CI artifact download into fixed repo-relative directory Clear stage boundary and provenance Artifact not downloaded or wrong revision
Container bind/volume mount Explicit namespace mapping Host path configured instead of container-visible path

Recommended default: materialize all reports under a predictable project-relative reports/ directory before starting the scanner, then prove that directory inside the scanner execution environment.

7. Native findings versus external findings

Native Sonar findings are produced by active quality-profile rules. External findings come with external rule IDs and lifecycle/configuration owned outside the profile. They can still affect analysis/gate status, but do not pretend their remediation/status semantics are identical to native rules.

Dedicated analyzer integration

Use documented producer output and analyzer-specific property; simplest when supported.

Generic Sonar issue format

Useful for unsupported producers; your converter owns schema correctness and rule metadata.

SARIF 2.1.0

Standard interchange; Sonar maps SARIF severities into current MQR/Standard semantics. Preserve producer name/rule IDs.

8. Ownership and least privilege

  • Test team/build owns producer commands and tool versions.
  • Repository/CI owns report placement and transfer.
  • Sonar project admins own import settings and exclusions.
  • Security/quality governance owns policy exceptions.
  • External analyzer owners own rule configuration for imported issues.
  • Scanner token needs analysis permission, not broad system administration.

9. Worked decision table

Scenario Choice Why Evidence
Python unit coverage pytest-cov → Cobertura XML → sonar.python.coverage.reportPaths Direct supported path Tool version, XML hash, scanner import log, coverage measure
Unsupported in-house test harness Convert to Generic coverage/execution format No native integration Converter version/tests + schema-valid output
Two CI stages Publish/download report artifact Scanner stage cannot see producer filesystem Artifact manifest, revision, hash, final path
Third-party security linter Dedicated integration or SARIF Preserve external ownership Producer/rule/version + report + imported issue
Generated code hurts coverage Evaluate narrow coverage exclusion Keep code analysis while excluding non-testable generated surface Generator ownership, path pattern, rationale, before/after denominator

Knowledge check

When is generic coverage preferable to a language-native parameter?

Why is sonar.coverage.exclusions safer than sonar.exclusions for genuinely non-testable generated source?

Can you simply concatenate two Cobertura XML files?

Who owns an imported SARIF rule’s configuration?

What is the first design preference for report paths?

Next lesson

Diagnose failures without hiding the evidence

Lesson 4 applies these design contracts to intentionally broken import and metric scenarios.

Official references and version notes

  • Test coverage overview — SonarQube consumes coverage reports generated by external build/test tools; it does not execute tests.
  • Test coverage parameters — language-specific and generic coverage import keys including sonar.python.coverage.reportPaths and sonar.coverageReportPaths.
  • Test execution parameters — execution-report keys including sonar.testExecutionReportPaths and sonar.python.xunit.reportPath; execution reports are branch-only.
  • Generic test data — generic coverage and execution XML formats when a producer has no native integration.
  • About external issues — supported external-analyzer integrations and how imported results participate in analysis.
  • External analyzer reports — analyzer-specific report paths including Python Pylint/Bandit/Flake8/Mypy/Ruff integrations.
  • Generic external-issue format — sonar.externalIssuesReportPaths and ownership semantics for third-party rules.
  • SARIF reports — SARIF 2.1.0 requirements and sonar.sarifReportPaths.
  • Scanner-only parameters — external report paths, report-task.txt, quality-gate wait behavior, and path resolution.
  • Metric definitions — coverage/test metrics and current duplication/CPD thresholds and keys.
  • SonarQube downloads — current Community Build release identity.
  • SonarScanner CLI 8.1.0.6389 — standalone scanner baseline used in the local lab.
Version and compatibility note

Rechecked 2026-09-07. Mandatory examples target SonarQube Community Build 26.9.0.129388 and SonarScanner CLI 8.1.0.6389. The Python lab uses Python 3.11+ with pytest/pytest-cov and Cobertura XML. Current Community Build accepts Python coverage through sonar.python.coverage.reportPaths, Python xUnit execution data through sonar.python.xunit.reportPath, generic coverage through sonar.coverageReportPaths, generic test execution through sonar.testExecutionReportPaths, generic external issues through sonar.externalIssuesReportPaths, and SARIF through sonar.sarifReportPaths. Paths are interpreted relative to sonar.projectBaseDir unless the property documents otherwise. Duplication is calculated from indexed source; non-Java duplication currently requires at least 100 successive duplicated tokens spread across at least 10 lines for languages other than COBOL/ABAP, while Java uses 10 successive duplicated statements. Re-check producer format compatibility and report parameters before applying these examples to another release/language.

Keep the academy open

Support free, practical DevOps education.

Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.

Ethereum / ERC-20
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0Send only Ethereum/ERC-20 compatible assets to this address.