Gradle Source Sets, JVM Test Suites, Test Filtering, Reporting, and Integration Testing: Diagnostics, Failure Modes, Security, and Performance

1. Diagnostic sequence

Use the same evidence-first order established in earlier chapters:

1. Preserve concise console output and the affected XML/HTML report.
2. Confirm Wrapper, Gradle, and JDK identity.
3. Inspect suite/build configuration and lifecycle declarations.
4. Inspect task graph plus test compile/runtime classpaths.
5. Inspect fresh report/output directories and isolated Gradle state.
6. Classify the failure: compilation, discovery/filtering, framework startup, assertion, resource/classpath, or infrastructure.
7. Apply the least destructive correction.
8. Run a controlled clean verification and compare evidence.

Do not start by deleting ~/.gradle, disabling filters, ignoring failures, or removing the integration suite from CI.

2. Failure mode: suite exists but check never schedules it

The build can contain a perfectly valid integrationTest task and still produce a green check that never runs it.

// Broken governance: integrationTest exists, but there is no check dependency.
testing {
    suites {
        register<JvmTestSuite>("integrationTest") {
            useJUnitJupiter("6.1.3")
            dependencies { implementation(project()) }
        }
    }
}

./gradlew clean check --dry-run
./gradlew clean check
find build/test-results -type f -name 'TEST-*.xml' -print | sort

# Now execute the missing evidence directly.
./gradlew integrationTest

If check omits the integration task but direct execution fails, the problem is lifecycle topology—not flaky tests. Repair the explicit dependsOn edge and rerun check --dry-run before the real build.

3. Failure mode: custom suite misses production/resource runtime state

Additional suites do not automatically see production output. In the chapter lab, remove implementation(project()) but leave GreetingResourceIT unchanged. The test itself has only JDK/JUnit compile requirements, so it can compile; at runtime the classloader lookup for /academy-banner.txt returns null.

./gradlew clean integrationTest --info
./gradlew dependencies --configuration integrationTestRuntimeClasspath
# Compare with the same report after restoring implementation(project()).

The assertion message “production resource should be visible” points at a classpath/output relationship. Do not copy the resource into the test source set to hide the problem if the intended contract is specifically to verify production packaging.

4. Failure mode: filter excludes every test

A typo in --tests should be loud. Current TestFilter defaults to fail when an explicit filter matches nothing.

set +e
./gradlew integrationTest --tests '*IntegrationDoesNotExist*' > filter-failure.log 2>&1
status=$?
set -e
printf 'exit=%s
' "$status"
grep -Ei 'No tests found|matching|filter' filter-failure.log || tail -n 30 filter-failure.log

Current Gradle exposes the explicit-filter control as filter.isFailOnNoMatchingTests (true by default). Separately, Test.failOnNoDiscoveredTests protects a different condition: test sources are present but the framework discovers no tests. Preserve both safety nets unless there is a documented exceptional workflow.

5. Failure mode: two tasks write the same report destination

// Intentionally broken: do not use this configuration.
tasks.named<Test>("test") {
    reports.junitXml.outputLocation.set(layout.buildDirectory.dir("test-results/shared"))
}
tasks.named<Test>("integrationTest") {
    reports.junitXml.outputLocation.set(layout.buildDirectory.dir("test-results/shared"))
}

Even if task execution itself succeeds, the artifact set is ambiguous and may be overwritten or combined unpredictably from a consumer’s perspective. Restore task-specific output directories, clean only build/ in the disposable project, rerun, and verify one XML family per task.

6. Failure mode: parallel tests share mutable external state

Suppose two integration classes both overwrite build/tmp/shared-state.txt or bind the same fixed TCP port. maxParallelForks=2 can expose nondeterminism. The wrong repair is “retry until green.” The durable repairs are to allocate unique per-test/per-worker resources, serialize the resource-owning tests, or keep the task at one fork.

tasks.named<Test>("integrationTest") {
    // Only after the fixture is isolated.
    maxParallelForks = 2
}

Performance is causal: separate dependency resolution, compilation, test JVM startup, test body time, and report writing. More forks can increase memory/connection pressure and become slower.

7. Security-sensitive test diagnostics

Do not print CI secrets, repository credentials, access tokens, or signing material to diagnose a test. Avoid --debug as a first response when credentials may be present. Test reports and standard streams are artifacts that may be retained by CI. Use fake credentials and synthetic local endpoints for the mandatory exercises.

If a test really requires a secret, inject it from the CI secret store with least privilege, access it through a provider/environment mechanism, and ensure failures redact values. A test suite is executable code running inside your trust boundary.

8. Isolate cache/repository questions without destroying normal state

# From the disposable project:
export GRADLE_USER_HOME="$PWD/.diag-gradle-home"
./gradlew --version
./gradlew clean check --refresh-dependencies

# Compare with the ordinary isolated lab home if needed; delete only diagnostic state.
rm -rf .diag-gradle-home

If the clean diagnostic home changes dependency/framework startup behavior, you have evidence about user-home state. It does not prove that deleting a developer’s entire normal ~/.gradle is the right fix. Preserve logs and identify the specific cache/repository cause.

9. Failure interpretation matrix

10. Summary and checkpoint bridge

The safe troubleshooting pattern preserves the failed evidence, classifies the layer, and fixes the graph/classpath/report contract rather than hiding it. Lesson 5 turns that into a false-green checkpoint with controlled failure and rollback.

Knowledge check

Official references and version notes

• Gradle JVM Test Suite Plugin — current suite/source-set/task model, additional-suite dependencies, target tasks, check wiring, and outgoing test-result variants.
• JvmTestSuite DSL — current API status and framework configuration. The API remains incubating in Gradle 9.7.1.
• Testing in Java & JVM Projects — test execution, filtering, reporting, JUnit Platform, manual integration-test fallback, and troubleshooting.
• Gradle Test task — forked test JVMs, reports, filtering, maxParallelForks, failure behavior, and framework options.
• TestFilter — class/method filters and fail-on-no-match behavior.
• Gradle 9.7.1 Release Notes — pinned Gradle baseline and current test-reporting behavior.
• JUnit 6.1.3 Overview — current JUnit Platform/Jupiter composition and Java 17+ runtime baseline.
• JUnit 6.1.3 Tagging and Filtering — @Tag semantics used in the examples.

Version-sensitive behavior was rechecked against current Gradle and JUnit primary documentation on 2026-08-24. Mandatory labs use Gradle 9.7.1 through the previously verified Wrapper, JDK 21 as the Gradle runtime, Java 17 as the project target, JUnit 6.1.3, and an isolated GRADLE_USER_HOME. The JVM Test Suite API is explicitly labeled incubating. No hosted CI, database, cloud service, commercial repository, or paid test platform is required.