Maven Testing with Surefire, Failsafe, Integration Tests, Reports, and Quality Gates: Guided Hands-On Workflow and Core Operations

1. Scenario and acceptance criteria

You will build testing-lab. CalculatorTest is a unit test selected by Surefire; HealthServiceIT is an integration test selected by Failsafe. Both live under the conventional src/test/java source tree, but their names and plugin bindings route them to different lifecycle stages.

2. Disposable setup and preflight

Start in a throwaway directory and keep Maven dependency/plugin downloads separate from your normal local repository. The first build may require network access to Maven Central; later runs can demonstrate warm-cache behavior from the isolated lab repository.

mkdir -p testing-lab/src/main/java/dev/academy/testing
mkdir -p testing-lab/src/test/java/dev/academy/testing
mkdir -p testing-lab/evidence
cd testing-lab
export LAB_REPO="$PWD/.lab-m2/repository"
./mvnw -v

3. Author the explicit test model

The POM pins the build plugins, imports JUnit 6.1.3 through its BOM, and activates both Failsafe goals in one named execution. failIfNoTests is enabled in the teaching lab so accidentally empty suites are visible rather than silently green.

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>dev.academy.testing</groupId>
  <artifactId>testing-lab</artifactId>
  <version>1.0.0</version>
  <properties>
    <maven.compiler.release>17</maven.compiler.release>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <junit.version>6.1.3</junit.version>
  </properties>

  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>org.junit</groupId>
        <artifactId>junit-bom</artifactId>
        <version>${junit.version}</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>

  <dependencies>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter</artifactId>
      <scope>test</scope>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.15.0</version>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>3.5.6</version>
        <configuration>
          <failIfNoTests>true</failIfNoTests>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-failsafe-plugin</artifactId>
        <version>3.5.6</version>
        <configuration>
          <failIfNoTests>true</failIfNoTests>
        </configuration>
        <executions>
          <execution>
            <id>integration-tests</id>
            <goals>
              <goal>integration-test</goal>
              <goal>verify</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

4. Add production and test code

The examples are intentionally small so the chapter remains about Maven evidence rather than test-design theory. The class names are the routing signal.

package dev.academy.testing;

public final class Calculator {
    public int add(int left, int right) {
        return left + right;
    }
}

package dev.academy.testing;

public final class HealthService {
    public String status() {
        return "UP";
    }
}

package dev.academy.testing;

import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.Test;

class CalculatorTest {
    @Test
    void addsTwoNumbers() {
        assertEquals(5, new Calculator().add(2, 3));
    }
}

package dev.academy.testing;

import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.Test;

class HealthServiceIT {
    @Test
    void serviceReportsUp() {
        assertEquals("UP", new HealthService().status());
    }
}

5. Run the unit stage and inspect evidence

test compiles test code and invokes Surefire. Because HealthServiceIT follows Failsafe naming, it should not appear in the Surefire report set.

set -o pipefail
./mvnw -Dmaven.repo.local="$LAB_REPO" clean test | tee evidence/unit.log
find target/surefire-reports -maxdepth 1 -type f -print | sort | tee evidence/surefire-files.txt
grep -R "tests=\|failures=\|errors=" target/surefire-reports/TEST-*.xml | tee evidence/unit-counts.txt

6. Compare package with verify

A common CI mistake is to treat packaging as the complete test gate. Run package, inspect the artifact, then run verify. The difference is lifecycle position, not source code.

./mvnw -Dmaven.repo.local="$LAB_REPO" clean package | tee evidence/package.log
ls -l target/*.jar
find target/failsafe-reports -maxdepth 1 -type f -print 2>/dev/null || true

./mvnw -Dmaven.repo.local="$LAB_REPO" verify | tee evidence/verify.log
find target/failsafe-reports -maxdepth 1 -type f -print | sort | tee evidence/failsafe-files.txt
grep -R "tests=\|failures=\|errors=" target/failsafe-reports/TEST-*.xml | tee evidence/it-counts.txt

7. Filter the correct runner deliberately

Use -Dtest for Surefire and -Dit.test for Failsafe. Filtering is a developer feedback tool; CI should still have a clearly defined full-suite job so a narrow filter cannot become accidental policy.

./mvnw -Dmaven.repo.local="$LAB_REPO" -Dtest=CalculatorTest test
./mvnw -Dmaven.repo.local="$LAB_REPO" -Dit.test=HealthServiceIT verify

8. Engineer a Surefire failure

Temporarily change the unit assertion from 5 to 99. Preserve the original test first. The Maven process should fail in the test stage, and the XML/text report should explain the assertion failure.

cp src/test/java/dev/academy/testing/CalculatorTest.java evidence/CalculatorTest.good.java
sed -i.bak 's/assertEquals(5,/assertEquals(99,/' src/test/java/dev/academy/testing/CalculatorTest.java
set +e
./mvnw -Dmaven.repo.local="$LAB_REPO" test > evidence/unit-failure.log 2>&1
status=$?
set -e
printf 'unit failure exit=%s
' "$status" | tee evidence/unit-failure-exit.txt
cp evidence/CalculatorTest.good.java src/test/java/dev/academy/testing/CalculatorTest.java

9. Engineer a Failsafe verify failure

Now preserve and replace the integration test with a deliberately wrong expectation. During integration-test, Failsafe records the failure so lifecycle cleanup can still proceed. At verify, the recorded failure becomes authoritative and Maven exits nonzero.

package dev.academy.testing;

import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.Test;

class HealthServiceIT {
    @Test
    void serviceReportsUp() {
        assertEquals("DOWN", new HealthService().status()); // intentional failure
    }
}

cp src/test/java/dev/academy/testing/HealthServiceIT.java evidence/HealthServiceIT.good.java
# replace the file with the intentional failing version shown above
set +e
./mvnw -Dmaven.repo.local="$LAB_REPO" verify > evidence/verify-failure.log 2>&1
status=$?
set -e
printf 'verify failure exit=%s
' "$status" | tee evidence/verify-failure-exit.txt
cp evidence/HealthServiceIT.good.java src/test/java/dev/academy/testing/HealthServiceIT.java

10. Challenge: choose the correct control

Your team wants a fast local command that runs only CalculatorTest, but CI must still run every unit and integration test. Which control belongs in a developer command, and which belongs in the committed POM/CI policy? Explain why permanently narrowing <includes> in the POM would change the build contract for everyone.

11. Verification and cleanup

Verify that both good tests pass under verify, both report directories exist, and no failure-ignore or skip property is left in the POM. Then remove only the disposable lab directory or its project-local .lab-m2. Never delete the learner's normal Maven cache for this exercise.

./mvnw -Dmaven.repo.local="$LAB_REPO" clean verify | tee evidence/final-verify.log
find target/surefire-reports target/failsafe-reports -type f -maxdepth 1 -print | sort
sha256sum target/*.jar | tee evidence/artifact.sha256

Knowledge check

12. Summary and bridge

You now have a causal workflow from naming and plugin configuration to report files and exit status. Lesson 3 turns those mechanics into design decisions: how much to separate, what to encode as naming or includes, and when retries/ignores weaken the quality signal.

Official references and version notes

Version-sensitive statements were checked against Apache Maven and JUnit primary documentation on 2026-08-23. The mandatory path pins Maven 3.9.16 via Maven Wrapper 3.3.4, JDK 21 to run Maven, Java 17 as the release target, JUnit 6.1.3, Maven Surefire Plugin 3.5.6, Maven Failsafe Plugin 3.5.6, Help Plugin 3.5.2, and Compiler Plugin 3.15.0.

Apache's current Surefire site advertises 3.6.0-M1. Because the version itself is a milestone identifier, these lessons treat it as version-sensitive preview/milestone material and keep the hands-on path on the final 3.5.6 line. Re-check before adopting a newer line in production.

The examples keep unit and integration classes in the conventional test source set so the distinction remains Maven selection/lifecycle policy rather than a custom source-layout lesson.

• Maven Surefire Plugin — Introduction
• Maven Failsafe Plugin — Introduction
• Maven Failsafe — Integration Test Goal
• Maven Failsafe — Verify Goal
• Maven Surefire — Skipping Tests
• Maven Failsafe — Skipping Tests
• Maven Failsafe — Running a Single Test
• JUnit 6.1.3 User Guide — Maven support
• JUnit 6.1.3 Release Notes
• Maven Help Plugin 3.5.2
• Maven Compiler Plugin 3.15.0
• Apache Maven Wrapper
• Maven 3.9.16 Release Notes