Checkpoint Lab — Assertions and Functional Correctness Under Load

1. Assumptions and safety limits

2. Start a clean fixture

mkdir -p results/checkpoint-classify results/overhead-baseline results/overhead-asserted
python fixtures/assertion_fixture.py --log results/checkpoint-classify/server-events.jsonl
curl --fail --silent http://127.0.0.1:8000/health

Restart the fixture before overhead runs if you want isolated event logs. The server responses are deterministic enough for the course, but workstation scheduling noise still affects micro-benchmarks.

3. Build the classification plan

Test Plan — Chapter 08 Checkpoint
└── Thread Group — 1 user × 5 loops
    ├── HTTP Request Defaults — HttpClient4 / 127.0.0.1:8000
    ├── Success — GET /success
    │   ├── Response Assertion — Response Code Equals 200
    │   └── JSON JMESPath Assertion — business Equals accepted
    ├── Protocol Error — GET /protocol-error
    │   └── Response Assertion — Response Code Equals 200
    └── Business Error — GET /business-error
        ├── Response Assertion — Response Code Equals 200
        └── JSON JMESPath Assertion — business Equals accepted

Keep Ignore Status off everywhere. The Protocol Error sampler must remain failed because 503 is not an expected normal success.

4. Predict classification before execution

Prediction 1: HTTP code alone distinguishes Protocol Error but not Business Error. Prediction 2: the dashboard should show two failed sampler populations with different failure reasons.

5. One bounded GUI verification

Run one loop with View Results Tree and Assertion Results only for authoring. Verify:

• Success has no assertion failures;
• Protocol Error remains failed with HTTP 503;
• Business Error remains HTTP 200 but has an assertion failure;
• failure messages name the violated invariant rather than dumping the full response.

Disable both listeners before the five-loop checkpoint run.

6. Run classification from CLI and generate dashboard

jmeter -n   -t plans/checkpoint-classify.jmx   -l results/checkpoint-classify/results.jtl   -j results/checkpoint-classify/jmeter.log   -e -o results/checkpoint-classify/report   -Jjmeter.save.saveservice.print_field_names=true   -Jjmeter.save.saveservice.successful=true   -Jjmeter.save.saveservice.response_code=true   -Jjmeter.save.saveservice.response_message=true   -Jjmeter.save.saveservice.thread_counts=true   -Jjmeter.save.saveservice.assertion_results_failure_message=true
python tools/analyze_assertions.py results/checkpoint-classify/results.jtl
curl --fail --silent http://127.0.0.1:8000/stats

PowerShell uses jmeter.bat with the same settings. The report directory must not already contain a previous report.

7. Expected classification evidence

With 5 loops:

• Success: 5 successful samples;
• Protocol Error: 5 failed samples with response code 503;
• Business Error: 5 failed samples with response code 200 plus JMESPath/business assertion message;
• overall: 15 samples, 10 failures, ~66.7% error rate by design.

This deliberately high error percentage is not a service-quality claim—it is a synthetic classification lab with two failure variants intentionally executed every loop.

8. Verify dashboard error table

In report/index.html, confirm:

• request summary reflects 5 successful / 10 failed samples;
• Error table contains the synthetic failure reasons;
• Top 5 Errors by Sampler separates Protocol Error and Business Error labels;
• Statistics table keeps label-level counts/timing visible.

Cross-check raw JTL before trusting dashboard grouping.

9. Prepare a clean overhead comparison

Create two separate Success-only plans:

1. overhead-baseline.jmx: 2 threads × 25 loops, GET /success, no assertions.
2. overhead-asserted.jmx: identical workload plus code=200 and business=accepted JMESPath assertions.

No debug listener, no dashboard generation during the timed command itself if you want to isolate test-engine overhead. Generate dashboards afterward from the JTL when needed.

10. Run and measure assertion overhead

Bash example:

/usr/bin/time -p jmeter -n   -t plans/overhead-baseline.jmx   -l results/overhead-baseline/results.jtl   -j results/overhead-baseline/jmeter.log

/usr/bin/time -p jmeter -n   -t plans/overhead-asserted.jmx   -l results/overhead-asserted/results.jtl   -j results/overhead-asserted/jmeter.log

python tools/compare_runs.py   results/overhead-baseline/results.jtl   results/overhead-asserted/results.jtl

PowerShell:

Measure-Command {
  jmeter.bat -n `
    -t plans\overhead-baseline.jmx `
    -l results\overhead-baseline\results.jtl `
    -j results\overhead-baseline\jmeter.log
}

Measure-Command {
  jmeter.bat -n `
    -t plans\overhead-asserted.jmx `
    -l results\overhead-asserted\results.jtl `
    -j results\overhead-asserted\jmeter.log
}

python tools\compare_runs.py `
  results\overhead-baseline\results.jtl `
  results\overhead-asserted\results.jtl

Record Task Manager/top CPU/memory during both runs. If the small narrow-assertion overhead is below noise, state that rather than exaggerating. Repeat a few times only if needed, keeping the same 50-request ceiling per run.

11. Interpret overhead correctly

Minimal assertions may leave sampler p50/p95 almost unchanged because the target service interval is the same. Assertion work happens after the sampler and can instead affect thread availability, achieved throughput, total engine wall time, and generator CPU.

A measurable slowdown in the asserted plan does not prove the target got slower unless server event timing or sampler elapsed also changed consistently.

12. Required evidence packet

13. Verification checklist

• Target is exactly loopback:8000.
• Classification plan produces 15 samples maximum.
• Success is green; Protocol Error is failed 503; Business Error is failed HTTP 200.
• Ignore Status is off in normal success/failure classification.
• JTL contains assertion failure messages.
• Dashboard counts match raw JTL.
• Assertion Results/View Results Tree are absent from load copies.
• Baseline and asserted overhead plans differ only by minimal assertions.
• Generator CPU/memory is recorded before making overhead claims.

14. Validity statement

15. Cleanup

1. Stop the local fixture.
2. Keep classification and overhead evidence until review is complete.
3. Remove debug listeners from later load profiles.
4. Do not carry synthetic failure percentages into production reporting.
5. No production target, real credential, recorder certificate, remote engine, database, container, plugin, or system-wide JVM/OS setting was changed.

16. What Chapter 08 adds to the operating model

The performance-testing operating model now has a correctness contract: each critical sampler defines transport/business invariants, assertion scope, expected failure classification, failure-message retention, dashboard grouping, and acceptable injector overhead.

Chapter 09 naturally extends this by extracting dynamic response data and correlating requests so correctness assertions can validate realistic session-dependent values rather than only static synthetic fields.

Knowledge check

Official references and version notes

• Component Reference — current Response, Duration, Size, JSON, JSON JMESPath, JSR223, and other assertion semantics.
• Elements of a Test Plan — assertion scope and execution order after the sampler/Post-Processors and before listeners.
• Generating Dashboard Report — failed-request summary, error table, Top 5 Errors by Sampler, and required CSV fields.
• Properties Reference — result-save configuration, including assertion failure messages.
• Best Practices — CLI execution, lean listeners, and generator-validity guidance.
• Apache JMeter downloads — current stable release and Java requirement.