Checkpoint Lab — HTML Dashboard Reports and Performance Result Interpretation

1. Exact assumptions and hard ceilings

2. Setup and target authorization/preflight

1. Start one fresh fixture process and event log.
2. GET health/stats; record baseline request counters.
3. Record JMeter/Java versions and generator CPU/heap/disk/network headroom.
4. Hash/version the JMX and local.properties; confirm both runs will use the same inputs except -Jrun.id/-Jmode.
5. Confirm new/empty result/report directories.

3. Predictions before execution

P1 — workload: both runs will produce 80 JTL rows and 80 target work events (40 Catalog + 40 Checkout).

P2 — error: baseline error rate ≈0; degraded Checkout error rate=20% and total error rate=10%.

P3 — latency: Catalog p95/target service time remains roughly unchanged; degraded Checkout average/p90/p95/p99 increases materially.

P4 — APDEX: Checkout APDEX decreases under the fixed 100/300 ms lab thresholds.

P5 — throughput: degraded overall/hit throughput likely decreases because this is a closed sequential workload whose threads wait longer on Checkout.

4. Execute baseline and degraded runs

Baseline at-end report:

& "$env:JMETER_HOME\bin\jmeter.bat" `
  -n -t .\plans\dashboard-local.jmx `
  -q .\config\local.properties `
  -Jrun.id=p23-check-baseline -Jmode=baseline `
  -l .\results\p23-check-baseline\results.jtl `
  -j .\results\p23-check-baseline\jmeter.log `
  -e -o .\results\p23-check-baseline\html-report

Degraded JTL then report:

& "$env:JMETER_HOME\bin\jmeter.bat" `
  -n -t .\plans\dashboard-local.jmx `
  -q .\config\local.properties `
  -Jrun.id=p23-check-degraded -Jmode=degraded `
  -l .\results\p23-check-degraded\results.jtl `
  -j .\results\p23-check-degraded\jmeter.log

& "$env:JMETER_HOME\bin\jmeter.bat" `
  -q .\config\local.properties `
  -g .\results\p23-check-degraded\results.jtl `
  -o .\results\p23-check-degraded\html-report

5. Raw-JTL verification gate

python tools/summarize_jtl.py   results/p23-check-baseline/results.jtl   results/p23-check-degraded/results.jtl

python tools/compare_runs.py   results/p23-check-baseline/results.jtl   results/p23-check-degraded/results.jtl

Do not open with a conclusion until row counts, labels and degraded error counts match P1/P2.

6. Independent target verification gate

python tools/analyze_events.py results/server-events.jsonl p23-check-baseline
python tools/analyze_events.py results/server-events.jsonl p23-check-degraded

Require:

• 80 events each;
• 40 Catalog +40 Checkout each;
• baseline all 200;
• degraded exactly eight 503 Checkout events;
• Catalog service-wall distribution similar across modes;
• Checkout service-wall distribution near the deliberately injected mode shift.

7. Dashboard verification worksheet

8. Sample-label mapping

Attach this to the evidence packet:

Catalog  -> GET /catalog  -> control operation, unchanged target behavior
Checkout -> GET /checkout -> treatment operation, mode-controlled latency/errors
No Transaction Controller parent sample in checkpoint

This removes ambiguity when reviewing table/graph labels later.

9. Workload/report manifest

Record:

• JMX/property SHA-256 and JMeter/Java versions;
• threads=2, loops=20, two samplers/loop, 50 ms pacing;
• run IDs/modes;
• labels/control/treatment mapping;
• save-service required fields and CSV format;
• APDEX=100/300 ms, p90/p95/p99, granularity=2000 ms;
• no sample_filter/start_date/end_date for the main checkpoint reports;
• generator CPU/heap/network/disk headroom;
• result/report directories and target event log.

10. Write the interpretation with uncertainty

Observation: Checkout latency distribution and error rate worsen in degraded mode; Catalog latency remains stable. Overall completion throughput falls.

Controlled mechanism: The fixture intentionally changes only Checkout from ~45 ms/success to ~180 ms with deterministic HTTP 503s every fifth sequence. Target telemetry confirms that exact operation-level change.

Interpretation: The Checkout regression is attributable to the controlled target change. The overall throughput reduction is consistent with a closed workload spending longer blocked on Checkout; it does not prove Catalog capacity changed.

Uncertainty/limits: This is a tiny loopback experiment with 40 samples/label; extreme-tail percentiles are low-resolution, network variability is minimal, and the result says nothing about production capacity. APDEX conclusions apply only to the lab's 100/300 ms thresholds.

11. Check one reporting failure without rerunning load

Copy the baseline JTL and remove one required field as in Lesson 4. Attempt -g into a new directory, preserve the failure, then regenerate from the untouched JTL. This proves report failures can be diagnosed at the artifact/configuration layer without sending more traffic.

12. Required evidence packet

13. Final validity statement

14. Verification checklist

• Only 127.0.0.1:8023.
• JMeter 5.6.3 / Java 17 / no plugins.
• Same JMX/hash/workload/report properties except run ID/mode.
• 80 JTL +80 target events each.
• Degraded exactly eight Checkout 503s; Catalog no intentional failures.
• Dashboard statistics/error table match raw JTL counts.
• Checkout latency/APDEX move in expected direction; Catalog control remains stable.
• Throughput interpretation states the closed-loop dependency.
• Generator headroom/filter/window/controller policy documented.
• Interpretation states uncertainty/sample-size/local-only limitations.

15. Cleanup / rollback

1. Stop JMeter processes and localhost fixture after target evidence is saved.
2. Keep raw JTL, jmeter.log, dashboard dirs, report/workload properties, manifests and event evidence until review.
3. Delete only disposable local artifacts after the defined retention period; do not delete first-failure evidence to rerun.
4. No public/production target, real credential/PII, recorder CA, remote RMI, container, paid telemetry/CI, database/message service or OS/JVM global tuning was changed.

16. What Chapter 23 adds to the operating model

The production performance-testing operating model now has a dashboard interpretation contract: report source JTL/schema, labels/controller policy, percentile set, APDEX thresholds, sample/series filters, measurement window, time granularity, workload manifest, configured-versus-achieved count/rate, generator validity, target telemetry, raw-to-dashboard cross-check and uncertainty statement must accompany any performance conclusion.

Chapter 24 moves to Backend Listener, InfluxDB/Graphite, Grafana, and Real-Time Telemetry. It extends this interpretation discipline into live time-series pipelines and explicitly measures the network/backend cost that a dashboard generated after the run does not impose during sampling.

Knowledge check

Official references and version notes

• JMeter User Manual — Generating Dashboard Report — required CSV fields, APDEX, statistics/percentiles, errors, graphs, filters, date windows, granularity, -g, -e, -o, and output-folder rules.
• JMeter Getting Started — CLI/load analysis — GUI authoring/debugging, CLI load execution, CSV/XML results and HTML analysis.
• JMeter Listeners / Result files — sample-log fields and raw-result interpretation.
• Component Reference — Transaction Controller — parent/additional transaction samples and reporting semantics.
• JMeter Properties Reference — report/save-service configuration and defaults.
• Apache JMeter downloads — current stable release and Java requirement.