CI/CD Performance Gates in GitHub Actions, GitLab CI, and Jenkins: Core Concepts and Mental Model

1. Why JMeter exit alone is not a performance gate

A CLI run can complete successfully and produce a complete JTL while p95 violates an application objective. A process can also fail before useful samples exist. Therefore “JMeter returned zero” answers only whether the engine/command path completed normally; it does not answer whether the application met a latency/error objective.

2. Mental model

flowchart TD
P[Pipeline/local launcher] --> V[Java17 + JMeter5.6.3]
V --> J[JMeter CLI + JMX/properties]
J --> T[Authorized target]
J --> R[JTL + jmeter.log + dashboard]
T --> E[Target events]
R --> G[Threshold evaluator]
E --> G
S[Reviewed SLO / optional baseline] --> G
J --> X[Engine exit]
G --> Q[Gate exit]
X --> L[Launcher classification]
Q --> L
L --> C[CI job status]
R --> A[Always-retained evidence]
E --> A
G --> A

JMeter owns load execution and raw results; the target independently proves achieved work. The evaluator validates the sample set before applying a documented p95/error policy. The launcher keeps engine and gate exit codes separate and CI treats any nonzero final code as blocking. Artifacts are retained independently of status.

3. Four independent signals

4. Policy vocabulary

An absolute SLO is a fixed requirement such as p95≤120 ms and errors≤1%. A baseline is an approved reference result/environment. A regression budget is the allowed degradation from that approved reference. Baselines are governance artifacts: do not auto-update them after a failure.

5. Percentile formula is part of policy

The evaluator computes nearest-rank p95 directly from CSV JTL. The HTML dashboard remains human diagnostic evidence. This avoids scraping rendered pages and makes the exact gate algorithm auditable even when another JMeter report computes percentiles with a different estimator.

6. Exit-code contract

7. Failure evidence must survive

Retain raw JTL, matching jmeter.log, generated dashboard, gate/launcher JSON, policy/input manifest, provider console and target events. A performance failure without evidence becomes an expensive retry request instead of an actionable diagnosis.

8. Read-only inspection before policy changes

java -version
"$JMETER_HOME/bin/jmeter" -v
python3 --version
python3 tools/evaluate_gate.py --help
python3 tools/run_performance_gate.py --help
cat policy/slo.json
cat policy/baseline.json
git diff -- plans/ config/ policy/ tools/

Also inspect the provider job trigger, timeout, runner/agent class, concurrency and artifact-on-failure logic. Do not lower thresholds or add retries before proving which state changed.

9. State inventory

10. DevOps connection

Performance gates are policy code. They should be deterministic, reviewable, cheap enough for the selected cadence and able to preserve evidence on failure. GitHub Actions, GitLab CI and Jenkins should orchestrate the same portable command rather than reimplementing thresholds three times.

Knowledge check

Official references and version notes

• Apache JMeter downloads — JMeter 5.6.3 and Java 8+ requirement.
• JMeter execution guidance — GUI for authoring/debug and CLI/non-GUI for load.
• JMeter Dashboard Report — -e -o/-g -o and dashboard CSV requirements.
• GitHub Actions build/test examples — artifact upload with always() after test failure.
• GitHub setup-java — stable Java setup action; example uses v5 with Temurin 17.
• GitLab CI YAML reference — artifacts:when: always semantics.
• Jenkins Declarative Pipeline syntax — post/always, timeout, agent behavior.
• Jenkins tests and artifacts — archiveArtifacts in an always-run post block.