CI/CD Performance Gates in GitHub Actions, GitLab CI, and Jenkins: Diagnostics, Failure Modes, and Production Practices

1. Preserve first-failure evidence

2. Diagnostic sequence

flowchart TD
E[Preserve JTL/log/gate/provider/target evidence] --> V[Confirm JMeter/Java/plugin/evaluator/provider versions]
V --> C[Confirm JMX/data/properties/CLI + authorized target]
C --> S[Validate tree scope/resolved properties/configured load/policy]
S --> P[Inspect protocol/session/data/assertion state]
P --> G[Inspect runner JVM/CPU/network/disk/container state]
G --> T[Inspect target telemetry/environment noise]
T --> CI[Inspect timeout/concurrency/artifacts/secrets/exit propagation]
CI --> F[Least-destructive correction]
F --> R[Small controlled rerun]

3. Intentionally broken: JMeter exit code treated as SLO

jmeter -n -t plans/ci-gate.jmx -l results.jtl -e -o dashboard
if [ $? -eq 0 ]; then
  echo "PERFORMANCE PASSED"
fi

Against degraded mode this can print PASS even though valid p95 is ~180 ms. Repair by preserving evidence and applying the common evaluator. Do not bury CI policy inside JSR223 or provider-specific JMX branches.

4. Flaky shared environment

If identical code alternates pass/fail while target telemetry shows unrelated deployments/noisy neighbors, retry-until-green samples the noise. Isolate/reserve the target, move the suite to scheduled/informational, or keep PR blocking on the deterministic fixture.

5. Huge load on every commit

A long, high-thread capacity test on each push burns runner capacity and repeatedly stresses shared systems. Cadence is a safety boundary. Keep PR smoke small; schedule/load-gate larger suites under explicit authorization and concurrency controls.

6. Success-only artifacts

Broken GitHub: upload step lacks always(). Broken GitLab: default success-only artifacts. Broken Jenkins: archive occurs only inside a successful stage. Repair with each provider's unconditional/finally mechanism and deliberately test a failing gate.

7. Baseline drift

# anti-pattern
if gate_failed:
    baseline["p95_ms"] = current_p95
    save_and_commit(baseline)

The next run passes because policy moved. Baseline updates require a reviewed reference/environment and preserved old/new evidence.

8. Secrets in CI logs

Do not put real tokens/passwords in echoed CLI -J values, JMX or artifacts. Use provider secret stores/least-privilege contexts and avoid printing values. If a real secret leaks, rotate/revoke it. This lab uses no real credentials.

9. Provider-specific logic inside JMX

A JSR223 sampler checking GITHUB_ACTIONS or constructing GitLab/Jenkins artifact paths couples orchestration to load generation. Keep JMX neutral; providers only invoke the common launcher and store its output.

10. Runner saturation

A smaller/throttled CI runner can inflate latency or cap throughput while the SUT is unchanged. Chapter 26 still applies: inspect runner CPU/GC/disk/network and classify the experiment invalid if generator constraints prevent the intended workload.

11. Causal table

12. Shortcuts to reject

• Do not add blanket retries or arbitrary long sleeps.
• Do not add giant heaps without runner/JVM evidence.
• Do not mass-disable evidence/listeners without evidence of their cost.
• Do not use global property hacks to fake success.
• Do not disable TLS/RMI verification.
• Do not reproduce on production/public targets.
• Do not increase workload while validity is unresolved.
• Do not delete failed JTL/log/dashboard/gate/provider evidence.

Knowledge check

Official references and version notes

• Apache JMeter downloads — JMeter 5.6.3 and Java 8+ requirement.
• JMeter execution guidance — GUI for authoring/debug and CLI/non-GUI for load.
• JMeter Dashboard Report — -e -o/-g -o and dashboard CSV requirements.
• GitHub Actions build/test examples — artifact upload with always() after test failure.
• GitHub setup-java — stable Java setup action; example uses v5 with Temurin 17.
• GitLab CI YAML reference — artifacts:when: always semantics.
• Jenkins Declarative Pipeline syntax — post/always, timeout, agent behavior.
• Jenkins tests and artifacts — archiveArtifacts in an always-run post block.