Checkpoint Lab — Performance Governance, Baselines, SLOs, Regression Budgets, and Reporting

1. Exact assumptions and ceilings

2. Predictions before execution

P1: all six runs will achieve20 samples and zero errors with valid generator state.

P2: Build B p50/p95 will be higher because only base service delay changes55→64 ms; absolute p95 should remain below90 ms.

P3: the relative p95 change should exceed the12% budget, so the technical gate should FAIL even though the absolute SLO passes.

P4: an approved scoped seven-day exception should produce release disposition ALLOW_WITH_EXCEPTION while gate.json remains FAIL.

3. Preflight/freeze the governance inputs

& "$env:JMETER_HOME\bin\jmeter.bat" -v
java -version

Get-FileHash `
  .\plans\governed-workload.jmx, `
  .\config\governance.properties, `
  .\policy\performance-policy.json `
  -Algorithm SHA256

Get-Content .\policy\performance-policy.json

Record JMeter/Java, hashes, workload version, environment ID and service authorization before starting either build.

4. Build A: repeat, analyze, aggregate, baseline

1. Start build-1.0.0 fixture with base55/jitter5/max70.
2. Run a1/a2/a3 with identical JMX/properties; separate JTL/log/dashboard directories.
3. Analyze each: configured20, achieved20, generator-valid true.
4. Aggregate medians and preserve all run values.
5. Create baseline-v1.json with owner/reason/policy/source hashes.
6. Stop Build A after60 measured requests.

5. Build B: repeat under identical protocol

1. Start build-1.1.0-demo with base64; keep jitter/error/ceiling/port identical.
2. Run b1/b2/b3 with the same workload/environment/analysis formula.
3. Analyze and aggregate; preserve target JSONL/JTL/jmeter.log/dashboard/generator notes.
4. Verify60 measured target requests and no rejections/errors.

6. Technical gate

Run evaluate_gate.py. Require a valid comparison. Expected pattern:

If the actual deterministic timing differs slightly, trust the generated gate output rather than forcing this table. The lab's purpose is transparent policy evaluation, not manufacturing a specific percentage.

7. Simulate one justified exception

If and only if relative_p95_regression_pct actually failed, create P34-EXC-001 with owner, approver, PERF-134, rationale and7-day expiry. If another metric also failed, either include it with explicit rationale or BLOCK; a partial exception cannot cover unapproved failures.

Record why this demonstration exception exists: the team accepts a known short-lived local regression while remediation is tracked, but does not promote the slower build into the baseline automatically.

8. Release decision remains two-layered

Run the decision tool. The packet should preserve:

technical_gate_status: FAIL
release_disposition: ALLOW_WITH_EXCEPTION
exception_id: P34-EXC-001
reason: approved_unexpired_scoped_exception

After expiry, the same gate/exception pair evaluates to BLOCK. A future baseline promotion requires a separate reviewed baseline-v2 event.

9. Append trend and build stakeholder report

Run build_report.py. The trend row must include build, baseline version, policy/workload/environment IDs, median metrics, technical gate, release disposition, exception ID and report hash. The Markdown report must state limitations and round values without losing raw JSON/JTL authority.

10. Prove the decision remains auditable later

Close/reopen the packet without rerunning JMeter. Verify:

• baseline-v1 still identifies build-1.0.0 and source/policy hashes;
• gate.json still says FAIL and names the exact failed metric(s);
• exception.json names build/baseline/owner/approver/issue/scope/expiry;
• decision.json explains ALLOW_WITH_EXCEPTION;
• trend history contains both the technical and release outcomes;
• report.json hashes match the preserved inputs;
• raw JTL/jmeter.log directories still support reanalysis during retention.

11. Negative governance test: make comparison INVALID

Copy the current aggregate and change its environment ID to different-runner (do not alter the original). Evaluate this copy. Require status INVALID/environment_mismatch. Attempting to create a performance exception for this INVALID result must be rejected. This proves the gate distinguishes regression from comparability failure.

12. Verification checklist

• JMeter5.6.3/Java17/no plugin.
• Only127.0.0.1:8034.
• 20 configured=20 achieved each run;3 repetitions/build.
• Same workload/environment/formula/result schema.
• Only target base_ms55→64 changes.
• Baseline immutable/versioned with owner/reason/hashes.
• Technical gate output preserved independently of exception.
• Exception scope/owner/approver/issue/expiry valid.
• Trend/report contain gate + release disposition + limitations.
• Invalid environment mismatch is BLOCK_INVALID_RUN, not exception-eligible.

13. Cleanup / rollback

1. Stop fixture and confirm port8034 closes.
2. Keep governance packet/trend/raw evidence for the lab's retention window.
3. Rollback policy means continue using baseline-v1/policy-v1; do not delete current FAIL/exception history.
4. After raw-retention expiry, keep compact trend/baseline/policy/decision history according to policy.
5. No production/shared target, credential, remote engine, cloud resource or system-wide setting was changed.

14. Production operating-model addition and Chapter35 bridge

Chapter34 adds a performance-governance contract: service objectives, approved workload/environment identity, immutable versioned baselines, transparent repeated-run reducers, absolute and relative regression budgets, fail-closed validity checks, immutable technical gate outputs, scoped expiring exceptions, trend history, stakeholder reporting, ownership/change control and raw-evidence retention are required for durable release policy.

Chapter35 is the capstone: Design and Operate a Production Performance-Testing Program. It combines workload modeling, correlation/data, CLI/distributed execution, telemetry, diagnosis, validity, safety, extensions, troubleshooting and governance into one end-to-end operating program.

Knowledge check

Official references and version notes

• Apache JMeter downloads — current stable JMeter 5.6.3 and Java 8+ requirement.
• JMeter current changes — Java 17+ recommendation for the 5.6.x line.
• JMeter Getting Started — non-GUI/CLI execution and result/log flags.
• JMeter Dashboard Report — CSV requirements, p90/p95/p99 defaults, report generation and percentile-estimator caveat.
• JMeter Properties Reference — result-save fields, aggregate percentile properties and reporting properties.
• JMeter Remote Testing — same-plan fan-out, exact JMeter parity, Java/data requirements and controller overhead.