Capstone: Design and Operate a Production Performance-Testing Program: Guided Hands-On Workflow

1. Complete project tree

p35-capstone/
├── charter/authorization.json
├── lock/versions.lock.json
├── plans/capstone.jmx
├── config/capstone.properties
├── data/users.csv
├── fixtures/capstone_service.py
├── tools/
│   ├── run_capstone.ps1
│   ├── analyze_capstone.py
│   ├── gate_capstone.py
│   └── fleet_preflight.py
├── fleet/
│   ├── engine-a/{manifest.json,data/users.csv}
│   └── engine-b/{manifest.json,data/users.csv}
├── governance/policy.json
├── runbook/RUNBOOK.md
├── baselines/
├── runs/
├── archive/
└── optional-container/Dockerfile

2. Freeze authorization and version assumptions

charter/authorization.json:

{
  "schema_version": 1,
  "charter_id": "P35-LOCAL-CAPSTONE",
  "owner": "learner",
  "purpose": "controlled capstone performance-engineering lab",
  "authorized_target": {
    "scheme": "http",
    "host": "127.0.0.1",
    "port": 8035,
    "load_endpoints": [
      "POST /session",
      "GET /catalog",
      "POST /cart",
      "POST /checkout"
    ],
    "inspection_endpoints": [
      "GET /health",
      "GET /stats"
    ]
  },
  "workload": {
    "profile_id": "p35-local-v1",
    "threads": 2,
    "loops": 5,
    "journeys": 10,
    "requests_per_journey": 4,
    "configured_http_samples": 40,
    "pacing_ms_before_each_sampler": 75,
    "maximum_duration_seconds": 30
  },
  "target_guard": {
    "maximum_requests": 50
  },
  "data": {
    "synthetic_only": true,
    "required_rows": 10,
    "unique_rows_across_threads": true
  },
  "forbidden": [
    "public or production targets",
    "real credentials",
    "unbounded thread/rate increases",
    "TLS/RMI verification disablement",
    "automatic retries added to hide failures"
  ]
}

lock/versions.lock.json:

{
  "schema_version": 1,
  "program_id": "p35-capstone-v1",
  "jmeter": "5.6.3",
  "java_major": 17,
  "http_implementation": "HttpClient4",
  "third_party_plugins": [],
  "workload_profile": "p35-local-v1",
  "data_version": "p35-users-v1",
  "analysis_version": "p35-analysis-v1",
  "gate_policy_version": "p35-policy-v1",
  "telemetry_schema": "p35-target-events-v1",
  "mandatory_container": false,
  "mandatory_remote_rmi": false,
  "notes": [
    "Resolve and record JMX/config/data SHA-256 before execution.",
    "Real distributed execution must reproduce this lock on every engine."
  ]
}

Before each governed run, add actual SHA-256 for JMX/config/data/charter to the run directory. The lock declares software/protocol assumptions; hashes identify the concrete files.

3. Synthetic parameterization data

data/users.csv:

USER_ID,SKU,QTY
user-001,SKU-A,1
user-002,SKU-B,2
user-003,SKU-C,1
user-004,SKU-D,3
user-005,SKU-E,1
user-006,SKU-F,2
user-007,SKU-G,1
user-008,SKU-H,2
user-009,SKU-I,1
user-010,SKU-J,3

CSV Data Set Config uses Recycle=false, Stop thread on EOF=true, and Sharing mode=All threads. With exactly ten rows and ten journeys, each local journey receives one unique user. A short file does not silently recycle and create shared state.

4. Multi-endpoint local service

fixtures/capstone_service.py:

from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from urllib.parse import urlparse, parse_qs
import argparse, hashlib, json, re, threading, time

FIXTURE_VERSION = "prompt35-capstone-service-v1"
EVENT_SCHEMA = "p35-target-events-v1"
SAFE = re.compile(r"^[A-Za-z0-9_.:-]{1,96}$")
lock = threading.Lock()
event_log = None
build_id = "unset"
checkout_ms = 35
max_requests = 50
state = {
    "requests": 0,
    "rejections": 0,
    "sessions": 0,
    "catalog": 0,
    "cart": 0,
    "checkout": 0,
    "errors": 0,
    "service_ms_sum": {"session": 0, "catalog": 0, "cart": 0, "checkout": 0}
}
sessions = {}
carts = {}

def fingerprint(value):
    return hashlib.sha256(value.encode()).hexdigest()[:12]

def token_for(user_id, request_no):
    return "p35-" + hashlib.sha256(f"{build_id}|{user_id}|{request_no}".encode()).hexdigest()[:20]

def item_for(sku):
    return "item-" + hashlib.sha256(sku.encode()).hexdigest()[:10]

def snapshot():
    with lock:
        return {
            "fixture_version": FIXTURE_VERSION,
            "event_schema": EVENT_SCHEMA,
            "build_id": build_id,
            "checkout_ms": checkout_ms,
            "max_requests": max_requests,
            **state
        }

def write_event(event):
    event["schema"] = EVENT_SCHEMA
    with lock:
        with event_log.open("a", encoding="utf-8") as h:
            h.write(json.dumps(event, sort_keys=True) + "\n")

class Handler(BaseHTTPRequestHandler):
    protocol_version = "HTTP/1.1"

    def send_json(self, status, payload, extra_headers=None):
        raw = json.dumps(payload, sort_keys=True).encode()
        self.send_response(status)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(raw)))
        self.send_header("X-Fixture-Version", FIXTURE_VERSION)
        self.send_header("X-Build-Id", build_id)
        for k, v in (extra_headers or {}).items():
            self.send_header(k, v)
        self.end_headers()
        self.wfile.write(raw)

    def read_json(self):
        length = int(self.headers.get("Content-Length", "0"))
        raw = self.rfile.read(length) if length else b"{}"
        try:
            return json.loads(raw.decode())
        except Exception:
            return None

    def guarded_request_no(self):
        with lock:
            state["requests"] += 1
            n = state["requests"]
            if n > max_requests:
                state["rejections"] += 1
                return None
            return n

    def auth(self):
        auth = self.headers.get("Authorization", "")
        cookie = self.headers.get("Cookie", "")
        if not auth.startswith("Bearer "):
            return None
        token = auth[7:]
        if not SAFE.fullmatch(token):
            return None
        if f"p35_session={token}" not in cookie:
            return None
        with lock:
            return sessions.get(token)

    def delay(self, endpoint, ms):
        time.sleep(ms / 1000.0)
        with lock:
            state["service_ms_sum"][endpoint] += ms

    def log_work(self, endpoint, status, started, token=None, **fields):
        write_event({
            "ts_ms": int(time.time() * 1000),
            "endpoint": endpoint,
            "status": status,
            "build_id": build_id,
            "service_wall_ms": int((time.perf_counter() - started) * 1000),
            "token_fingerprint": fingerprint(token) if token else None,
            **fields
        })

    def do_GET(self):
        parsed = urlparse(self.path)
        if parsed.path == "/health":
            self.send_json(200, {"status": "ok", "state": snapshot()})
            return
        if parsed.path == "/stats":
            self.send_json(200, {"status": "ok", "state": snapshot()})
            return
        if parsed.path != "/catalog":
            self.send_json(404, {"status": "not_found"})
            return
        n = self.guarded_request_no()
        if n is None:
            self.send_json(429, {"status": "sample_ceiling"}); return
        started = time.perf_counter()
        user_id = self.auth()
        if not user_id:
            with lock: state["errors"] += 1
            self.send_json(401, {"status": "unauthorized"})
            self.log_work("catalog", 401, started); return
        sku = parse_qs(parsed.query).get("sku", [""])[0]
        if not SAFE.fullmatch(sku):
            with lock: state["errors"] += 1
            self.send_json(400, {"status": "invalid_sku"})
            self.log_work("catalog", 400, started); return
        self.delay("catalog", 12)
        with lock: state["catalog"] += 1
        item_id = item_for(sku)
        self.send_json(200, {"status": "ok", "item_id": item_id, "sku": sku})
        token = self.headers["Authorization"][7:]
        self.log_work("catalog", 200, started, token, user_id=user_id, sku=sku, item_id=item_id)

    def do_POST(self):
        parsed = urlparse(self.path)
        if parsed.path not in ("/session", "/cart", "/checkout"):
            self.send_json(404, {"status": "not_found"}); return
        n = self.guarded_request_no()
        if n is None:
            self.send_json(429, {"status": "sample_ceiling"}); return
        started = time.perf_counter()
        body = self.read_json()
        if body is None:
            with lock: state["errors"] += 1
            self.send_json(400, {"status": "invalid_json"}); return

        if parsed.path == "/session":
            user_id = str(body.get("user_id", ""))
            if not SAFE.fullmatch(user_id):
                with lock: state["errors"] += 1
                self.send_json(400, {"status": "invalid_user"}); return
            self.delay("session", 10)
            token = token_for(user_id, n)
            with lock:
                sessions[token] = user_id
                carts[token] = []
                state["sessions"] += 1
            self.send_json(200, {"status": "ok", "session_token": token},
                           {"Set-Cookie": f"p35_session={token}; Path=/; HttpOnly; SameSite=Strict"})
            self.log_work("session", 200, started, token, user_id=user_id)
            return

        user_id = self.auth()
        if not user_id:
            with lock: state["errors"] += 1
            self.send_json(401, {"status": "unauthorized"})
            self.log_work(parsed.path[1:], 401, started); return
        token = self.headers["Authorization"][7:]

        if parsed.path == "/cart":
            item_id = str(body.get("item_id", ""))
            qty = body.get("qty")
            if not SAFE.fullmatch(item_id) or not isinstance(qty, int) or not 1 <= qty <= 3:
                with lock: state["errors"] += 1
                self.send_json(400, {"status": "invalid_cart"})
                self.log_work("cart", 400, started, token); return
            self.delay("cart", 15)
            with lock:
                carts[token].append({"item_id": item_id, "qty": qty})
                state["cart"] += 1
            self.send_json(200, {"status": "ok", "cart_size": len(carts[token])})
            self.log_work("cart", 200, started, token, user_id=user_id, item_id=item_id, qty=qty)
            return

        if not carts.get(token):
            with lock: state["errors"] += 1
            self.send_json(409, {"status": "empty_cart"})
            self.log_work("checkout", 409, started, token); return
        self.delay("checkout", checkout_ms)
        order_id = "order-" + hashlib.sha256(f"{token}|{n}".encode()).hexdigest()[:12]
        with lock: state["checkout"] += 1
        self.send_json(200, {"status": "ok", "order_id": order_id, "build_id": build_id})
        self.log_work("checkout", 200, started, token, user_id=user_id, order_id=order_id)

    def log_message(self, format, *args):
        return

def main():
    p = argparse.ArgumentParser()
    p.add_argument("--host", default="127.0.0.1")
    p.add_argument("--port", type=int, default=8035)
    p.add_argument("--build-id", required=True)
    p.add_argument("--checkout-ms", type=int, required=True)
    p.add_argument("--max-requests", type=int, default=50)
    p.add_argument("--log", required=True)
    args = p.parse_args()
    if args.host != "127.0.0.1":
        raise SystemExit("fixture may bind only to loopback")
    if not 5 <= args.checkout_ms <= 150:
        raise SystemExit("checkout-ms must be 5..150")
    if not 1 <= args.max_requests <= 50:
        raise SystemExit("max-requests must be 1..50")
    global event_log, build_id, checkout_ms, max_requests
    build_id, checkout_ms, max_requests = args.build_id, args.checkout_ms, args.max_requests
    event_log = Path(args.log).resolve()
    event_log.parent.mkdir(parents=True, exist_ok=True)
    event_log.write_text("", encoding="utf-8")
    print(json.dumps(snapshot(), sort_keys=True), flush=True)
    ThreadingHTTPServer((args.host, args.port), Handler).serve_forever()

if __name__ == "__main__":
    main()

The target keeps only synthetic in-memory state. It never logs the bearer token itself—only a SHA-256 fingerprint. The cookie/token pair must agree, which makes Cookie Manager and correlation observable. Session/catalog/cart delays are fixed at10/12/15ms; checkout delay is a build parameter used later for a controlled regression.

5. Author the modular JMeter tree

Test Plan — P35 Capstone
├── User Defined Variables
│   workload.version = p35-local-v1
├── HTTP Request Defaults
│   server = ${__P(target.host,127.0.0.1)}
│   port = ${__P(target.port,8035)}
│   implementation = HttpClient4
│   connect timeout = ${__P(connect.timeout.ms,500)}
│   response timeout = ${__P(response.timeout.ms,2000)}
└── Thread Group — Local governed journey
    threads = ${__P(threads,2)}
    loops = ${__P(loops,5)}
    Action after Sampler error = Stop Thread
    ├── CSV Data Set Config — synthetic users
    │   Filename = ${__P(data.file,data/users.csv)}
    │   Variable Names = USER_ID,SKU,QTY
    │   Recycle on EOF = false
    │   Stop thread on EOF = true
    │   Sharing mode = All threads
    ├── HTTP Cookie Manager
    │   Clear cookies each iteration = true
    ├── Constant Timer
    │   Delay = ${__P(pacing.ms,75)} ms
    └── Simple Controller — User Journey
        ├── HTTP Request — Session
        │   POST /session
        │   body = {"user_id":"${USER_ID}"}
        │   ├── Response Assertion: code 200
        │   ├── JSON Assertion: $.status == ok
        │   └── JSON Extractor
        │       variable = SESSION_TOKEN
        │       path = $.session_token
        │       match = 1
        │       default = CORRELATION_MISSING
        ├── Simple Controller — Protected API
        │   ├── HTTP Header Manager
        │   │   Content-Type = application/json
        │   │   Authorization = Bearer ${SESSION_TOKEN}
        │   ├── HTTP Request — Catalog
        │   │   GET /catalog?sku=${SKU}
        │   │   ├── Response Assertion: code 200
        │   │   ├── JSON Assertion: $.status == ok
        │   │   └── JSON Extractor
        │   │       variable = ITEM_ID
        │   │       path = $.item_id
        │   │       match = 1
        │   │       default = CORRELATION_MISSING
        │   ├── HTTP Request — Cart
        │   │   POST /cart
        │   │   body = {"item_id":"${ITEM_ID}","qty":${QTY}}
        │   │   ├── Response Assertion: code 200
        │   │   └── JSON Assertion: $.status == ok
        │   └── HTTP Request — Checkout
        │       POST /checkout
        │       body = {"confirm":true}
        │       ├── Response Assertion: code 200
        │       ├── JSON Assertion: $.status == ok
        │       └── JSON Extractor
        │           variable = ORDER_ID
        │           path = $.order_id
        │           match = 1
        │           default = CORRELATION_MISSING

Configured local load:
10 journeys × 4 HTTP samplers = 40 HTTP samples.
The 75ms Constant Timer applies before each sampler and is not part of sampler elapsed time.

Author in the GUI because tree scope is easier to inspect there. Use View Results Tree only for a tiny one-user/one-loop debug pass, then remove/disable heavy listeners before governed load. The JSON Extractor defaults to CORRELATION_MISSING; an assertion/failure should stop that thread rather than let bad state continue.

6. Why each component is scoped here

7. CLI runner and evidence directory

tools/run_capstone.ps1:

param(
  [Parameter(Mandatory=$true)][string]$RunId,
  [Parameter(Mandatory=$true)][ValidateSet("baseline","current")][string]$Profile
)

$ErrorActionPreference = "Stop"
$Repo = (Resolve-Path ".").Path
$JMeter = Join-Path $env:JMETER_HOME "bin\jmeter.bat"
$Plan = Join-Path $Repo "plans\capstone.jmx"
$Props = Join-Path $Repo "config\capstone.properties"
$Charter = Join-Path $Repo "charter\authorization.json"
$Lock = Join-Path $Repo "lock\versions.lock.json"
$Data = Join-Path $Repo "data\users.csv"
$RunDir = Join-Path $Repo "runs\$RunId"

if (Test-Path $RunDir) { throw "Run directory already exists: $RunDir" }
New-Item -ItemType Directory -Force $RunDir | Out-Null

$CharterDoc = Get-Content $Charter -Raw | ConvertFrom-Json
if ($CharterDoc.authorized_target.host -ne "127.0.0.1" -or $CharterDoc.authorized_target.port -ne 8035) {
  throw "Charter target is not the approved loopback endpoint"
}
if ($CharterDoc.workload.configured_http_samples -ne 40) {
  throw "Unexpected configured sample count"
}

& $JMeter -v | Tee-Object -FilePath (Join-Path $RunDir "jmeter-version.txt")
java -version 2>&1 | Tee-Object -FilePath (Join-Path $RunDir "java-version.txt")

Get-FileHash $Plan,$Props,$Charter,$Lock,$Data -Algorithm SHA256 |
  Select-Object Path,Hash |
  ConvertTo-Json |
  Set-Content (Join-Path $RunDir "input-hashes.json")

$Health = Invoke-RestMethod "http://127.0.0.1:8035/health"
if ($Health.status -ne "ok") { throw "Target preflight failed" }

$Args = @(
  "-n",
  "-t", $Plan,
  "-q", $Props,
  "-Jrun.id=$RunId",
  "-l", (Join-Path $RunDir "results.jtl"),
  "-j", (Join-Path $RunDir "jmeter.log"),
  "-e",
  "-o", (Join-Path $RunDir "dashboard")
)

@{
  run_id = $RunId
  profile = $Profile
  jmeter_args = $Args
  started_utc = [DateTime]::UtcNow.ToString("o")
} | ConvertTo-Json -Depth 5 | Set-Content (Join-Path $RunDir "command.json")

& $JMeter @Args
$EngineExit = $LASTEXITCODE

$Stats = Invoke-RestMethod "http://127.0.0.1:8035/stats"
$Stats | ConvertTo-Json -Depth 8 | Set-Content (Join-Path $RunDir "target-stats.json")

# Record observed generator state; use these observations to decide generator_valid.
Get-Process java -ErrorAction SilentlyContinue |
  Select-Object Id,CPU,WorkingSet64,PrivateMemorySize64,Threads |
  ConvertTo-Json -Depth 5 |
  Set-Content (Join-Path $RunDir "generator-process-snapshot.json")

@{
  engine_exit_code = $EngineExit
  finished_utc = [DateTime]::UtcNow.ToString("o")
} | ConvertTo-Json | Set-Content (Join-Path $RunDir "engine-result.json")

exit $EngineExit

The runner refuses an existing directory, proves charter target/sample count, records versions/input hashes/command, invokes CLI with raw JTL + separate jmeter.log + HTML dashboard, captures target stats and a generator process snapshot, and never uses destructive -f.

Bash-equivalent core command:

jmeter -n \
  -t plans/capstone.jmx \
  -q config/capstone.properties \
  -Jrun.id=p35-baseline \
  -l runs/p35-baseline/results.jtl \
  -j runs/p35-baseline/jmeter.log \
  -e -o runs/p35-baseline/dashboard

8. Start the baseline service and preflight

python .\fixtures\capstone_service.py `
  --host 127.0.0.1 --port 8035 `
  --build-id build-baseline `
  --checkout-ms 35 `
  --max-requests 50 `
  --log .\runs\baseline-target-events.jsonl

Invoke-RestMethod http://127.0.0.1:8035/health

The health response must say build-baseline/checkout35/max50 before load. Run run_capstone.ps1 -RunId p35-baseline -Profile baseline. Expected target counts after the run: sessions10/catalog10/cart10/checkout10, requests40, errors0, rejections0.

9. Correlate JTL with target telemetry

tools/analyze_capstone.py:

import argparse, csv, json, math
from collections import Counter, defaultdict
from pathlib import Path

LABELS = ("Session","Catalog","Cart","Checkout")

def nearest_rank(values, pct):
    data = sorted(values)
    if not data:
        return 0
    return data[max(1, math.ceil(len(data)*pct/100.0))-1]

def main():
    p = argparse.ArgumentParser()
    p.add_argument("--jtl", required=True)
    p.add_argument("--target-events", required=True)
    p.add_argument("--build-id", required=True)
    p.add_argument("--run-id", required=True)
    p.add_argument("--generator-valid", choices=["true","false"], required=True)
    p.add_argument("--out", required=True)
    args = p.parse_args()

    rows = list(csv.DictReader(Path(args.jtl).open(newline="", encoding="utf-8")))
    rows = [r for r in rows if r.get("label") in LABELS]
    by_label = defaultdict(list)
    failed = 0
    for r in rows:
        by_label[r["label"]].append(int(float(r["elapsed"])))
        if r.get("success","").lower() != "true":
            failed += 1

    events = []
    for line in Path(args.target_events).read_text(encoding="utf-8").splitlines():
        if line.strip():
            d = json.loads(line)
            if d.get("build_id") == args.build_id:
                events.append(d)
    target_counts = Counter(e["endpoint"] for e in events)
    target_service = defaultdict(list)
    for e in events:
        target_service[e["endpoint"]].append(int(e["service_wall_ms"]))

    summary = {
        "schema_version": 1,
        "analysis_version": "p35-analysis-v1",
        "build_id": args.build_id,
        "run_id": args.run_id,
        "configured_http_samples": 40,
        "achieved_http_samples": len(rows),
        "generator_valid": args.generator_valid == "true",
        "error_rate_pct": round(100*failed/len(rows),4) if rows else 100.0,
        "labels": {},
        "target_counts": dict(target_counts),
        "target_service": {}
    }
    for label in LABELS:
        vals = by_label[label]
        summary["labels"][label] = {
            "count": len(vals),
            "avg_ms": round(sum(vals)/len(vals),3) if vals else 0,
            "p95_ms": nearest_rank(vals,95)
        }
        endpoint = label.lower()
        sv = target_service[endpoint]
        summary["target_service"][endpoint] = {
            "count": len(sv),
            "avg_ms": round(sum(sv)/len(sv),3) if sv else 0,
            "p95_ms": nearest_rank(sv,95)
        }

    summary["valid"] = (
        summary["generator_valid"]
        and summary["achieved_http_samples"] == 40
        and summary["error_rate_pct"] == 0
        and all(summary["labels"][x]["count"] == 10 for x in LABELS)
        and all(target_counts[x.lower()] == 10 for x in LABELS)
    )
    Path(args.out).write_text(json.dumps(summary, indent=2), encoding="utf-8")
    print(json.dumps(summary, indent=2))

if __name__ == "__main__":
    main()

python .\tools\analyze_capstone.py `
  --jtl .\runs\p35-baseline\results.jtl `
  --target-events .\runs\baseline-target-events.jsonl `
  --build-id build-baseline `
  --run-id p35-baseline `
  --generator-valid true `
  --out .\runs\p35-baseline\summary.json

The summary is valid only when 40 HTTP samples exist, each label has10, target telemetry independently has10 events per endpoint, error rate is zero, and generator validity is explicitly true. Compare Checkout JTL p95 with server-side checkout p95: they should move in the same direction, but JTL includes client/network/JMeter overhead while target service time does not.

10. HTML versus telemetry

The HTML dashboard is generated from JTL and helps explore response time, errors, throughput, active threads and connect/latency views. Target JSONL//stats is independent server telemetry. Generator snapshot is the third plane. If the dashboard says checkout worsened but target checkout service time did not, investigate generator/network/client state before blaming the SUT.

Chapter24's Backend Listener/Grafana path can replace or supplement local target telemetry in a real program. It is optional here because the capstone must remain free/local with no telemetry server.

11. Simulated distributed preflight

tools/fleet_preflight.py:

import argparse, csv, hashlib, json
from pathlib import Path

def sha256(path):
    h = hashlib.sha256()
    with Path(path).open("rb") as f:
        for chunk in iter(lambda:f.read(1024*1024), b""):
            h.update(chunk)
    return h.hexdigest()

def load_manifest(path):
    return json.loads(Path(path).read_text(encoding="utf-8"))

def users(path):
    with Path(path).open(newline="", encoding="utf-8") as f:
        return {r["USER_ID"] for r in csv.DictReader(f)}

def main():
    p = argparse.ArgumentParser()
    p.add_argument("--engine-a-manifest", required=True)
    p.add_argument("--engine-b-manifest", required=True)
    p.add_argument("--engine-a-data", required=True)
    p.add_argument("--engine-b-data", required=True)
    p.add_argument("--out", required=True)
    args = p.parse_args()
    a, b = load_manifest(args.engine_a_manifest), load_manifest(args.engine_b_manifest)
    problems = []

    for key in ("jmeter","java_major","plan_sha256","config_sha256","analysis_version"):
        if a.get(key) != b.get(key):
            problems.append(f"manifest_mismatch:{key}")
    if a.get("jmeter") != "5.6.3" or b.get("jmeter") != "5.6.3":
        problems.append("jmeter_not_5.6.3")
    if a.get("java_major") != 17 or b.get("java_major") != 17:
        problems.append("java_not_17")
    if a.get("threads") != 1 or b.get("threads") != 1 or a.get("loops") != 5 or b.get("loops") != 5:
        problems.append("distributed_profile_not_1x5_per_engine")

    au, bu = users(args.engine_a_data), users(args.engine_b_data)
    if au & bu:
        problems.append("shared_user_data_between_engines")
    if len(au) != 5 or len(bu) != 5:
        problems.append("each_engine_must_have_5_unique_users")

    expected_http = (a.get("threads",0)*a.get("loops",0) + b.get("threads",0)*b.get("loops",0))*4
    if expected_http != 40:
        problems.append(f"fleet_configured_http_samples={expected_http}_not_40")

    result = {
        "status":"PASS" if not problems else "FAIL",
        "problems":problems,
        "fleet_configured_http_samples":expected_http,
        "engine_a_data_sha256":sha256(args.engine_a_data),
        "engine_b_data_sha256":sha256(args.engine_b_data),
        "note":"Preflight simulation only; no RMI traffic was generated."
    }
    Path(args.out).write_text(json.dumps(result, indent=2), encoding="utf-8")
    print(json.dumps(result, indent=2))
    raise SystemExit(0 if not problems else 3)

if __name__ == "__main__":
    main()

For a two-engine profile, each engine is configured as 1 thread×5 loops, not2×5, because real JMeter remote servers each run the whole plan. Split the ten CSV users into five unique rows per engine. Each engine therefore produces5 journeys×4=20 samples; fleet total remains40. The preflight checks JMeter5.6.3, Java17, plan/config hashes, analysis version, profile math, and non-overlapping user data. No RMI traffic occurs.

12. Optional container path

The mandatory service runs directly with local Python. If a team prefers a disposable container, a minimal fixture image can be built from:

# OPTIONAL — not required for the course.
# The moving tag is convenient for a disposable lab but is NOT an auditable release lock.
# In a real governed environment, resolve and pin an immutable digest.
FROM python:3.13-slim
WORKDIR /app
COPY capstone_service.py /app/capstone_service.py
EXPOSE 8035
ENTRYPOINT ["python", "/app/capstone_service.py"]

python:3.13-slim is intentionally treated as a moving lab tag in this example, not an immutable production lock. A governed container workflow resolves/pins a digest, records Docker/runtime/image identity, binds only loopback/private ports, and preserves the same charter/sample guard. Containerizing JMeter itself is optional and must not hide engine resource limits or extension/data parity.

13. CI-style gate and governance policy

governance/policy.json:

{
  "schema_version": 1,
  "policy_version": "p35-policy-v1",
  "baseline": {
    "mode": "fixed_versioned",
    "owner": "performance-program-owner",
    "change_reason_required": true,
    "silent_overwrite_forbidden": true
  },
  "validity": {
    "required_http_samples": 40,
    "required_journeys": 10,
    "required_error_rate_pct": 0.0,
    "generator_must_be_valid": true,
    "workload_profile": "p35-local-v1",
    "analysis_version": "p35-analysis-v1"
  },
  "gates": {
    "checkout_absolute_p95_ms_max": 80.0,
    "checkout_relative_p95_regression_pct_max": 25.0
  },
  "evidence": {
    "raw_jtl_and_jmeter_log_required": true,
    "target_telemetry_required": true,
    "input_hashes_required": true,
    "dashboard_required": true,
    "generator_observation_required": true
  },
  "exceptions": {
    "supported": true,
    "must_not_modify_technical_gate": true,
    "owner_approver_issue_expiry_required": true
  }
}

tools/gate_capstone.py:

import argparse, json
from pathlib import Path

def main():
    p = argparse.ArgumentParser()
    p.add_argument("--baseline", required=True)
    p.add_argument("--current", required=True)
    p.add_argument("--out", required=True)
    args = p.parse_args()
    base = json.loads(Path(args.baseline).read_text(encoding="utf-8"))
    cur = json.loads(Path(args.current).read_text(encoding="utf-8"))

    invalid = []
    for doc, name in ((base,"baseline"),(cur,"current")):
        if not doc.get("valid"):
            invalid.append(f"{name}_invalid")
        if doc.get("analysis_version") != "p35-analysis-v1":
            invalid.append(f"{name}_analysis_version")
        if doc.get("achieved_http_samples") != 40:
            invalid.append(f"{name}_achieved_sample_count")

    b = float(base["labels"]["Checkout"]["p95_ms"])
    c = float(cur["labels"]["Checkout"]["p95_ms"])
    regression = 100.0*(c-b)/b if b else float("inf")

    checks = [
        {"name":"checkout_absolute_p95_ms","observed":c,"limit":80.0,"pass":c <= 80.0},
        {"name":"checkout_relative_p95_regression_pct","observed":round(regression,4),"limit":25.0,"pass":regression <= 25.0},
        {"name":"current_error_rate_pct","observed":cur["error_rate_pct"],"limit":0.0,"pass":cur["error_rate_pct"] <= 0.0},
        {"name":"current_achieved_http_samples","observed":cur["achieved_http_samples"],"limit":40,"pass":cur["achieved_http_samples"] == 40}
    ]
    status = "INVALID" if invalid else ("PASS" if all(x["pass"] for x in checks) else "FAIL")
    result = {
        "schema_version": 1,
        "policy_version": "p35-policy-v1",
        "status": status,
        "invalid_reasons": invalid,
        "checks": checks,
        "failed_metrics": [x["name"] for x in checks if not x["pass"]],
        "release_disposition": "ALLOW" if status == "PASS" else ("BLOCK_INVALID_RUN" if status == "INVALID" else "BLOCK")
    }
    Path(args.out).write_text(json.dumps(result, indent=2), encoding="utf-8")
    print(json.dumps(result, indent=2))
    raise SystemExit(0 if status == "PASS" else (2 if status == "FAIL" else 3))

if __name__ == "__main__":
    main()

Baseline/current summaries must both be valid. Gate policy: checkout p95≤80ms and relative checkout p95 regression≤25%, with zero current errors and40 achieved samples. Exit0=PASS,2=FAIL,3=INVALID. A provider-specific GitHub/GitLab/Jenkins job from Chapter28 can simply call this local gate and archive its evidence.

14. Runbook

runbook/RUNBOOK.md:

# P35 local performance-program runbook

## Before a run
1. Confirm authorization charter is `P35-LOCAL-CAPSTONE`.
2. Confirm target is exactly `127.0.0.1:8035`.
3. Verify JMeter 5.6.3 and Java 17.
4. Verify JMX/config/data/charter/lock hashes.
5. Verify CSV contains exactly ten unique synthetic users.
6. Start the fixture for the intended build and verify `/health`.
7. Confirm no prior run directory will be overwritten.

## Execute
1. Author/debug only in GUI with tiny data if needed.
2. Execute meaningful run in CLI with `run_capstone.ps1`.
3. Do not add retries, extra listeners, or workload while diagnosing.
4. Abort on target guard, non-loopback target, unexpected errors, generator saturation, or missing evidence.

## Validate
1. 40 configured HTTP samples == 40 achieved samples.
2. Each label Session/Catalog/Cart/Checkout has exactly ten samples.
3. Target telemetry has exactly ten events per endpoint.
4. Error rate is zero.
5. Correlation default `CORRELATION_MISSING` never appears in a failed chain.
6. Generator observation indicates adequate headroom for the small run.
7. Dashboard and raw JTL tell a consistent story.

## Gate / governance
1. Compare only valid summaries generated by `p35-analysis-v1`.
2. Keep the technical gate immutable.
3. Baseline promotion creates a new version; never overwrite.
4. Exception handling follows Chapter 34 and never repairs INVALID evidence.

## Failure handoff
Preserve JTL, jmeter.log, command, hashes, target events/stats, generator state, plan/config/data, and exact first symptom before changing anything.

## Cleanup
Stop fixture; confirm port 8035 is closed. Retain governed evidence per policy; delete synthetic runtime data only after review.

The runbook is part of the platform: ownership/abort/evidence/handoff rules must be executable by someone who did not author the JMX.

15. Challenge

You want two injectors but must keep the authorized total at ten journeys. Should you keep2 threads×5 loops on both engines and trust the target guard?

No. That config creates20 journeys/80 HTTP requests before the guard, violating the charter. Reprofile to1×5 per engine (or otherwise partition the workload), shard data, verify fleet math and parity, then request authorization if the desired total changes.

Knowledge check

Official references and version notes

• Apache JMeter downloads — current stable JMeter 5.6.3, Java 8+.
• JMeter current changes — Java 17+ recommended for the 5.6.x line.
• JMeter Getting Started — GUI authoring/debug, CLI flags/properties and runtime configuration.
• JMeter Component Reference — HTTP Request, Cookie Manager, CSV Data Set Config, JSON Extractor/assertions and component semantics.
• JMeter Best Practices — non-GUI load execution, listener cost and cached JSR223 Groovy guidance.
• JMeter Dashboard — CSV-backed HTML report and report generation.
• JMeter Remote Testing — whole-plan fan-out, exact JMeter parity, Java/data requirements, controller overhead and RMI SSL.