CLI Mode, Headless Execution, Result Files, and Reproducible Runs: Guided Hands-On Workflow

1. Safety envelope

2. Create the disposable local fixture

Save as fixtures/cli_fixture.py:

from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from urllib.parse import urlparse, parse_qs
import argparse
import json
import re
import threading
import time

FIXTURE_VERSION = "prompt21-cli-fixture-v1"
SAFE_TOKEN = re.compile(r"^[A-Za-z0-9_.-]{1,64}$")
lock = threading.Lock()
event_log = None
metrics = {
    "requests": 0,
    "errors": 0,
    "work_requests": 0,
    "by_run": {},
    "by_thread": {},
}

def now_ms():
    return int(time.time() * 1000)

def log_event(event):
    if event_log is None:
        return
    with lock:
        with event_log.open("a", encoding="utf-8") as handle:
            handle.write(json.dumps(event, sort_keys=True) + "\n")

class Handler(BaseHTTPRequestHandler):
    protocol_version = "HTTP/1.1"

    def send_json(self, status, payload):
        raw = json.dumps(payload, sort_keys=True).encode("utf-8")
        self.send_response(status)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(raw)))
        self.send_header("X-Fixture-Version", FIXTURE_VERSION)
        self.end_headers()
        self.wfile.write(raw)

    def record(self, started, operation, status, **extra):
        with lock:
            metrics["requests"] += 1
            if status >= 400:
                metrics["errors"] += 1
            if operation == "work" and status == 200:
                metrics["work_requests"] += 1
            run_id = extra.get("run_id", "")
            thread_id = extra.get("thread", "")
            if run_id:
                metrics["by_run"][run_id] = metrics["by_run"].get(run_id, 0) + 1
            if thread_id:
                metrics["by_thread"][thread_id] = metrics["by_thread"].get(thread_id, 0) + 1
        event = {
            "ts_ms": now_ms(),
            "operation": operation,
            "status": status,
            "service_wall_ms": now_ms() - started,
        }
        event.update(extra)
        log_event(event)

    def do_GET(self):
        started = now_ms()
        parsed = urlparse(self.path)

        if parsed.path == "/health":
            self.send_json(200, {"status": "ok", "fixture_version": FIXTURE_VERSION})
            self.record(started, "health", 200)
            return

        if parsed.path == "/stats":
            with lock:
                snapshot = json.loads(json.dumps(metrics))
            self.send_json(200, {"fixture_version": FIXTURE_VERSION, "metrics": snapshot})
            self.record(started, "stats", 200)
            return

        if parsed.path != "/work":
            self.send_json(404, {"status": "not_found"})
            self.record(started, "unknown", 404)
            return

        q = parse_qs(parsed.query)
        run_id = q.get("run_id", [""])[0]
        thread_id = q.get("thread", [""])[0]
        seq_raw = q.get("seq", [""])[0]
        payload = q.get("payload", [""])[0]

        if not SAFE_TOKEN.fullmatch(run_id) or not SAFE_TOKEN.fullmatch(thread_id):
            self.send_json(400, {"status": "invalid_metadata"})
            self.record(started, "work", 400, run_id=run_id, thread=thread_id)
            return
        if payload != "synthetic":
            self.send_json(400, {"status": "invalid_payload"})
            self.record(started, "work", 400, run_id=run_id, thread=thread_id)
            return
        try:
            seq = int(seq_raw)
        except ValueError:
            seq = -1
        if not 1 <= seq <= 1000:
            self.send_json(400, {"status": "invalid_seq"})
            self.record(started, "work", 400, run_id=run_id, thread=thread_id)
            return

        self.send_json(200, {
            "status": "ok",
            "run_id": run_id,
            "thread": thread_id,
            "seq": seq,
            "payload": payload,
        })
        self.record(started, "work", 200, run_id=run_id, thread=thread_id, seq=seq)

    def log_message(self, format, *args):
        return

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--host", default="127.0.0.1")
    parser.add_argument("--port", type=int, default=8021)
    parser.add_argument("--log", default="results/server-events.jsonl")
    args = parser.parse_args()

    global event_log
    event_log = Path(args.log).resolve()
    event_log.parent.mkdir(parents=True, exist_ok=True)
    event_log.write_text("", encoding="utf-8")

    print(f"fixture_version={FIXTURE_VERSION}")
    print(f"listen=http://{args.host}:{args.port}")
    print(f"event_log={event_log}")
    ThreadingHTTPServer((args.host, args.port), Handler).serve_forever()

if __name__ == "__main__":
    main()

Start it:

python .\fixtures\cli_fixture.py `
  --host 127.0.0.1 `
  --port 8021 `
  --log .\results\server-events.jsonl

The fixture has no persistent business state. It validates synthetic run/thread/sequence metadata and writes JSONL target evidence.

3. Target preflight

curl --fail --silent http://127.0.0.1:8021/health
curl --fail --silent http://127.0.0.1:8021/stats

Require fixture version prompt21-cli-fixture-v1. Record the current work count so later deltas can be attributed to one run ID.

4. Create the explicit property file

config/local.properties:

# Prompt 21 mandatory localhost CLI environment
target.host=127.0.0.1
target.port=8021
threads=2
loops=4
pacing.ms=20
connect.timeout.ms=500
response.timeout.ms=2000

# Lab-only post-run acceptance criteria.
# These are not production SLOs.
gate.expected.samples=8
gate.max.failures=0
gate.p95.ms=500

# Keep JTL dashboard-friendly and bounded.
jmeter.save.saveservice.output_format=csv
jmeter.save.saveservice.print_field_names=true
jmeter.save.saveservice.response_data=false
jmeter.save.saveservice.response_data.on_error=false
jmeter.save.saveservice.samplerData=false
jmeter.save.saveservice.responseHeaders=false
jmeter.save.saveservice.requestHeaders=false

The first group controls target/workload. The gate.* values are lab acceptance criteria for the post-run validator, not JMeter SLO logic. The save-service values keep response/body/header data out of the CSV JTL.

5. Build plans/cli-local.jmx in GUI

Test Plan
├── HTTP Request Defaults
│   Protocol=http
│   Host=${__P(target.host,127.0.0.1)}
│   Port=${__P(target.port,8021)}
│   Connect Timeout=${__P(connect.timeout.ms,500)}
│   Response Timeout=${__P(response.timeout.ms,2000)}
└── Thread Group
    Threads=${__P(threads,1)}
    Ramp-up=1 second
    Loop Count=${__P(loops,1)}
    ├── Counter
    │   Start=1, Increment=1
    │   Reference Name=SEQ
    │   Track counter independently for each user=true
    └── HTTP Request — Work
        GET /work
        run_id=${__P(run.id,p21-local)}
        thread=T${__threadNum}
        seq=${SEQ}
        payload=synthetic
        ├── Constant Timer ${__P(pacing.ms,20)} ms
        ├── JSON JMESPath Assertion: status == ok
        └── Response Assertion: response contains "payload": "synthetic"

The configured sample count is exactly threads × loops = 8 because the tree has one sampler per iteration.

6. First raw CLI run

PowerShell:

New-Item -ItemType Directory -Force .\results\raw | Out-Null

& "$env:JMETER_HOME\bin\jmeter.bat" `
  -n `
  -t .\plans\cli-local.jmx `
  -q .\config\local.properties `
  -Jrun.id=p21-raw `
  -l .\results\raw\results.jtl `
  -j .\results\raw\jmeter.log

$LASTEXITCODE

Bash:

mkdir -p results/raw
"$JMETER_HOME/bin/jmeter"   -n   -t plans/cli-local.jmx   -q config/local.properties   -Jrun.id=p21-raw   -l results/raw/results.jtl   -j results/raw/jmeter.log
printf 'engine_exit=%s
' "$?"

Inspect JTL header/rows, jmeter.log, console summary and target events before adding report generation.

7. Add at-end HTML report generation

Create only the parent run directory. Do not pre-create the report folder:

& "$env:JMETER_HOME\bin\jmeter.bat" `
  -n `
  -t .\plans\cli-local.jmx `
  -q .\config\local.properties `
  -Jrun.id=p21-report `
  -l .\results\p21-report\results.jtl `
  -j .\results\p21-report\jmeter.log `
  -e `
  -o .\results\p21-report\html-report

The raw JTL remains the primary source; HTML is derived after the run.

8. Regenerate the dashboard later with -g

From a retained CSV JTL:

& "$env:JMETER_HOME\bin\jmeter.bat" `
  -g .\results\p21-report\results.jtl `
  -o .\results\p21-report\html-report-regenerated

Use a new/empty output directory. This demonstrates why preserving JTL and compatible property settings matters.

9. Add the post-run validator

Save as tools/verify_run.py:

import csv
import json
import math
import sys
from pathlib import Path

def load_properties(path):
    props = {}
    for raw in Path(path).read_text(encoding="utf-8").splitlines():
        line = raw.strip()
        if not line or line.startswith("#") or "=" not in line:
            continue
        k, v = line.split("=", 1)
        props[k.strip()] = v.strip()
    return props

def percentile(values, pct):
    data = sorted(values)
    if not data:
        return 0
    idx = max(0, min(len(data) - 1, math.ceil((pct / 100.0) * len(data)) - 1))
    return data[idx]

if len(sys.argv) != 5:
    raise SystemExit("usage: verify_run.py <jtl> <properties> <engine_exit> <out_json>")

jtl = Path(sys.argv[1])
props_path = Path(sys.argv[2])
engine_exit = int(sys.argv[3])
out_json = Path(sys.argv[4])
props = load_properties(props_path)

result = {
    "engine_exit_code": engine_exit,
    "jtl_exists": jtl.is_file(),
    "gate_status": "NOT_RUN",
    "samples": 0,
    "failures": None,
    "p95_ms": None,
    "expected_samples": int(props.get("gate.expected.samples", "0")),
    "max_failures": int(props.get("gate.max.failures", "0")),
    "p95_limit_ms": int(props.get("gate.p95.ms", "0")),
    "reasons": [],
}

gate_exit = 0
if engine_exit != 0:
    result["gate_status"] = "ENGINE_FAILED"
    result["reasons"].append(f"JMeter engine/process exit code was {engine_exit}")
    gate_exit = 20
elif not jtl.is_file():
    result["gate_status"] = "ARTIFACT_FAILED"
    result["reasons"].append("JTL file is missing")
    gate_exit = 21
else:
    rows = list(csv.DictReader(jtl.open(newline="", encoding="utf-8")))
    required = {"timeStamp", "elapsed", "label", "responseCode", "success"}
    missing = required - set(rows[0].keys() if rows else [])
    if missing:
        result["gate_status"] = "ARTIFACT_FAILED"
        result["reasons"].append(f"JTL missing required columns: {sorted(missing)}")
        gate_exit = 22
    else:
        elapsed = [int(float(r["elapsed"])) for r in rows]
        failures = sum(r["success"].lower() != "true" for r in rows)
        result["samples"] = len(rows)
        result["failures"] = failures
        result["p95_ms"] = percentile(elapsed, 95)

        if result["samples"] != result["expected_samples"]:
            result["reasons"].append(
                f"sample count {result['samples']} != expected {result['expected_samples']}"
            )
        if failures > result["max_failures"]:
            result["reasons"].append(
                f"failures {failures} > allowed {result['max_failures']}"
            )
        if result["p95_ms"] > result["p95_limit_ms"]:
            result["reasons"].append(
                f"p95 {result['p95_ms']}ms > lab limit {result['p95_limit_ms']}ms"
            )

        if result["reasons"]:
            result["gate_status"] = "PERFORMANCE_OR_RESULT_FAILED"
            gate_exit = 30
        else:
            result["gate_status"] = "PASS"

out_json.parent.mkdir(parents=True, exist_ok=True)
out_json.write_text(json.dumps(result, indent=2), encoding="utf-8")
print(json.dumps(result, indent=2))
raise SystemExit(gate_exit)

The validator has three layers: engine/process exit, artifact/JTL structure, and performance/result criteria. It intentionally returns different non-zero codes so CI can preserve the failure class.

10. Finalize a run manifest

Save tools/finalize_manifest.py:

import hashlib
import json
import sys
from pathlib import Path

if len(sys.argv) != 12:
    raise SystemExit(
        "usage: finalize_manifest.py <pre_json> <jmx> <props> <jtl> <log> <report_dir> "
        "<gate_json> <engine_exit> <gate_exit> <jmeter_version_file> <java_version_file>"
    )

pre_path, jmx, props, jtl, log, report_dir, gate_json, engine_exit, gate_exit, jmeter_ver, java_ver = sys.argv[1:]

def sha256(path):
    p = Path(path)
    h = hashlib.sha256()
    with p.open("rb") as f:
        for chunk in iter(lambda: f.read(1024 * 1024), b""):
            h.update(chunk)
    return h.hexdigest()

pre = json.loads(Path(pre_path).read_text(encoding="utf-8"))
pre.update({
    "input_sha256": {
        "jmx": sha256(jmx),
        "properties": sha256(props),
    },
    "engine_exit_code": int(engine_exit),
    "gate_exit_code": int(gate_exit),
    "gate": json.loads(Path(gate_json).read_text(encoding="utf-8")) if Path(gate_json).is_file() else None,
    "artifacts": {
        "jtl": str(Path(jtl).resolve()) if Path(jtl).exists() else None,
        "jmeter_log": str(Path(log).resolve()) if Path(log).exists() else None,
        "html_report": str(Path(report_dir).resolve()) if Path(report_dir).exists() else None,
    },
    "versions": {
        "jmeter": Path(jmeter_ver).read_text(encoding="utf-8", errors="replace").strip(),
        "java": Path(java_ver).read_text(encoding="utf-8", errors="replace").strip(),
    },
})
out = Path(pre_path).with_name("run-manifest.json")
out.write_text(json.dumps(pre, indent=2), encoding="utf-8")
print(out)

The script hashes the actual JMX/property inputs after the run and combines them with command paths, version files, engine/gate exits and artifact locations. Inputs should not be modified between execution and hashing; a stricter production wrapper can hash both before and after and fail if they differ.

11. PowerShell launcher

Save tools/run-local.ps1:

param(
  [string]$RunId = "",
  [string]$PlanRelative = "plans\cli-local.jmx"
)

$ErrorActionPreference = "Stop"
if (-not $env:JMETER_HOME) {
  throw "JMETER_HOME must point to Apache JMeter 5.6.3."
}

$ProjectRoot = (Resolve-Path (Join-Path $PSScriptRoot "..")).Path
$Props = Join-Path $ProjectRoot "config\local.properties"
$Jmx = Join-Path $ProjectRoot $PlanRelative
$Tools = Join-Path $ProjectRoot "tools"

if ([string]::IsNullOrWhiteSpace($RunId)) {
  $RunId = [DateTime]::UtcNow.ToString("yyyyMMddTHHmmssZ") + "-" + (Get-Random -Minimum 1000 -Maximum 9999)
}
if ($RunId -notmatch '^[A-Za-z0-9_.-]{1,64}$') {
  throw "RunId must contain only A-Z a-z 0-9 _ . - and be <=64 chars."
}
if (-not (Test-Path -LiteralPath $Props -PathType Leaf)) {
  throw "Property file missing: $Props"
}
if (-not (Test-Path -LiteralPath $Jmx -PathType Leaf)) {
  throw "JMX file missing: $Jmx"
}

$RunDir = Join-Path $ProjectRoot ("results\{0}" -f $RunId)
if (Test-Path -LiteralPath $RunDir) {
  throw "Refusing to overwrite existing run directory: $RunDir"
}
New-Item -ItemType Directory -Path $RunDir | Out-Null

$Jtl = Join-Path $RunDir "results.jtl"
$Log = Join-Path $RunDir "jmeter.log"
$Report = Join-Path $RunDir "html-report"   # MUST NOT be created before JMeter -o
$Pre = Join-Path $RunDir "manifest-pre.json"
$Gate = Join-Path $RunDir "gate.json"
$JMeterVersion = Join-Path $RunDir "jmeter-version.txt"
$JavaVersion = Join-Path $RunDir "java-version.txt"
$CommandFile = Join-Path $RunDir "command.txt"

(& "$env:JMETER_HOME\bin\jmeter.bat" -v 2>&1 | Out-String) | Set-Content -Encoding UTF8 $JMeterVersion
(& java -version 2>&1 | Out-String) | Set-Content -Encoding UTF8 $JavaVersion

$CommandForManifest = @(
  "$env:JMETER_HOME\bin\jmeter.bat", "-n",
  "-t", $Jmx,
  "-q", $Props,
  "-Jrun.id=$RunId",
  "-l", $Jtl,
  "-j", $Log,
  "-e",
  "-o", $Report
)
$CommandForManifest -join " " | Set-Content -Encoding UTF8 $CommandFile

[ordered]@{
  run_id = $RunId
  created_utc = [DateTime]::UtcNow.ToString("o")
  cwd = (Get-Location).Path
  project_root = $ProjectRoot
  jmx = $Jmx
  property_file = $Props
  command = $CommandForManifest
  run_directory = $RunDir
  report_directory = $Report
} | ConvertTo-Json -Depth 5 | Set-Content -Encoding UTF8 $Pre

& "$env:JMETER_HOME\bin\jmeter.bat" `
  -n `
  -t "$Jmx" `
  -q "$Props" `
  -Jrun.id="$RunId" `
  -l "$Jtl" `
  -j "$Log" `
  -e `
  -o "$Report"
$EngineExit = $LASTEXITCODE
Set-Content -Encoding ASCII (Join-Path $RunDir "engine-exit.txt") $EngineExit

python (Join-Path $Tools "verify_run.py") `
  "$Jtl" "$Props" "$EngineExit" "$Gate"
$GateExit = $LASTEXITCODE
Set-Content -Encoding ASCII (Join-Path $RunDir "gate-exit.txt") $GateExit

python (Join-Path $Tools "finalize_manifest.py") `
  "$Pre" "$Jmx" "$Props" "$Jtl" "$Log" "$Report" "$Gate" `
  "$EngineExit" "$GateExit" "$JMeterVersion" "$JavaVersion"

if ($EngineExit -ne 0) { exit $EngineExit }
if ($GateExit -ne 0) { exit $GateExit }
exit 0

Important state changes:

• creates exactly one previously non-existent run directory;
• does not use -f;
• creates JTL/log/report in that run directory;
• captures versions and command before load;
• records the engine exit before running the gate;
• returns engine failure first, otherwise gate failure/success.

12. Bash launcher

Save tools/run-local.sh:

#!/usr/bin/env bash
set -u -o pipefail

RUN_ID="${1:-}"
PLAN_RELATIVE="${2:-plans/cli-local.jmx}"
: "${JMETER_HOME:?JMETER_HOME must point to Apache JMeter 5.6.3}"

SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd -- "$SCRIPT_DIR/.." && pwd)"
PROPS="$PROJECT_ROOT/config/local.properties"
JMX="$PROJECT_ROOT/$PLAN_RELATIVE"
TOOLS="$PROJECT_ROOT/tools"

if [[ -z "$RUN_ID" ]]; then
  RUN_ID="$(date -u +%Y%m%dT%H%M%SZ)-$RANDOM"
fi
if [[ ! "$RUN_ID" =~ ^[A-Za-z0-9_.-]{1,64}$ ]]; then
  echo "Invalid RunId" >&2
  exit 11
fi
[[ -f "$PROPS" ]] || { echo "Property file missing: $PROPS" >&2; exit 12; }
[[ -f "$JMX" ]] || { echo "JMX file missing: $JMX" >&2; exit 13; }

RUN_DIR="$PROJECT_ROOT/results/$RUN_ID"
if [[ -e "$RUN_DIR" ]]; then
  echo "Refusing to overwrite existing run directory: $RUN_DIR" >&2
  exit 14
fi
mkdir -p "$RUN_DIR"

JTL="$RUN_DIR/results.jtl"
LOG="$RUN_DIR/jmeter.log"
REPORT="$RUN_DIR/html-report"   # do not create before -o
PRE="$RUN_DIR/manifest-pre.json"
GATE="$RUN_DIR/gate.json"
JMETER_VERSION="$RUN_DIR/jmeter-version.txt"
JAVA_VERSION="$RUN_DIR/java-version.txt"
COMMAND_FILE="$RUN_DIR/command.txt"

"$JMETER_HOME/bin/jmeter" -v >"$JMETER_VERSION" 2>&1
java -version >"$JAVA_VERSION" 2>&1

python - "$PRE" "$RUN_ID" "$PWD" "$PROJECT_ROOT" "$JMX" "$PROPS" "$RUN_DIR" "$REPORT" <<'PY'
import json, sys
pre, run_id, cwd, root, jmx, props, run_dir, report = sys.argv[1:]
cmd = [
    "jmeter", "-n", "-t", jmx, "-q", props, f"-Jrun.id={run_id}",
    "-l", f"{run_dir}/results.jtl", "-j", f"{run_dir}/jmeter.log",
    "-e", "-o", report,
]
with open(pre, "w", encoding="utf-8") as f:
    json.dump({
        "run_id": run_id,
        "cwd": cwd,
        "project_root": root,
        "jmx": jmx,
        "property_file": props,
        "command": cmd,
        "run_directory": run_dir,
        "report_directory": report,
    }, f, indent=2)
with open(f"{run_dir}/command.txt", "w", encoding="utf-8") as f:
    f.write(" ".join(cmd) + "\n")
PY

set +e
"$JMETER_HOME/bin/jmeter" \
  -n \
  -t "$JMX" \
  -q "$PROPS" \
  -Jrun.id="$RUN_ID" \
  -l "$JTL" \
  -j "$LOG" \
  -e \
  -o "$REPORT"
ENGINE_EXIT=$?
printf '%s\n' "$ENGINE_EXIT" >"$RUN_DIR/engine-exit.txt"

python "$TOOLS/verify_run.py" "$JTL" "$PROPS" "$ENGINE_EXIT" "$GATE"
GATE_EXIT=$?
printf '%s\n' "$GATE_EXIT" >"$RUN_DIR/gate-exit.txt"

python "$TOOLS/finalize_manifest.py" \
  "$PRE" "$JMX" "$PROPS" "$JTL" "$LOG" "$REPORT" "$GATE" \
  "$ENGINE_EXIT" "$GATE_EXIT" "$JMETER_VERSION" "$JAVA_VERSION"

if [[ "$ENGINE_EXIT" -ne 0 ]]; then exit "$ENGINE_EXIT"; fi
exit "$GATE_EXIT"

Make it executable on Unix-like systems: chmod +x tools/run-local.sh. Python is used for JSON/hash/gate portability rather than assuming platform-specific JSON/hash utilities.

13. Execute the safe wrapper

PowerShell:

Set-Location "F:\Labs\p21-cli-lab"
.\tools\run-local.ps1 -RunId p21-success-a
$LASTEXITCODE

Bash:

cd /path/to/p21-cli-lab
./tools/run-local.sh p21-success-a
printf 'wrapper_exit=%s
' "$?"

Expected successful run: engine exit 0, gate exit 0, eight JTL sample rows, zero failed samples, p95 below the lab threshold, HTML index.html present, and eight target work events for that run ID.

14. Verify target evidence

Save tools/analyze_events.py:

import json
import sys
from collections import Counter
from pathlib import Path

path = Path(sys.argv[1])
run_id = sys.argv[2] if len(sys.argv) > 2 else None
events = [json.loads(line) for line in path.read_text(encoding="utf-8").splitlines() if line.strip()]
work = [e for e in events if e.get("operation") == "work" and (run_id is None or e.get("run_id") == run_id)]

print(f"work_events={len(work)}")
print(f"statuses={dict(Counter(e.get('status') for e in work))}")
print(f"threads={dict(Counter(e.get('thread') for e in work))}")
print(f"sequences={sorted((e.get('thread'), e.get('seq')) for e in work)}")
print(f"max_service_wall_ms={max([int(e.get('service_wall_ms',0)) for e in work] or [0])}")

python tools/analyze_events.py results/server-events.jsonl p21-success-a

Expect 8 work events, two thread IDs with four sequences each, all HTTP 200 and stable low local service time. This independently checks that “8 JTL rows” also correspond to 8 target operations.

15. Inspect jmeter.log deliberately

PowerShell:

Select-String `
  -Path .\results\p21-success-a\jmeter.log `
  -Pattern "\b(ERROR|FATAL)\b"

A zero match is useful evidence but not a complete gate. Some warnings may be relevant, and sample failures live in JTL.

16. Rerun from a clean/unrelated working directory

PowerShell:

Set-Location "$env:TEMP"
& "F:\Labs\p21-cli-lab\tools\run-local.ps1" -RunId p21-success-b

Because the launcher derives project root from its own path, the same JMX/property inputs are used. The new run directory keeps evidence isolated.

17. Challenge

A run has engine exit 0, eight JTL rows, one failed sample, p95 120 ms, and the lab limit is zero failures/500 ms. Should CI pass because JMeter exited successfully?

No. The engine layer succeeded, but the independent result gate fails on sample correctness. Preserve both statuses and fix/classify the sample failure before rerunning.

Knowledge check

Official references and version notes

• JMeter Getting Started — CLI mode — -n, -t, -l, -j, -g, -e, -o, -J, -L, remote flags, and CLI/load guidance.
• JMeter Listeners / Result files — CSV versus XML, save-service defaults, CLI -l listener, result fields, and memory guidance.
• JMeter Generating Dashboard Report — dashboard-required CSV fields, -g, -e -o, output-folder rules, graphs, and report properties.
• JMeter Best Practices — GUI authoring/debugging and CLI execution for load.
• Apache JMeter downloads — current stable release and Java requirement.