Checkpoint Lab — Scripting with JSR223 and Groovy for Advanced Test Logic

1. Exact assumptions and hard ceilings

2. Setup and target preflight

1. Create data/names.csv, scripts/canonicalize.groovy, and the fixture from Lesson 2.
2. Start fixture at 127.0.0.1:8019 with a fresh event log.
3. GET /health//stats; record fixture version and zero work count.
4. Record JMeter 5.6.3, Java 17, Groovy 3.0.20 and effective/default cache note.
5. Verify no extra plugin/ScriptEngine dependency is used.

3. Helper self-test is a gate

java -cp "$env:JMETER_HOME\lib\*" `
  groovy.ui.GroovyMain `
  .\scripts\canonicalize.groovy

Require PASS: prompt19 canonical helper self-test. If it fails, do not start JMeter load. Fix pure logic first.

4. State ownership before execution

5. Exact before/after trees

Before — over-scripted:

Thread Group
├── Counter -> SEQ
├── Constant Timer 10 ms
└── HTTP Work
    ├── JSR223 PreProcessor
    │   inline Groovy, cache OFF, ${RAW_NAME} substitution
    └── JSR223 PostProcessor
        inline Groovy, cache OFF, JsonSlurper + manual validation

After — refactored:

Thread Group
├── Counter -> SEQ
├── Constant Timer 10 ms
└── HTTP Work
    ├── JSR223 PreProcessor -> scripts/canonicalize.groovy
    ├── JSON JMESPath Assertion -> status == ok
    ├── JMESPath Extractors -> SERVER_NAME / SCORE / SIGNATURE
    └── Response Assertion -> SERVER_NAME == ${REQUEST_NAME}

Target URL/data/timer/thread count/result policy remain identical.

6. Predictions before running

Prediction A — behavior: both plans produce the same canonical-name distribution and zero unexpected target/JTL failures. Refactoring changes generator implementation, not business requests.

Prediction B — state isolation: sequence values are independent per thread because Counter and variables are thread-local; neither plan needs a mutable property for user state.

Prediction C — overhead: the refactored plan should eliminate repeated uncached Groovy compilation/interpreting and manual JSON parsing. It may improve achieved throughput/lower generator CPU, but the magnitude must be measured rather than asserted.

Prediction D — target: server service_wall_ms should remain broadly similar because target code/work is unchanged. If target timing changes materially, investigate before attributing the difference to Groovy.

7. Functional equivalence gate

Run each plan 1×4 with separate event/result directories. Require all four canonical names once (scheduler/order can differ when threads increase), correct status/name, zero unexpected failures and no Groovy exception.

Do not benchmark until this gate is green.

8. Bounded benchmark runs

Run over-scripted:

jmeter.bat -n `
  -t plans\over-scripted.jmx `
  -l results\over-scripted\results.jtl `
  -j results\over-scripted\jmeter.log `
  -JRUN_ID=p19-over `
  -JVARIANT=over-scripted

Restart/freshen fixture evidence as needed, then refactored:

jmeter.bat -n `
  -t plans\refactored.jmx `
  -l results\refactored\results.jtl `
  -j results\refactored\jmeter.log `
  -JRUN_ID=p19-ref `
  -JVARIANT=refactored

Each plan must enforce 2 threads ×300 loops max, 10 ms timer, ≤15-second duration safety cap.

9. Generator evidence during each variant

Resolve the current JMeter Java PID and record multiple CPU/memory snapshots plus jcmd PID GC.heap_info. Record jmeter.log size. Do not change heap, GC, log level, listener set, JVM flags, or OS priority between variants.

If the run is too short for reliable process sampling, repeat the same bounded benchmark a few times as separate runs rather than increasing threads/payload/target scope. Compare medians and keep each run under the stated cap.

10. Analyze configured versus achieved load

python tools/compare_jtl.py   results/over-scripted/results.jtl   results/refactored/results.jtl

python tools/analyze_server_events.py   results/server-events.jsonl

Configured per variant = at most 600 HTTP samples. Achieved = successful samples divided by measured result span, plus actual target event count. If a plan ends early, has failures, or sends a different request distribution, do not compare throughput as if the workloads were equal.

11. Exception evidence gate

Run a separate one-shot JSR223 Sampler:

throw new IllegalStateException("prompt19 synthetic exception - diagnostic only")

Require one failed sample and corresponding jmeter.log exception. Then disable/remove this diagnostic element. The purpose is to prove your evidence path catches script errors; it must not contaminate benchmark JTL.

12. Cache/configuration proof

Evidence packet must include:

• Groovy 3.0.20 output;
• documented/default jsr223.compiled_scripts_cache_size=100 or explicit override;
• refactored JMX showing Script File, not dynamic inline source;
• script source containing vars.get('RAW_NAME') and no direct JMeter variable replacement in script text;
• over-scripted source/cache-off configuration preserved for comparison.

13. Prove target behavior independently

Target event analysis must show:

• variant event counts consistent with successful JTL samples;
• only canonical names alice, bob_2, carol-3, delta;
• thread/sequence metadata in valid ranges;
• HTTP status 200 for valid workload;
• no hidden increase in target work for one variant.

14. Required evidence packet

15. Validity statement

16. Verification checklist

• Target exactly 127.0.0.1:8019.
• Helper self-test passes outside JMeter.
• Refactored source uses vars.get, not direct JMeter variable replacement in cached script text.
• RUN_ID/VARIANT props are immutable; per-user values stay in vars.
• Functional 1×4 behavior matches before benchmark.
• Both benchmark variants use identical threads/loops/timer/target/data/results.
• Achieved sample counts/throughput and target event counts agree.
• Generator CPU/heap/log evidence recorded; no tuning changes between variants.
• One exception sample is preserved separately and remains failed.
• No script marks real HTTP failures successful.

17. Cleanup / rollback

1. Stop JMeter runs and the localhost fixture.
2. Preserve JTL/jmeter.log/script/unit-test/target-event evidence until review completes.
3. Delete only disposable local results and fixture files after review.
4. No external API, real credential, environment secret, plugin, remote engine, recorder certificate, container, paid service, database/message target, OS/JVM global tuning, or production system was changed.

18. What Chapter 19 adds to the operating model

The production performance-testing operating model now has a scripting contract: built-in-first justification, JSR223 element lifecycle/scope, pinned JMeter/Java/Groovy engine, compiled-cache policy, cache-safe runtime bindings, vars/props ownership, script-file deployment/user.dir, pure-helper tests, exception/logging policy, SampleResult integrity, generator CPU/heap/log budget, target-equivalence evidence, and configured-versus-achieved load comparison are reviewed before scripted results become release evidence.

Chapter 20 moves to Functions, Custom Properties, Reusable Fragments, and Modular Plans. It builds on this chapter by standardizing reusable configuration and composition while keeping state ownership and execution semantics explicit.

Knowledge check

Official references and version notes

• JMeter Component Reference — JSR223 Sampler — compilation caching, bindings, script-file paths, SampleResult behavior, and Groovy guidance.
• JMeter Component Reference — JSR223 PreProcessor — scoped pre-sample scripting and bindings.
• JMeter Component Reference — JSR223 PostProcessor — post-sample scripting and previous-sample access.
• JMeter Component Reference — JSR223 Assertion — scripted assertion scope and bindings.
• JMeter Functions — __groovy — Groovy function bindings and cache-safe variable access.
• JMeter Properties Reference — jsr223.compiled_scripts_cache_size and advanced Groovy/JSR223 configuration.
• JMeter Best Practices — JSR223/Groovy recommendations, compiled scripting, lean results, and non-GUI load execution.
• Apache JMeter downloads — current stable release and Java requirement.
• Apache JMeter issue #6402 — JMeter 5.6.3 stack traces identify bundled Groovy 3.0.20 and document newer-JDK compatibility context.