Scripting with JSR223 and Groovy for Advanced Test Logic: Diagnostics, Failure Modes, and Production Practices

1. Preserve first-failure evidence

2. Diagnostic sequence

flowchart TD
E[Preserve JTL + jmeter.log + script + target evidence] --> V[Confirm JMeter / Java / Groovy / plugin/tool versions]
V --> C[Confirm exact JMX/data/properties/CLI + authorized target]
C --> S[Inspect JSR223 element scope / cache / source / parameters]
S --> D[Inspect vars vs props vs shared objects + resolved values]
D --> P[Inspect protocol/session/sample state and exception]
P --> G[Inspect generator CPU/GC/log/file/classpath]
G --> T[Inspect SUT event count/service timing]
T --> X[Inspect remote/CI/container user.dir/JAR/file state if relevant]
X --> F[Least destructive correction]
F --> R[Small controlled rerun and compare]

3. Intentionally broken example: ${RAW_NAME} inside cached script

Cache compiled script is checked:

// BROKEN ON PURPOSE when cache is enabled:
def raw = '${RAW_NAME}'
vars.put('REQUEST_NAME', raw.trim().toLowerCase(java.util.Locale.ROOT))

JMeter expands ${RAW_NAME} before compilation. The first expanded script can be cached, so later CSV rows can keep producing the first canonical value. Target events show repeated one-name traffic even though CSV advanced.

Repair:

// Cache-safe: script text stays constant; runtime value is read from JMeterVariables.
def raw = vars.get('RAW_NAME')
vars.put('REQUEST_NAME', raw.trim().toLowerCase(java.util.Locale.ROOT))

Script source is now stable and the current thread's value is read at execution time. Preserve the broken JTL/log/events before rerunning 1 thread ×4 rows.

4. Failure mode: non-thread-safe global object in props

Broken concept:

if (props.get('SEEN_NAMES') == null) {
    props.put('SEEN_NAMES', new ArrayList())
}
props.get('SEEN_NAMES').add(vars.get('REQUEST_NAME'))

Properties being shared does not make the stored ArrayList thread-safe, and the lazy initialization itself races. Worse, per-user collection does not belong in global test-plan state at all.

Repair: keep per-user data in vars. For real cross-thread aggregation, prefer JMeter results/backend telemetry or an explicitly designed concurrent structure with bounded memory and documented lifecycle—not ad-hoc properties.

5. Failure mode: INFO logging per sample

Broken:

log.info("user=${vars.get('REQUEST_NAME')} body=${prev.getResponseDataAsString()}")

This formats data and writes to jmeter.log for every sample, possibly including sensitive response data. Under load it can become disk/lock/GC pressure.

Repair: log one setup summary, use assertions/JTL fields/target telemetry for routine evidence, and temporarily enable narrowly scoped DEBUG only for a small reproduction.

6. Failure mode: BeanShell or JavaScript in a hot path without reason

JMeter explicitly recommends migration from BeanShell to JSR223 + Groovy for performance and newer Java support. Groovy implements Compilable; BeanShell/JavaScript do not provide the same compiled-cache path in JMeter guidance.

Do not mechanically translate a script if a built-in JMeter component can replace it. Migration order: built-in first → cached Groovy for irreducible logic → benchmark again.

7. Failure mode: swallowing exceptions and continuing with stale state

Broken:

try {
    vars.put('REQUEST_NAME', canonicalize(vars.get('RAW_NAME')))
} catch (Exception ignored) {
    // continue; REQUEST_NAME may still contain a previous iteration's value
}

The HTTP sampler can now send stale data while the script appears quiet. Repair by validating inputs and allowing the exception/failure to surface, or set an explicit sentinel and fail with an assertion. Preserve the original stack trace in jmeter.log.

8. Failure mode: mutating SampleResult to hide a real failure

Broken PostProcessor:

if (!prev.isSuccessful()) {
    prev.setSuccessful(true)
    prev.setResponseCode('200')
    prev.setResponseMessage('forced success')
}

This destroys the original HTTP/error taxonomy. A performance gate can pass while the target is returning 500/401/assertion failures.

Repair: never force real target samples to success. If an error is expected by scenario design, preserve the original status and classify it with a dedicated label/assertion/analysis rule.

9. Failure mode: Script File works in GUI but fails in CI

Relative JSR223 Script File paths resolve from the Java user.dir system property. Running JMeter from another working directory can make scripts/canonicalize.groovy disappear.

Repair: define a reproducible project root and run CLI from it, or supply an explicit project-root property/path policy. In distributed mode, deploy the file to every engine at the same expected path; controller-local existence is insufficient.

10. Failure mode: dynamic source defeats cache

A script embeds current timestamps, IDs, or variable replacement into source text, creating many unique MD5 cache keys. Compilation/cache churn consumes CPU/memory and can evict legitimate scripts from the default 100-entry cache.

Keep source constant and pass data through bindings/Parameters. Do not “fix” dynamic source by blindly increasing cache size.

11. Failure mode: deep ctx/sampler manipulation when a component exists

ctx/sampler are powerful, but reflective/deep manipulation tightly couples the plan to JMeter implementation details. If HTTP Header Manager, Counter, extractor, assertion, controller, or property field already provides the behavior, use the component.

12. Causal performance table

13. Security-sensitive scripting

Scripts can read environment variables, filesystem files, JVM properties, credentials, process state, databases/messages/APIs, and can start processes. Treat Groovy as code with the same trust as JMeter itself. Use fake values locally; never dump environment variables/secrets to logs; do not invoke shell/processes from a hot path; keep recorder CA, RMI, CI secrets, containers and OS/JVM tuning outside this lab.

14. Troubleshooting shortcuts to reject

• Do not add blanket retries or arbitrary long sleeps around script exceptions.
• Do not increase heap to compensate for unbounded global objects/logs.
• Do not disable all listeners without evidence; isolate the actual scripting/log/result cost.
• Do not use global properties as per-user state or synchronization hacks.
• Do not disable TLS/RMI verification.
• Do not experiment on production/public targets.
• Do not increase threads while script correctness/cache/thread-safety is unresolved.
• Do not delete failed JTL/jmeter.log or target evidence.

Knowledge check

Official references and version notes

• JMeter Component Reference — JSR223 Sampler — compilation caching, bindings, script-file paths, SampleResult behavior, and Groovy guidance.
• JMeter Component Reference — JSR223 PreProcessor — scoped pre-sample scripting and bindings.
• JMeter Component Reference — JSR223 PostProcessor — post-sample scripting and previous-sample access.
• JMeter Component Reference — JSR223 Assertion — scripted assertion scope and bindings.
• JMeter Functions — __groovy — Groovy function bindings and cache-safe variable access.
• JMeter Properties Reference — jsr223.compiled_scripts_cache_size and advanced Groovy/JSR223 configuration.
• JMeter Best Practices — JSR223/Groovy recommendations, compiled scripting, lean results, and non-GUI load execution.
• Apache JMeter downloads — current stable release and Java requirement.
• Apache JMeter issue #6402 — JMeter 5.6.3 stack traces identify bundled Groovy 3.0.20 and document newer-JDK compatibility context.