Checkpoint Lab — File Uploads, Downloads, Multipart Requests, and Large Payloads

1. Exact assumptions and ceilings

2. Setup and authorization/preflight

1. Run generate_payloads.py; archive manifest/CSV output.
2. Start fixture on 127.0.0.1:8018 with exact lab-storage root.
3. GET /health and /stats; require stored=0.
4. Record JMeter/Java/Python versions, generator free disk, payload directory bytes, result directory bytes.
5. Confirm JMX uses only manifest paths and no sensitive files.

3. Lean checkpoint tree

Test Plan
├── User Defined Variables -> RUN_ID=${__P(RUN_ID,prompt18-checkpoint)}
├── HTTP Request Defaults -> 127.0.0.1:8018
├── CSV Data Set Config -> data/payloads.csv
└── Thread Group — 2 threads × 2 loops max
    ├── Upload Payload
    │   POST /upload
    │   fields: run_id, payload_name
    │   file: ${payload_path}, parameter=file, MIME=${mime_type}
    │   ├── extract UPLOAD_ID, SERVER_BYTES, SERVER_SHA256, SERVER_MD5
    │   └── assert against expected_bytes/hash variables
    ├── Download Payload
    │   GET /download/${UPLOAD_ID}
    │   Save response as MD5 hash=true
    │   └── assert body == ${expected_md5}
    └── Delete Upload
        DELETE /upload/${UPLOAD_ID}
        └── assert 204

4. Predictions before lean execution

Prediction A — bytes: each upload's JTL sentBytes will exceed raw expected_bytes because HTTP/multipart adds envelope bytes. Download network traffic remains the full payload even though the SampleResult retains only an MD5 string.

Prediction B — storage: target storage rises after upload and returns after delete. With two concurrent threads, peak storage may contain up to roughly two in-flight payloads, but final stored bytes must be zero.

Prediction C — size scaling: medium transfers should generally send/receive more bytes and often take longer than small transfers; the exact latency ratio is not assumed because loopback/cache/server overhead is nonlinear.

5. Run the lean profile

jmeter.bat -n `
  -t plans\checkpoint-transfer-lean.jmx `
  -l results\lean\results.jtl `
  -j results\lean\jmeter.log `
  -JRUN_ID=prompt18-checkpoint `
  -Jsample_variables=payload_name,expected_bytes

Record generator CPU/RSS/heap if practical, free disk before/after, JTL size, and fixture process/storage state.

6. Verify lean profile independently

python tools/analyze_transfer_jtl.py results/lean/results.jtl
python tools/analyze_server_events.py results/lean/server-events.jsonl
python tools/disk_snapshot.py . data results lab-storage
curl --fail --silent http://127.0.0.1:8018/stats

Require zero unexpected JTL failures, correct server hash/bytes assertions, upload/download/delete operation counts consistent with completed journeys, final stored_uploads=0/stored_bytes=0, and generator headroom.

7. Prepare the deliberately inefficient retention run

Use a separate tiny plan with exactly one 512 KiB upload → download → delete. Change the download sampler:

• Save response as MD5 hash = false;
• no View Results Tree;
• same server/payload/asserted upload metadata;
• 1 thread ×1 journey only.

For the -l result listener, use XML output and enable response data:

jmeter.bat -n `
  -t plans\checkpoint-transfer-retained-body.jmx `
  -l results\retained-body\results.jtl `
  -j results\retained-body\jmeter.log `
  -JRUN_ID=prompt18-retained `
  -Jjmeter.save.saveservice.output_format=xml `
  -Jjmeter.save.saveservice.response_data=true `
  -Jjmeter.save.saveservice.response_data.on_error=false

8. Predictions before the inefficient run

Prediction D — artifact: the XML JTL will be much larger than a lean CSV file for one comparable transfer because it contains the entire 512 KiB text response plus XML/sample metadata.

Prediction E — target: server download payload size and service work should remain broadly comparable to a lean 512 KiB download; the major new cost is on JMeter/result serialization/storage.

Prediction F — memory/disk: generator result-disk activity and retained SampleResult/JTL body state increase. If the run is long enough to sample with jcmd, heap evidence may show additional retained data; regardless, JTL file size is direct proof of storage amplification.

9. Quantify retention overhead

Record:

python tools/disk_snapshot.py .   results/lean/results.jtl   results/retained-body/results.jtl   lab-storage

Compare:

• lean CSV JTL bytes on disk;
• retained-body XML JTL bytes on disk;
• server download payload_bytes/service_wall_ms;
• JMeter elapsed and generator state;
• final server storage state.

Do not expect exact equality between JTL file-size delta and 524,288 bytes because XML escaping/metadata/other sample results add overhead.

10. Restore the lean configuration

Restore:

• CSV JTL;
• response_data=false;
• Download Save response as MD5 hash=true;
• original bounded thread/loop limits.

Rerun one medium journey as results/restored/... and verify the small result artifact plus identical server file hash/size behavior.

11. Quantify network and generator overhead correctly

12. Required evidence packet

13. Validity statement

14. Verification checklist

• Only generated synthetic manifest files are uploaded.
• Target is exactly 127.0.0.1:8018.
• Files are 64 KiB/512 KiB and below 1 MiB hard limit.
• Upload bytes/hashes match manifest.
• Download MD5 matches manifest in lean profile.
• Lean CSV JTL keeps bytes/sentBytes and no full bodies.
• Inefficient XML/body run is exactly one medium journey and preserved separately.
• Restored profile returns to CSV + MD5.
• Final /stats shows zero stored uploads/bytes.
• Generator headroom/result disk size are recorded before any capacity statement.

15. Cleanup / rollback

1. Verify /stats is zero before stopping the fixture.
2. Stop JMeter and the localhost fixture.
3. Retain JTL/jmeter.log/manifest/server-event evidence until review completes.
4. Only after review, remove data/payloads, lab-storage, and results from this lab root.
5. Before any recursive manual deletion, print/resolve the path and require the exact expected directory name; never use a data-driven arbitrary path.
6. No production file/object store, real document, credential, plugin, container, remote engine, paid service, OS/JVM global tuning, or uncontrolled endpoint was modified.

16. What Chapter 18 adds to the operating model

The performance-testing operating model now has a payload/storage contract: synthetic payload provenance, size distribution/hash manifest, immutable/unique file ownership, multipart/raw-body semantics, configured/achieved bytes and transfer counts, JMeter response-retention mode, JTL byte/artifact policy, generator disk/heap/CPU/network budget, server storage/hash/processing telemetry, guarded cleanup root, and explicit distinction between network/server/storage versus injector/result overhead.

Chapter 19 moves to Scripting with JSR223 and Groovy for Advanced Test Logic. That chapter introduces a controlled scripting escape hatch for transformations/validation that genuinely cannot be expressed cleanly with built-in components, while measuring scripting/cache/generator CPU cost and preserving thread-safe variable ownership.

Knowledge check

Official references and version notes

• JMeter Component Reference — HTTP Request — multipart/form-data, file path/parameter/MIME fields, browser-compatible headers, and Save response as MD5 hash.
• JMeter Component Reference — Save Responses to a file — response-to-disk behavior, filename prefix, numbering, and variable output.
• JMeter Listeners / Result files — CSV fields including bytes/sentBytes, listener memory guidance, response-data retention, and CLI -l behavior.
• JMeter Properties Reference — save-service defaults, httpsampler.max_bytes_to_store_per_request, buffer sizing, and result retention settings.
• JMeter 5.6.3 Changes — expression support for HTTP Request “store as MD5” and current release changes.
• JMeter Best Practices — GUI for authoring/debug and CLI mode for meaningful load.
• Apache JMeter downloads — current stable release and Java requirement.