redis-benchmark and Custom Load Tests: Pipeline Depth, Payload, Durability, Skew, and Tail Latency

Learning outcomes

This lesson turns redis-benchmark and Custom Load Tests: Pipeline Depth, Payload, Durability, Skew, and Tail Latency into an observable AtlasMart workflow with explicit correctness, failure, and production boundaries.

1. Problem: a benchmark says 400k ops/s, but checkout misses its SLO

A throughput number is only meaningful for the workload that produced it. AtlasMart has mixed reads/writes, hundreds-to-thousands-of-byte payloads, hot-key skew, connection reuse, AOF everysec, and a tail-latency SLO. A synthetic benchmark that uses 3-byte values and uniform keys answers a different question.

2. Define the workload contract before running a tool

3. redis-benchmark is a useful baseline when its assumptions are written down

Redis documents -c clients, -n requests, -d payload size, -r random keyspace, and -P pipeline depth. Use exact parameters and never publish the tool defaults as production capacity. Run only against the disposable Chapter 24 node.

docker exec atlasmart-redis-ch24 redis-benchmark -h 127.0.0.1 -p 6379 --user academy-admin -a AtlasMart-Ch24-Admin-Lab-Only-2026 -t get,set -n 20000 -c 20 -d 512 -r 10000 -P 1 --csvdocker exec atlasmart-redis-ch24 redis-benchmark -h 127.0.0.1 -p 6379 --user academy-admin -a AtlasMart-Ch24-Admin-Lab-Only-2026 -t get,set -n 20000 -c 20 -d 512 -r 10000 -P 8 --csv

The second run tests a different transport shape. Higher throughput with -P 8 does not prove the application can or should batch eight independent operations.

4. Custom AtlasMart workload: realistic payload, deterministic 80/20 skew

import random, statistics, time, redisrng = random.Random(2403)r = redis.Redis(host="127.0.0.1", port=6441, username="atlasmart-app", password="AtlasMart-Ch24-App-Lab-Only-2026")hot = [f"atlasmart:ch24:bench:product:{i}" for i in range(20)]cold = [f"atlasmart:ch24:bench:product:{i}" for i in range(20,1000)]payload = b"p" * 512p = r.pipeline(transaction=False)for k in hot + cold: p.set(k, payload)p.execute()def choose_key(): return rng.choice(hot if rng.random() < 0.80 else cold)for _ in range(500): r.get(choose_key())  # warmuplat=[]; errors=0; start=time.perf_counter()for _ in range(5000):    t0=time.perf_counter_ns()    try: r.get(choose_key())    except Exception: errors += 1    lat.append((time.perf_counter_ns()-t0)/1e6)elapsed=time.perf_counter()-starts=sorted(lat)def pct(q): return s[min(len(s)-1, int((len(s)-1)*q))]print({"rps": round(len(lat)/elapsed,1), "p50_ms": pct(.50), "p95_ms": pct(.95), "p99_ms": pct(.99), "max_ms": max(s), "errors": errors})

5. Pipeline depth must preserve result mapping and memory bounds

Pipelining is transport batching, not atomicity. Compare the same logical work at depth 1, 4, and 8; verify response ordering and equality; measure client memory and server output buffers if batch size grows. A giant pipeline can improve throughput while worsening per-batch latency and memory pressure.

import time, redisr = redis.Redis(host="127.0.0.1", port=6441, username="atlasmart-app", password="AtlasMart-Ch24-App-Lab-Only-2026")keys=[f"atlasmart:ch24:bench:product:{i}" for i in range(1000)]for depth in (1,4,8):    t0=time.perf_counter(); count=0    for i in range(0, 4000, depth):        p=r.pipeline(transaction=False)        batch=[keys[(i+j)%len(keys)] for j in range(depth)]        for k in batch: p.get(k)        out=p.execute(); assert len(out)==len(batch); count += len(out)    print(depth, round(count/(time.perf_counter()-t0),1), "ops/s")

6. Coordinated omission: a closed-loop client can hide queueing pain

A client that sends the next request only after the previous one completes automatically reduces offered load during a pause. That can under-sample the requests users would have attempted during the stall. At minimum, record scheduled-start lateness or use a load generator capable of open-loop arrivals. The following local harness records both response latency and schedule lag; it is educational, not a replacement for a mature performance framework.

import time, redisr=redis.Redis(host="127.0.0.1", port=6441, username="atlasmart-app", password="AtlasMart-Ch24-App-Lab-Only-2026")interval=0.002  # 500 offered requests/s in this bounded examplen=2000; base=time.perf_counter(); response=[]; lateness=[]for i in range(n):    target=base+i*interval    now=time.perf_counter()    if now < target: time.sleep(target-now)    actual=time.perf_counter(); lateness.append(max(0.0, actual-target)*1000)    t0=time.perf_counter(); r.get(f"atlasmart:ch24:bench:product:{i%1000}"); response.append((time.perf_counter()-t0)*1000)print("max_schedule_lag_ms", max(lateness), "max_response_ms", max(response))

7. Capacity is an SLO-constrained region, not the crash point

Increase offered load in bounded steps and stop when tail latency, errors/timeouts, CPU, memory headroom, replication lag, or durability behavior crosses an acceptance threshold. Keep a reserve for failover, fork copy-on-write, AOF rewrite, backups, resharding, or traffic bursts. Dataset bytes alone are not capacity.

8. Deliberately wrong: publish redis-benchmark defaults as production capacity

9. Benchmark evidence worksheet

10. Production judgment and bridge

Benchmark versions, payloads, durability, client libraries, topology, warmup, errors, and host conditions alongside the numbers. Never compare runs that changed multiple dimensions without labeling the difference. The next lesson proves that backup and upgrade safety are also measured properties, not configuration checkboxes.

Summary and next step

redis-benchmark and Custom Load Tests: Pipeline Depth, Payload, Durability, Skew, and Tail Latency is now connected to observable Redis behavior, bounded failure cases, and production tradeoffs. Keep the evidence and cleanup state from this lesson; next, continue with Back Up RDB/AOF, Test Restore, Perform Rolling/Topology-Aware Upgrades, and Maintain Rollback Plans.

Authoritative references

• Info
• Latency
• Latmon
• Slowlog
• Client
• Monitor
• Benchmark
• Persistence
• Release
• Cli