Batch Size, Memory, Backpressure, and Latency/Throughput Tradeoffs

Learning outcomes

After Lesson 1, AtlasMart can batch commands correctly. The next failure mode is operational: treating “larger batch” as synonymous with “better performance.” Every queued request and reply occupies memory and increases the amount of work one application request waits to complete. This lesson measures those tradeoffs with bounded synthetic data.

1. Batch size creates a queueing boundary

A batch of 1 minimizes “time waiting for the rest of the batch” but pays many round trips. A very large batch amortizes RTT aggressively but holds more request/reply state, creates a larger unit of retry/diagnosis, and can monopolize application work for longer. The useful operating point depends on RTT, payload size, concurrency, parser cost, Redis CPU, output buffers, persistence, and the latency objective.

2. Measure the client queue before sending anything

redis-py queues pipeline commands in the application process until execute(). The following fixture uses tracemalloc to compare Python allocation peaks while queuing different numbers of 512-byte values. It deliberately does not execute the pipeline, so it isolates client-side queued-command cost.

import gc, tracemalloc, redisr = redis.Redis(host="127.0.0.1", port=6379, username="atlasmart-app", password="AtlasMart-App-Lab-Only-2026", decode_responses=False)payload = b"x" * 512for n in (10, 100, 1000, 5000):    gc.collect()    tracemalloc.start()    pipe = r.pipeline(transaction=False)    for i in range(n):        pipe.set(f"atlasmart:ch15:l2:queued:{i}", payload)    current, peak = tracemalloc.get_traced_memory()    tracemalloc.stop()    pipe.reset()  # drop queued commands without sending them    print(n, "peak_bytes", peak)# Record your measured peaks; Python allocator/client version affect exact numbers.

3. A slow reader can make reply buffering visible

Redis must send one reply for every command. To make that normally brief state observable, this bounded experiment creates a raw TCP client, authenticates it, names it, sends many GET requests for a 4 KiB value, and intentionally delays reading. A second administrative connection inspects the named client. Kernel socket buffering may absorb some or all replies, so output-buffer fields can still be zero on a fast machine; that is evidence about the environment, not proof that buffering does not exist.

import socket, time, redisHOST, PORT = "127.0.0.1", 6379KEY = "atlasmart:ch15:l2:blob"admin = redis.Redis(host=HOST, port=PORT, username="academy-admin", password="AtlasMart-Admin-Lab-Only-2026", decode_responses=True)app = redis.Redis(host=HOST, port=PORT, username="atlasmart-app", password="AtlasMart-App-Lab-Only-2026")app.set(KEY, b"z" * 4096)def resp(*parts):    out = [f"*{len(parts)}\r\n".encode()]    for part in parts:        b = str(part).encode()        out += [f"${len(b)}\r\n".encode(), b, b"\r\n"]    return b"".join(out)sock = socket.create_connection((HOST, PORT), timeout=2)sock.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 4096)wire = resp("AUTH", "atlasmart-app", "AtlasMart-App-Lab-Only-2026") + resp("CLIENT", "SETNAME", "atlasmart-ch15-slow-reader")wire += b"".join(resp("GET", KEY) for _ in range(2000))sock.sendall(wire)time.sleep(0.20)matches = [c for c in admin.client_list() if c.get("name") == "atlasmart-ch15-slow-reader"]print(matches)sock.close()app.unlink(KEY)# Inspect obl/oll/omem when present. Exact values are environment-dependent.

4. Compare batch sizes with identical total work

The benchmark below writes the same number of keys with the same 512-byte payload for every batch size. It reports throughput and the distribution of batch completion latency. Larger batches naturally contain more commands, so compare both throughput and latency—not latency alone.

import math, statistics, time, redisr = redis.Redis(host="127.0.0.1", port=6379, username="atlasmart-app", password="AtlasMart-App-Lab-Only-2026", decode_responses=False)N, payload = 2000, b"p" * 512def pct(values, q):    values = sorted(values)    return values[min(len(values)-1, math.ceil(q*len(values))-1)]for batch in (1, 10, 50, 250, 1000):    lat = []    t_all = time.perf_counter()    for start in range(0, N, batch):        p = r.pipeline(transaction=False)        for i in range(start, min(start + batch, N)):            p.set(f"atlasmart:ch15:l2:b{batch}:{i}", payload)        t0 = time.perf_counter()        p.execute()        lat.append((time.perf_counter()-t0)*1000)    elapsed = time.perf_counter() - t_all    print({"batch":batch,"ops_s":round(N/elapsed,1),"batch_p50_ms":round(pct(lat,.50),3),"batch_p95_ms":round(pct(lat,.95),3),"batch_p99_ms":round(pct(lat,.99),3)})    for start in range(0, N, 500):        r.unlink(*(f"atlasmart:ch15:l2:b{batch}:{i}" for i in range(start, min(start+500, N))))# Warm the environment separately if you want stable comparative runs; do not invent expected percentiles.

5. Backpressure means the producer is not allowed to outrun the consumer forever

If an application can enqueue Redis work faster than it can send/read replies, memory becomes the queue. Backpressure is the rule that caps in-flight work and makes producers wait, reject, shed, or persist work elsewhere when capacity is full. A bounded chunk generator is the simplest synchronous form.

def chunks(items, size):    batch = []    for item in items:        batch.append(item)        if len(batch) == size:            yield batch            batch = []    if batch:        yield batchfor batch in chunks(range(10000), 250):    pipe = r.pipeline(transaction=False)    for i in batch:        pipe.set(f"atlasmart:ch15:l2:chunk:{i}", "x")    pipe.execute()# At most 250 commands are queued in this client pipeline at a time.

6. Giant batches are also a timeout and retry problem

A larger batch takes longer to complete and creates a larger ambiguous unit if the connection fails after Redis processed some or all commands but before the client received every reply. Do not automatically retry a giant non-idempotent pipeline. Bound the batch and classify commands by retry safety before choosing recovery behavior.

7. Server memory is more than maxmemory

The Chapter 01 lab has no explicit maxmemory setting. Even when a production deployment does, client query/output buffers and allocator/process overhead are separate from the logical dataset budget. A batch-sizing experiment should therefore record Redis process memory, client/output state, payload size, and concurrent clients—not just MEMORY USAGE for keys.

docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin PINGdocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin INFO serverdocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin INFO clientsdocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin CONFIG GET appendonly appendfsync maxmemory maxmemory-policydocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin ACL WHOAMI# Record redis_version, persistence, client count, maxmemory policy, and ACL identity before interpreting measurements.

8. Select a measured operating range

Do not choose the batch with the maximum operations/second automatically. If batch 1000 adds little throughput over 250 but substantially increases p99 batch completion and memory, 250 may be the safer operating point. Conversely, a remote high-RTT path may justify larger batches. Record the curve and choose against your service-level objective (SLO), memory budget, and failure semantics.

9. Verification and cleanup

• Client queue memory was measured before network execution.
• The delayed-reader test used one named disposable connection and a bounded reply volume.
• Every batch-size run performed identical total writes and payload sizes.
• Both throughput and p50/p95/p99 batch latency were recorded.
• Cleanup used explicit prefixes/keys rather than FLUSHDB.

SCAN 0 MATCH atlasmart:ch15:l2:chunk:* COUNT 500# Iterate SCAN to cursor 0 and UNLINK only returned atlasmart:ch15:l2:chunk:* keys; do not use KEYS/FLUSHDB on shared data.

10. Production judgment

Batching is appropriate when per-command RTT dominates and operations can tolerate the chosen failure unit. Measure realistic payloads, concurrency, TLS/network distance, persistence, parsing, and reply sizes. Add bounded queues/semaphores in application code, enforce request deadlines, and alert on rising client output memory, timeouts, queue depth, and p99. Do not transfer one environment's “optimal batch size” into another.

11. Summary and next step

Batch size is a queueing and memory control, not a trophy number. Lesson 3 moves one layer outward to connection management: pools versus multiplexing, pool exhaustion, timeout budgets, reconnects, and retry ambiguity.

Authoritative references

• Redis Open Source 8.10 release notes
• Redis pipelining
• Using Redis commands
• Redis transactions
• redis-py guide
• redis-py pipelines and transactions
• redis-py production usage
• redis-py connection guide
• redis-py asynchronous usage
• redis-py error handling
• Connection pools and multiplexing
• Client-side caching introduction
• Client-side caching reference
• CLIENT TRACKING
• CLIENT TRACKINGINFO
• CLIENT CACHING
• CLIENT LIST
• CLIENT INFO
• CLIENT ID
• CLIENT SETNAME
• MONITOR
• INFO
• SLOWLOG
• Redis latency diagnosis
• Redis security
• Redis ACLs
• Redis Cluster specification
• Redis persistence
• Redis replication
• redis-py documentation
• redis-py 8.1.0 on PyPI
• Redis licenses