WATCH/UNWATCH for Optimistic Concurrency and Retry Loops

Learning outcomes

AtlasMart now needs a conditional update: “decrement stock only if the value I read is still current.” An unconditional MULTI/EXEC cannot detect that the read became stale before the transaction began. WATCH supplies optimistic concurrency by making the later EXEC conditional on watched keys remaining unchanged.

1. Optimistic locking means “verify, then commit if unchanged”

WATCH does not lock other clients out. Instead, Redis remembers the watched keys on this connection. The client reads state, computes a proposal, enters MULTI, queues writes, and calls EXEC. If any watched key was modified before EXEC arrives, Redis aborts the whole transaction and returns a null transaction result. The application decides whether to retry.

2. A two-client conflict trace makes the mechanism visible

Use two terminals. Terminal A must remain open because WATCH state is tied to its connection. Terminal B deliberately changes the watched key after A reads it but before A calls EXEC.

docker exec -it -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-adminSET atlasmart:ch13:l2:stock 3WATCH atlasmart:ch13:l2:stockGET atlasmart:ch13:l2:stock# Expected: "3". Leave this connection open.

docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin DECR atlasmart:ch13:l2:stock# Expected: 2

MULTISET atlasmart:ch13:l2:stock 2EXEC# RESP2 redis-cli shows a nil transaction; RESP3 uses a null reply representation.GET atlasmart:ch13:l2:stock# Expected: "2" from Terminal B; A did not execute its queued SET.

3. The read belongs after WATCH, not before it

If you GET, then later call WATCH, a competitor can modify the key in the gap and your subsequent WATCH cannot detect that earlier change. Correct optimistic concurrency begins watching before reading the state used to compute the write.

GET atlasmart:ch13:l2:stock# Another client may change the key here.WATCH atlasmart:ch13:l2:stock# This WATCH starts too late to protect the already-read value.

4. Expiration and eviction can invalidate WATCH too

Redis treats changes made by Redis itself—such as key expiration or eviction—as modifications for WATCH in modern versions. Therefore a transaction can abort even when no competing application explicitly wrote the key. This is correct: the state you observed no longer exists in the same form.

SET atlasmart:ch13:l2:ephemeral 1 PX 1500WATCH atlasmart:ch13:l2:ephemeral# Wait until the key expires, then verify EXISTS returns 0.MULTISET atlasmart:ch13:l2:ephemeral 2EXEC# On current Redis, expiry after WATCH invalidates EXEC. Do not use this as a timing benchmark.

5. UNWATCH is for a deliberate early exit

If the watched state tells you no write is needed—for example, inventory is already zero—UNWATCH clears the connection’s watch state so it can safely be reused. EXEC and DISCARD also clear watched keys automatically.

SET atlasmart:ch13:l2:stock 0WATCH atlasmart:ch13:l2:stockGET atlasmart:ch13:l2:stock# Application decides: no reservation is possible.UNWATCHPING# Expected: PONG; connection is back to ordinary use.

6. Bounded retries turn conflicts into an explicit SLO/capacity signal

A WATCH abort is not an exceptional Redis corruption event; it is an expected concurrency outcome. The dangerous pattern is an unbounded retry loop that can turn one hot key into a retry storm. Cap attempts, add jitter/backoff where appropriate, expose conflict counts, and let the caller receive a controlled “try again” or “busy” outcome when the budget is exhausted.

import randomimport timeimport redisr = redis.Redis(host="127.0.0.1", port=6379, username="academy-admin",                password="AtlasMart-Admin-Lab-Only-2026", decode_responses=True)key = "atlasmart:ch13:l2:stock"r.set(key, 3)MAX_ATTEMPTS = 8for attempt in range(1, MAX_ATTEMPTS + 1):    try:        with r.pipeline() as pipe:            pipe.watch(key)                 # pins WATCH state to this pipeline connection            current = int(pipe.get(key))    # executes immediately while watching            if current <= 0:                pipe.unwatch()                print("sold_out")                break            pipe.multi()                    # subsequent commands are buffered            pipe.set(key, current - 1)            result = pipe.execute()          # raises WatchError on conflict            print("committed", result, "attempt", attempt)            break    except redis.WatchError:        if attempt == MAX_ATTEMPTS:            raise RuntimeError("contention budget exhausted")        time.sleep(random.uniform(0.001, 0.008) * attempt)

7. redis-py pipeline behavior is connection-sensitive

redis-py executes reads directly after watch() until multi() is called; after multi(), later commands are buffered for EXEC. A pipeline used as a context manager resets itself and returns the pinned connection to the pool. That lifecycle is not cosmetic—losing the connection also loses WATCH state.

python -m pip install "redis==8.1.0"python -c "import redis; print(redis.__version__)"# Expected client version: 8.1.0

8. Redis 8.4+ can replace some WATCH loops with one conditional SET

For a single String key whose new value can be computed from an exact previously-read value, Redis 8.4 introduced SET ... IFEQ/IFNE (and digest variants). That moves the compare-and-set check into one atomic command. It does not replace WATCH for every multi-key or multi-command invariant.

SET atlasmart:ch13:l2:cas-stock 3SET atlasmart:ch13:l2:cas-stock 2 IFEQ 3# Expected: OKSET atlasmart:ch13:l2:cas-stock 1 IFEQ 3# Expected: nil/null because current value is now 2.GET atlasmart:ch13:l2:cas-stock# Expected: "2"

9. Cluster: every watched/transaction key must respect slot locality

The mandatory lab is standalone. In Redis Open Source Cluster, a transaction cannot freely span hash slots. Keep the watched keys and transaction keys in the same slot—for example with a deliberate {order:9001} tag—or redesign the invariant. Do not add hash tags mechanically: a single popular tag can concentrate traffic.

SET atlasmart:ch13:{order:9001}:stock 3SET atlasmart:ch13:{order:9001}:status pending# Both keys share the order:9001 hash tag. This standalone example does not test Cluster routing.

10. Conflict observability

Measure more than final success. Record attempts per logical operation, WATCH conflicts, exhausted retries, success/failure counts, and latency distributions. A rising p95/p99 with high conflict counts usually indicates a hot coordination key or an overly broad watched set. It is a data-model/load signal, not a reason to blindly increase retries.

11. Reproducible AtlasMart lab

Run the two-terminal conflict trace, the expiry case, and the redis-py bounded loop. Keep the dataset tiny and deterministic. The goal is correctness evidence, not maximum throughput.

MGET atlasmart:ch13:l2:stock atlasmart:ch13:l2:cas-stockEXISTS atlasmart:ch13:l2:ephemeralINFO server# Record redis_version and final values with your lab notes.

12. Failure injection and repair

The deliberately wrong case is an infinite WATCH retry loop around one hot inventory key. Under contention it can amplify load until latency and abort rates both worsen. Repair it with a finite attempt budget, jitter/backoff, a clear caller-visible failure mode, and a redesign threshold—for example partitioning independent inventory or moving bounded logic server-side in the next chapter.

13. Cleanup

UNLINK atlasmart:ch13:l2:stock atlasmart:ch13:l2:ephemeral atlasmart:ch13:l2:cas-stock atlasmart:ch13:{order:9001}:stock atlasmart:ch13:{order:9001}:status

14. Production judgment

WATCH is appropriate when client-side logic must read current Redis state and conditionally update it, and conflicts are expected to be uncommon enough that retries stay bounded. Do not use it as a distributed lock. It protects only watched Redis keys on the relevant server/slot and only until EXEC. Persistence, replication, failover, external side effects, and application authorization remain separate. If conflict rate is structurally high, reduce coordination scope or choose a server-side atomic primitive instead of creating a retry storm.

15. Summary and next step

WATCH converts stale-read risk into an explicit abort/retry path. The correctness loop is WATCH → read → validate/compute → MULTI → queue → EXEC, all on one connection. Lesson 3 now focuses on failure taxonomy so queueing errors, runtime errors, DISCARD, and WATCH aborts are never collapsed into the misleading phrase “the transaction failed.”

Authoritative references

• Redis Open Source 8.10 release notes
• Redis 8.10 command reference
• Redis transactions
• MULTI
• EXEC
• DISCARD
• WATCH
• UNWATCH
• Redis multi-key operations
• Redis pipelining
• redis-py pipelines and transactions
• redis-py documentation
• SET
• DELEX
• MSETNX
• INCR
• HINCRBY
• EVAL
• FCALL
• Redis ACLs
• Redis persistence
• Redis replication
• Redis licenses