Big Data AcademyQuery · move · process · govern

Build leaderboards, time windows, delayed-work indexes, and time-ordered secondary paths while exposing cleanup and reliability boundaries.

ZINCRBY, Leaderboards, Sliding Windows, Delayed Work, and Time-Ordered Indexes

Choose hashes, Redis JSON, or many keys from structure, update/query patterns, expiration granularity, memory/cardinality, indexing, ACL boundaries, and migration evidence.

Intermediate150–180 minutesLeaderboard/window/scheduler labRedis Open Source 8.10.1Free/local-firstLast reviewed: September 6, 2026

Learning outcomes

AtlasMart now wants four behaviors from the same underlying idea: increment a leaderboard, keep only requests from the last 60 seconds, find jobs whose due time has arrived, and query records in timestamp order. A sorted set can model each when the score is chosen deliberately, but none of these patterns is automatically a complete queue, scheduler, or rate limiter.

01

Use ZINCRBY while preserving the distinction between relative score mutation and authoritative counters.

02

Build a bounded sliding-window fixture with timestamp scores and explicit cleanup.

03

Model delayed work and time-ordered indexes with due/created timestamps.

04

Explain duplicate-score/timestamp collisions, cleanup cost, and atomicity gaps in multi-command patterns.

05

Identify hot-key, precision, persistence, retry, and Cluster risks before production use.

Exact lab baseline

All Chapter 06 mandatory labs reuse the disposable Chapter 01 environment: Redis Open Source 8.10.1 from Docker Official Image redis:8.10.1, container atlasmart-redis-ch01, standalone topology, host publication 127.0.0.1:6379, TLS disabled only because traffic stays on loopback, default ACL user disabled, named ACL users atlasmart-app and academy-admin, logical database 0, AOF with appendfsync everysec plus RDB snapshots, persistent /data volume, and no explicit Redis maxmemory limit or eviction policy. The primary interface is the redis-cli shipped in the same pinned image. Mandatory examples use only bounded synthetic keys under atlasmart:ch06:*.

1. ZINCRBY is an atomic score update

ZINCRBY key increment member atomically adds a floating increment to the member's score, creating the member if absent. Atomic command execution prevents a lost update that would occur with client-side ZSCORE → add → ZADD.

redis-cli · leaderboard increments 
docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app DEL atlasmart:ch06:sales-rankdocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZINCRBY atlasmart:ch06:sales-rank 5 seller:anadocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZINCRBY atlasmart:ch06:sales-rank 8 seller:bodocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZINCRBY atlasmart:ch06:sales-rank 4 seller:anadocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZRANGE atlasmart:ch06:sales-rank 0 -1 REV WITHSCORES

One command is atomic; the surrounding business workflow is not. If awarding points must be exactly coordinated with an order ledger, reconcile against authoritative order IDs rather than assuming a score increment is the transaction.

2. Sliding-window mental model: event time is the score

For an exact sliding window, each request can be a unique member and its timestamp the score. Before counting, remove scores older than the window boundary. Then add the current request and count the remaining members. This makes retention explicit instead of letting old entries accumulate forever.

redis-cli · deterministic 60-second window fixture 
docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app DEL atlasmart:ch06:window:user42docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZADD atlasmart:ch06:window:user42 100000 req:1 120000 req:2 155000 req:3 161000 req:4docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZREMRANGEBYSCORE atlasmart:ch06:window:user42 -inf 101000docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZCOUNT atlasmart:ch06:window:user42 101001 161000docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZRANGE atlasmart:ch06:window:user42 0 -1 WITHSCORES

This deterministic fixture treats numbers as milliseconds. A live implementation must define clock source, boundary inclusivity, request identity, retry behavior, and the exact atomicity requirement. Chapter 23 revisits production rate limiting.

3. Cleanup is part of the algorithm, not housekeeping

Without ZREMRANGEBYSCORE, the window key grows with lifetime traffic. Cleanup is O(log N + M) where M is the number removed. That means bursty catch-up cleanup can itself become expensive.

redis-cli · observe cleanup cardinality 
docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZCARD atlasmart:ch06:window:user42docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZREMRANGEBYSCORE atlasmart:ch06:window:user42 -inf 150000docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZCARD atlasmart:ch06:window:user42docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app MEMORY USAGE atlasmart:ch06:window:user42

Measure both steady-state cardinality and worst cleanup bursts. A “sliding window” without a retention policy is an unbounded time index.

4. Delayed work: due time as score

A delayed-work index stores job IDs as members and due timestamps as scores. A worker queries the smallest due scores up to “now.” This discovers due work efficiently, but a plain read does not claim ownership and a pop/remove can lose work if processing fails.

redis-cli · due-work discovery 
docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app DEL atlasmart:ch06:duedocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZADD atlasmart:ch06:due 1700000000000 job:a 1700000005000 job:b 1700000010000 job:cdocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZRANGE atlasmart:ch06:due -inf 1700000006000 BYSCORE LIMIT 0 10 WITHSCORESdocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZCOUNT atlasmart:ch06:due -inf 1700000006000

The query identifies due members. It does not provide acknowledgment, retry history, consumer ownership, or exactly-once processing. Those are separate messaging/workflow concerns.

5. Wrong approach: ZPOPMIN as a complete reliable scheduler

ZPOPMIN atomically removes the lowest-score member, which is useful when removal itself is the desired state transition. But if a worker pops a due job and crashes before completing the business action, Redis no longer contains that job in the sorted set.

redis-cli · bounded loss-window demonstration 
docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app DEL atlasmart:ch06:unsafe-schedulerdocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZADD atlasmart:ch06:unsafe-scheduler 1 job:lost-demodocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZPOPMIN atlasmart:ch06:unsafe-scheduler 1docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZCARD atlasmart:ch06:unsafe-scheduler

ZCARD = 0 proves removal, not completion. Repair with an explicit processing/ownership state machine, idempotent job handling, or a structure/system whose delivery semantics match the requirement.

6. Time-ordered secondary access path

A sorted set can index an authoritative object by timestamp: member = object ID, score = creation/update time. The sorted set is then a secondary access path, not the record itself. Updates must keep the index and primary data synchronized according to an explicit consistency contract.

redis-cli · secondary time index 
docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app DEL atlasmart:ch06:orders-by-timedocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZADD atlasmart:ch06:orders-by-time 1700001000000 order:7001 1700001002000 order:7002 1700001002000 order:7003docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZRANGE atlasmart:ch06:orders-by-time 1700001000000 1700001002000 BYSCORE WITHSCORESdocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZREVRANK atlasmart:ch06:orders-by-time order:7003 WITHSCORE

Ties at the same millisecond are ordered by member bytes. If the source order changes or an object is deleted, the secondary index needs reconciliation or explicit transactional/function logic.

7. Timestamp precision and collision design

Milliseconds and contemporary microseconds fit inside the exact ±2^53 integer region of Redis scores; nanosecond epoch timestamps do not. Exact representability does not imply uniqueness: multiple events can share a millisecond or microsecond. Use unique members, a tie-break strategy, or a Stream when append/history semantics—not merely time ordering—are the real requirement.

8. Multi-command windows have race boundaries

A rate-limit sequence such as remove-old → add-current → count is three separate atomic commands, not one atomic algorithm. Concurrent clients can interleave. Later chapters cover MULTI/EXEC and Functions; Chapter 23 applies those tools to production rate-limit patterns. Here, the important lesson is to identify the race rather than silently calling the pattern atomic.

9. Hot-key and retention model

Pattern Growth driver Required control Typical risk
Leaderboard number of ranked members scope/shard/archive policy global write hot key
Sliding window events inside retention horizon continuous score cleanup burst cleanup + cardinality
Delayed work scheduled outstanding jobs claim/completion cleanup loss/duplicate processing
Time index indexed object population delete/update reconciliation stale secondary index

10. Reproducible combined lab

Exercise score mutation, due-time reads, and explicit cleanup with small deterministic values.

redis-cli · combined chapter fixture 
docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app DEL atlasmart:ch06:lab:rank atlasmart:ch06:lab:timedocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZINCRBY atlasmart:ch06:lab:rank 10 user:adocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZINCRBY atlasmart:ch06:lab:rank 15 user:bdocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZINCRBY atlasmart:ch06:lab:rank 8 user:adocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZRANGE atlasmart:ch06:lab:rank 0 1 REV WITHSCORESdocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZADD atlasmart:ch06:lab:time 100 e:1 200 e:2 300 e:3 400 e:4docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZREMRANGEBYSCORE atlasmart:ch06:lab:time -inf 150docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app ZRANGE atlasmart:ch06:lab:time 151 350 BYSCORE WITHSCORES

Verification: user:a reaches 18; user:b remains 15; event e:1 is removed; events e:2 and e:3 remain in the queried interval.

11. Production judgment

Use Sorted Sets when numeric ordering plus point membership is the core primitive. Rate limiting and scheduling require more than ordering: define atomicity, retry/idempotency, clock assumptions, ownership, retention, and failure recovery. Persisted/replicated sorted-set state still does not guarantee external work completion. Monitor cardinality, cleanup removals, hot-key commands/sec, range-result sizes, memory, and latency distributions.

12. Summary and next step

ZINCRBY supports atomic score mutation; timestamp scores support windows, due-work discovery, and time indexes; cleanup and reliability remain application responsibilities. Next, we derive rankings across multiple sorted sets with weighted union, intersection, and difference.

Check your understanding

  1. Why is ZINCRBY safer than client-side read/add/write for one score?
  2. What happens to a sliding-window key if old scores are never removed?
  3. Does ZRANGE due work claim ownership?
  4. Why can ZPOPMIN lose work?
  5. Are remove-old/add/count automatically one atomic rate-limit decision?
Review the answers

ZINCRBY performs the score increment atomically on the server.

It grows with lifetime events instead of the intended retention horizon.

No. It only reads matching members.

The member is removed before external processing succeeds, so a worker crash can make it disappear.

No. They are separate commands unless combined with an explicit atomic mechanism.

Authoritative references

Keep knowledge open

Help the academy stay free and grow.

If these tutorials save you time, a small donation supports new lessons, technical review, diagrams, examples, and long-term maintenance.

ETHEthereum / ERC-20 only
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0

Send only Ethereum or ERC-20 compatible assets to this address.