Design a Key Lifecycle Policy for Retention, Privacy, Cache Freshness, and Capacity

Learning outcomes

AtlasMart now knows how to name keys, attach and inspect deadlines, scan incrementally, inspect types/memory, move values, and delete without blindly blocking the foreground server. The remaining problem is governance: different key families have different reasons to exist. A cart may expire from inactivity, a cache entry may expire because its source changes, an idempotency key may need a bounded replay window, a business order may not belong in Redis as ephemeral truth at all, and privacy deletion may require action beyond Redis. A lifecycle policy turns those distinctions into testable rules.

1. Start with why the key exists

A lifecycle policy should begin with the key’s role, not a default TTL. An authoritative key is treated by the application as the source of truth for some decision; a derived key can be reconstructed from another source; a cache is a derived copy intended to reduce latency/load. The more authoritative and difficult to reconstruct a value is, the less acceptable silent eviction or accidental expiry becomes.

Also classify freshness (how stale can the value be?), retention (how long should it remain accessible?), privacy sensitivity (what deletion/backup obligations apply?), and cardinality (how many keys can exist at once?). These dimensions can point to different mechanisms. A 5-minute cache TTL is freshness policy; a legal 30-day retention rule is governance; maxmemory eviction is pressure response; none is automatically interchangeable.

2. Retention, freshness, privacy, and capacity need separate controls

Retention answers “how long should this state remain available?” Freshness answers “how old may a derived answer be?” Privacy lifecycle answers “where can personal data remain and how is deletion proven?” Capacity answers “how much memory, key metadata, persistence growth, replication bandwidth, and operational work can this family consume?” A mature policy records each independently.

TTL can enforce a Redis-side deadline. Explicit deletion can remove state earlier. Maxmemory/eviction, introduced in Chapter 18, may remove eligible keys under memory pressure according to a configured policy; it is not a substitute for a business retention timer. Persistence (RDB/AOF) affects restart recovery, not an off-host backup retention policy. Replication, Sentinel, and Cluster affect availability/topology, not whether the application is legally allowed to keep a copy.

3. Cardinality budgets connect lifecycle to memory

A simple first-order estimate for a key family is peak creation rate × maximum retention window, adjusted for retries, failure backlog, fan-out, and skew. This estimates key count, not bytes. Multiply by measured representative memory distributions and add non-key memory headroom to reason about capacity. Because values vary, use percentiles or buckets rather than one “average key size” when the tail matters.

For example, if AtlasMart creates 50,000 idempotency tokens per hour and retains them 24 hours, the steady-state order of magnitude is 1.2 million live tokens before retry/failure effects. That calculation is not a Redis tuning recommendation; it is a reason to test whether one-key-per-token is a suitable model, how much memory/persistence it consumes, and whether a different structure or authoritative store fits better.

docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin DBSIZEdocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin INFO keyspacedocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 sh -lc 'redis-cli --user academy-admin INFO memory | grep -E "used_memory_human|maxmemory_human|maxmemory_policy"'# Interpret these as current server evidence.# They do not identify per-prefix ownership or future cardinality by themselves.

4. Build the policy around state transitions, not only final TTL

For each family, document creation, mutation, refresh, expiration, deletion, migration, and failure behavior. If a session refresh uses SET, require KEEPTTL or a deliberate new TTL; if an idempotency record must never be extended after first creation, use conditional expiry semantics that reject extension; if a cache may be proactively invalidated, define explicit deletion and re-fetch behavior. The policy should say what happens when the key is already missing or unexpectedly persistent.

5. Secondary structures can outlive or amplify primary-key policy

Later AtlasMart chapters add Redis Search indexes, JSON documents, vectors, time-series keys, probabilistic structures, Streams, and application-maintained indexes. A lifecycle policy must include them. Deleting a source key may or may not have the same effect on a separately maintained application index, external search system, analytics export, or vector embedding store. Secondary structures also consume memory and can amplify write/deletion cost.

For integrated Redis Search/JSON features, verify current feature behavior on the exact Redis version rather than assuming historical module packaging. For any external index, make source deletion a reconciliation event and test orphan detection. Authorization must be evaluated at query time; hiding a tenant identifier in a key prefix or search filter is not a security control by itself.

6. Deliberately wrong policy: “everything gets 24 hours, tenant per DB”

A universal AtlasMart rule—“every key expires after 24 hours, each tenant uses a different logical database, and maxmemory eviction can clean up the rest”—looks simple but collapses unrelated requirements. Durable business state may disappear; a reset token may persist much longer than intended; a cache may be unnecessarily stale; idempotency coverage may be too short; logical databases do not enforce tenant security and do not map to Cluster; eviction under pressure can violate assumptions before TTL.

The repair is classification plus enforcement. Separate authoritative and derived state. Give each family an owner, naming grammar, cardinality budget, freshness/retention policy, ACL/application authorization model, persistence/recovery requirement, and deletion strategy. Validate missing/persistent states as errors when policy requires expiry. Monitor actual memory/cardinality/expiry behavior and revise from evidence.

7. Hands-on capstone: create and audit a lifecycle matrix

Create four bounded policy fixtures representing a session, cache entry, idempotency key, and durable-looking order snapshot. The last one is intentionally persistent so the audit can flag that “persistent” is a policy decision rather than a Redis default to ignore.

docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app SET atlasmart:ch02:policy:session:1 "customer=431" EX 300docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app SET atlasmart:ch02:policy:cache:product:7 "price=39.50" EX 120docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app SET atlasmart:ch02:policy:idempotency:req-abc "done" EX 600docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app SET atlasmart:ch02:policy:order:9001 "paid"# Audit representative lifecycle state.docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin SCAN 0 MATCH 'atlasmart:ch02:policy:*' COUNT 50docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin TTL atlasmart:ch02:policy:session:1docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin TTL atlasmart:ch02:policy:cache:product:7docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin TTL atlasmart:ch02:policy:idempotency:req-abcdocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin TTL atlasmart:ch02:policy:order:9001# The order snapshot reports -1: persistent. Decide whether that is acceptable; do not "fix" it automatically.

docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 sh -lc 'for key in   atlasmart:ch02:policy:session:1   atlasmart:ch02:policy:cache:product:7   atlasmart:ch02:policy:idempotency:req-abc   atlasmart:ch02:policy:order:9001; do   echo "=== $key ===";   redis-cli --user academy-admin TYPE "$key";   redis-cli --user academy-admin PTTL "$key";   redis-cli --user academy-admin MEMORY USAGE "$key"; done'# Record values with timestamp and server version; this is an audit sample, not a snapshot.

The persistent order fixture should trigger a design discussion: is Redis merely a projection of an authoritative order database, or is someone accidentally making Redis the sole durable truth? The correct answer depends on architecture, not on making TTL nonnegative.

Verification checklist:

• Every key family has an owner, purpose, naming grammar, authority/recomputability classification, and cardinality budget.
• Freshness TTL, business retention, privacy deletion, maxmemory eviction, persistence, and backup retention are documented as separate mechanisms.
• Update commands have an explicit TTL-preserve/extend/shorten policy and tests for -1/-2 sentinel states.
• Inventory jobs use SCAN or maintained indexes and are duplicate-safe; exact business inventory is not inferred from SCAN snapshot semantics.
• Deletion/migration policy includes type/value/TTL verification, collision behavior, big-key cost, topology, rollback, and secondary-index reconciliation.

docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app UNLINK \  atlasmart:ch02:policy:session:1 atlasmart:ch02:policy:cache:product:7 \  atlasmart:ch02:policy:idempotency:req-abc atlasmart:ch02:policy:order:9001

8. Production lifecycle review checklist

Before approving a Redis key family, review the complete path. The following list is an acceptance checklist, not a universal configuration recipe.

• Latency: command complexity, big-key behavior, synchronized expiration waves, scan/maintenance load, p50/p95/p99 and timeout budgets.
• Memory/cardinality: key creation rate, retention, value-size distribution, internal encodings, allocator/process overhead, future Search/index/vector/time-series/probabilistic memory.
• Durability: whether RDB/AOF recovery is required, tolerated data-loss window, off-host backup and tested restore; persistence is not backup.
• Availability/topology: standalone, asynchronous replication, Sentinel failover, or Cluster slot distribution; expiry/scan/copy semantics are tested in the chosen topology.
• Security: ACL user/key patterns, TLS/network boundaries, application authorization, secrets, tenant model, and sensitive identifiers in key names.
• Clients: exact library version, retry/timeout/idempotency behavior, connection pooling, protocol mode, and handling of missing/persistent TTL states.
• Operations: observability signals, failure injection, on-call runbook, cleanup safety, migration/rollback, version/license/platform constraints, and cost.

No single TTL or key length can satisfy those dimensions. The correct policy is a testable system contract tied to the actual workload and current Redis release.

9. Summary and bridge to Chapter 03

Chapter 02 turned Redis’s generic keyspace into an explicit lifecycle model. Namespaces help humans but do not enforce security; hash tags are Cluster placement hints; TTL is deadline metadata, not an exact scheduler; SCAN is incremental and duplicate-tolerant rather than a snapshot; introspection and migration commands have real memory/compatibility costs; UNLINK changes where reclamation work happens. With those boundaries explicit, Chapter 03 can safely focus on what lives behind a key: binary-safe Strings, counters, bitmaps, bitfields, and their numeric/memory semantics.

Authoritative references

• Redis keyspace documentation — key lifecycle and logical databases
• EXPIRE command — deadline semantics and conditional expiry
• SCAN command — incremental inventory semantics
• MEMORY USAGE command — representative per-key memory evidence
• Redis persistence — RDB/AOF durability boundaries
• Redis eviction — maxmemory and eviction-policy semantics
• Redis security / ACL — authorization boundaries