used_memory vs RSS, Allocator Overhead, Fragmentation, Buffers, and Replication/AOF Memory

Learning outcomes

AtlasMart sees a container at 70% of its memory limit while Redis reports a much smaller dataset. The discrepancy is not automatically a leak: Redis exposes several layers of memory that answer different questions. This lesson builds the vocabulary before any eviction policy is changed.

1. Four numbers, four different questions

used_memory is Redis allocator-accounted memory. used_memory_dataset estimates the dataset portion after Redis overhead. allocator_active includes active allocator pages and therefore external fragmentation. used_memory_rss is resident memory reported by the operating system for the Redis process. RSS can stay elevated after keys are deleted because the allocator may retain pages for reuse instead of immediately returning them to the OS.

2. Start a bounded node and take a baseline

docker volume create atlasmart-redis-ch18-memory_datadocker run --rm -v atlasmart-redis-ch18-memory_data:/data redis:8.10.1 sh -lc 'printf "%s\n" "user default off" "user academy-admin on >AtlasMart-Admin-Lab-Only-2026 ~* &* +@all" > /data/users.acl'docker run -d --name atlasmart-redis-ch18-memory --memory 192m -p 127.0.0.1:6391:6379 -v atlasmart-redis-ch18-memory_data:/data redis:8.10.1 redis-server --dir /data --aclfile /data/users.acl --appendonly yes --appendfsync everysec --save '' --maxmemory 0 --maxmemory-policy noevictiondocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch18-memory redis-cli --user academy-admin PINGdocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch18-memory redis-cli --user academy-admin ACL WHOAMI

docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch18-memory redis-cli --user academy-admin INFO memorydocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch18-memory redis-cli --user academy-admin MEMORY STATSdocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch18-memory redis-cli --user academy-admin MEMORY DOCTORdocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch18-memory redis-cli --user academy-admin CONFIG GET maxmemory maxmemory-policy appendonly appendfsync

Record the actual allocator name. MEMORY PURGE and allocator internals are jemalloc-specific; on another allocator, a purge can be a benign no-op.

3. Grow logical data and observe allocator/process growth

The payload is deterministic and bounded: 1,200 keys × 8 KiB is under 10 MiB of raw payload before key/object/allocator overhead. Do not infer an exact overhead percentage from one run; the encoding, allocator, architecture, and Redis version affect it.

import os, redisHOST = "127.0.0.1"PORT = 6391r = redis.Redis(host=HOST, port=PORT, username="academy-admin", password="AtlasMart-Admin-Lab-Only-2026", decode_responses=False)assert r.ping()payload = b"x" * 8192pipe = r.pipeline(transaction=False)for i in range(1200):    pipe.set(f"atlasmart:ch18:l1:blob:{i}", payload)pipe.execute()m = r.info("memory")for k in ("used_memory","used_memory_dataset","used_memory_rss","allocator_allocated","allocator_active","allocator_resident","mem_fragmentation_bytes","mem_not_counted_for_evict","mem_aof_buffer"):    print(k, m.get(k))

docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch18-memory redis-cli --user academy-admin DBSIZEdocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch18-memory redis-cli --user academy-admin MEMORY USAGE atlasmart:ch18:l1:blob:0docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch18-memory redis-cli --user academy-admin INFO memorydocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch18-memory redis-cli --user academy-admin MEMORY STATS

4. Freeing logical data does not promise an immediate RSS drop

UNLINK removes a key from the keyspace immediately and lets background threads reclaim the value later. Even after Redis frees those allocations, jemalloc may keep pages resident for reuse. This is why “DBSIZE fell but RSS did not” is not sufficient evidence of a leak.

import os, redisHOST = "127.0.0.1"PORT = 6391r = redis.Redis(host=HOST, port=PORT, username="academy-admin", password="AtlasMart-Admin-Lab-Only-2026", decode_responses=False)assert r.ping()keys = [f"atlasmart:ch18:l1:blob:{i}" for i in range(0, 1200, 2)]for start in range(0, len(keys), 100):    r.unlink(*keys[start:start+100])print("remaining", r.dbsize())print("memory", {k:r.info("memory").get(k) for k in ("used_memory","used_memory_rss","lazyfree_pending_objects","lazyfreed_objects")})

docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch18-memory redis-cli --user academy-admin INFO memorydocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch18-memory redis-cli --user academy-admin MEMORY DOCTORdocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch18-memory redis-cli --user academy-admin MEMORY PURGEdocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch18-memory redis-cli --user academy-admin INFO memory# MEMORY PURGE asks the allocator to return reclaimable pages; it is not a guarantee that RSS reaches used_memory.

5. maxmemory intentionally excludes some transient buffers

Redis documents mem_not_counted_for_evict as memory, principally transient replica and AOF buffers, that is not counted when deciding whether key eviction is required. The reason is to avoid an eviction feedback loop: evicting a key itself generates replication/AOF traffic. The consequence is operationally important—maxmemory cannot equal the container or host RAM limit.

docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch18-memory redis-cli --user academy-admin INFO memorydocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch18-memory redis-cli --user academy-admin INFO persistencedocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch18-memory redis-cli --user academy-admin INFO replication

6. Redis 8.10 adds another memory dimension: compact-hash templates

Redis 8.10 adds compact hashes and reports used_memory_hash_templates in INFO MEMORY. MEMORY USAGE for a compact-hash key includes that key's proportional share of a shared template. This is a useful reminder that per-key memory accounting can include shared structures and that a memory model frozen on an older release can become incomplete.

7. Deliberately wrong model: “set maxmemory to container RAM”

Suppose AtlasMart allocates a 2 GiB container and sets maxmemory 2gb. That leaves no room for excluded buffers, RSS/allocator slack, persistence or replication COW, client buffers, executable/library pages, or other process overhead. The kernel/container runtime—not Redis eviction—may become the final limiter.

8. Verification checklist

• Capture used_memory, used_memory_dataset, used_memory_rss, allocator metrics, and mem_not_counted_for_evict together.
• Verify RSS may remain above logical allocation after deletes without claiming a leak.
• Confirm AOF/replication buffers are budgeted separately from maxmemory.
• Record the allocator before relying on jemalloc-specific diagnostics.
• Never infer a production headroom percentage from this tiny fixture.

Production judgment and next bridge

Monitor memory as a vector of signals, not one percentage. Alert on available host/container memory, RSS, allocator fragmentation bytes, excluded buffers, client buffers, fork/COW peaks, and the workload-specific index footprint. Lesson 2 now turns that measured envelope into an explicit maxmemory and eviction-policy choice.

Summary and next step

used_memory vs RSS, Allocator Overhead, Fragmentation, Buffers, and Replication/AOF Memory is now connected to observable Redis behavior, bounded failure cases, and production tradeoffs. Keep the evidence and cleanup state from this lesson; next, continue with maxmemory and Eviction Policies: noeviction, LRU, LFU, Random, and TTL-Aware Choices.

Authoritative references

• Redis key eviction
• Redis memory optimization
• INFO
• MEMORY STATS
• MEMORY USAGE
• MEMORY DOCTOR
• MEMORY PURGE
• MEMORY MALLOC-STATS
• OBJECT FREQ
• OBJECT IDLETIME
• UNLINK
• Redis CLI keyspace analysis
• Redis latency monitoring
• Redis persistence
• Redis replication
• Redis Sentinel
• Redis Cluster specification
• Redis Search and query
• Redis 8.10 release notes
• Redis 8.10 whats new
• Redis ACLs
• Redis security
• Redis licenses
• Docker Official Redis image