BITFIELD for Packed Integers, Overflow Modes, and Memory-Efficient Counters

Learning outcomes

AtlasMart has a telemetry record containing several tiny bounded counters: quality score 0–15, retry count 0–7, temperature delta −32–31, and a few boolean flags. One Redis key per counter is easy to read but expensive in metadata/cardinality. A plain bitmap stores only booleans. BITFIELD lets one String hold packed integers, but correctness now depends on signedness, bit width, offsets, and overflow policy.

1. BITFIELD interprets selected bits as integers

BITFIELD is a String/bitmap-family command that can chain integer GET, SET, and INCRBY subcommands. An encoding starts with i for signed or u for unsigned, followed by a width in bits. Current Redis supports signed widths through 64 bits and unsigned widths through 63 bits because RESP integer replies cannot represent a full unsigned 64-bit range.

docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app UNLINK atlasmart:ch03:bf:recorddocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app BITFIELD atlasmart:ch03:bf:record SET u8 '#0' 12 SET i8 '#1' -5docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app BITFIELD atlasmart:ch03:bf:record GET u8 '#0' GET i8 '#1'docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app STRLEN atlasmart:ch03:bf:record# First SET values return previous values (0,0 on a new zero-filled String).# GET returns 12 and -5; #0/#1 mean field indexes, so length is 2 bytes.

2. Raw bit offsets and #field indexes are different

A plain numeric offset is a zero-based bit offset. Prefixing the offset with # multiplies it by the current encoding width. Therefore GET u8 #1 reads bits 8–15, while GET u8 1 reads an 8-bit window beginning at bit 1. Both are valid commands and can produce very different values.

docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app BITFIELD atlasmart:ch03:bf:record GET u8 '#0' GET u8 '#1'docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app BITFIELD atlasmart:ch03:bf:record GET u8 0 GET u8 1# # syntax indexes fields; bare offsets index bits.

BITFIELD numbers bit 0 as the most significant bit of the first byte. Byte-aligned fields therefore resemble big-endian bit order, while non-aligned fields require careful schema documentation. Persist the schema version outside tribal knowledge—for example in the key prefix or adjacent metadata—because the bytes do not self-describe their field widths.

3. One BITFIELD command can chain operations in order

Redis applies subcommands in the order given and returns one array entry per subcommand. Because the whole BITFIELD invocation is one Redis command, another client does not interleave a command halfway through it. This can implement compact multi-field updates, but longer command payloads still consume server execution time and should remain bounded.

docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app BITFIELD atlasmart:ch03:bf:record GET u8 '#0' INCRBY u8 '#0' 3 GET u8 '#0'# Expected array shape: old 12, new 15, then current 15.

Do not confuse this with SQL rollback semantics. Choose encodings and overflow policy so each packed update has a defined result; validate application invariants before sending the command.

4. Overflow policy is part of the data model

The default overflow mode is WRAP. SAT clamps at the encoding's minimum/maximum. FAIL performs no write for the overflowing subcommand and returns null for that result. An OVERFLOW directive affects following SET/INCRBY subcommands until another overflow directive changes the mode.

docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app UNLINK atlasmart:ch03:bf:overflowdocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app BITFIELD atlasmart:ch03:bf:overflow SET u8 '#0' 250# Default WRAP: 250 + 10 -> 4 modulo 256.docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app BITFIELD atlasmart:ch03:bf:overflow INCRBY u8 '#0' 10 GET u8 '#0'# Reset, then SAT: clamp at 255.docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app BITFIELD atlasmart:ch03:bf:overflow SET u8 '#0' 250 OVERFLOW SAT INCRBY u8 '#0' 10 GET u8 '#0'# Reset, then FAIL: increment returns nil and stored value remains 250.docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app BITFIELD atlasmart:ch03:bf:overflow SET u8 '#0' 250 OVERFLOW FAIL INCRBY u8 '#0' 10 GET u8 '#0'

For signed i8, WRAP turns 127 + 1 into −128. SAT keeps it at 127. FAIL leaves 127 unchanged and returns null for the overflowing increment. Pick the mode from domain semantics: wrap is useful for modular arithmetic, saturation for bounded meters, and fail for invariants that must reject overflow.

5. Deliberately wrong schema: signedness changes the same bits

The bit pattern 11111111 is 255 as u8 and −1 as i8. Redis cannot infer which meaning AtlasMart intended. A client that writes u8 and a later client that reads i8 can disagree without any Redis error.

docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app UNLINK atlasmart:ch03:wrong:signednessdocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app BITFIELD atlasmart:ch03:wrong:signedness SET u8 '#0' 255docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app BITFIELD atlasmart:ch03:wrong:signedness GET u8 '#0' GET i8 '#0'# Expected: 255 and -1 from the same eight bits.

The repair is schema governance: define field order, signedness, width, unit, scale, overflow policy, and schema version. Add deterministic encode/decode tests in every client language. If the packed schema evolves incompatibly, write a new key version and migrate/dual-read deliberately rather than reinterpreting old bytes in place.

6. Packed counters save metadata but concentrate contention

One String containing 1,000 u8 counters needs roughly 1,000 payload bytes plus one key/object's metadata. One thousand independent String keys need at least 1,000 key/object metadata structures before their tiny values. Packing can therefore be dramatically more memory-efficient. But all 1,000 counters now live in one key, share one TTL, one Cluster slot, one persistence/replication unit, and one hot-key contention point.

7. Hands-on lab: versioned warehouse telemetry record

Pack four fields into a two-byte schema: u4 quality at bits 0–3, u4 retries at bits 4–7, i8 temp_delta at bits 8–15. Use SAT for retries so a counter cannot wrap back to zero and mislead operators. The key name contains v1 so future incompatible layouts can coexist during migration.

docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app UNLINK atlasmart:ch03:l5:telemetry:v1:warehouse-7# quality=12, retries=2, temp_delta=-7docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app BITFIELD atlasmart:ch03:l5:telemetry:v1:warehouse-7 SET u4 0 12 SET u4 4 2 SET i8 8 -7docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app BITFIELD atlasmart:ch03:l5:telemetry:v1:warehouse-7 GET u4 0 GET u4 4 GET i8 8# Increment retries with saturation at 15.docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app BITFIELD atlasmart:ch03:l5:telemetry:v1:warehouse-7 OVERFLOW SAT INCRBY u4 4 1 GET u4 4docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app STRLEN atlasmart:ch03:l5:telemetry:v1:warehouse-7docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin MEMORY USAGE atlasmart:ch03:l5:telemetry:v1:warehouse-7docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app TTL atlasmart:ch03:l5:telemetry:v1:warehouse-7# Expected: initial fields 12,2,-7; retries becomes 3; STRLEN is 2 bytes; TTL -1 unless policy adds one.# Cleanup:docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app UNLINK atlasmart:ch03:l5:telemetry:v1:warehouse-7 atlasmart:ch03:bf:record atlasmart:ch03:bf:overflow atlasmart:ch03:wrong:signedness

Verification checklist:

• You can calculate each field bit range and distinguish bare bit offsets from #field indexes.
• You can explain u8 versus i8 interpretation and the signed/unsigned width limits in the current Redis command.
• You observed WRAP, SAT, and FAIL rather than treating overflow as a universal behavior.
• You can state why packing reduces key metadata but creates a shared TTL, hot key, one Cluster slot, and schema-tooling requirement.
• You versioned the packed schema and defined a migration/rollback path instead of silently reinterpreting old bytes.

8. Production judgment

BITFIELD fits compact, bounded numeric fields that are updated/read frequently and whose shared lifecycle/locality is acceptable. Avoid it when fields need independent TTLs, rich query/indexing, frequent schema evolution, or human inspection. Bound packed-key size and update rate; measure MEMORY USAGE, command latency distributions, key hotness, replication/AOF traffic, fork/copy-on-write effects, and migration cost. Far offsets have the same allocation risk as SETBIT.

Choose overflow behavior explicitly and test boundary values in every client. Cluster places the packed record in one slot, which is useful for atomic locality but can create shard skew. ACLs can restrict access to the key pattern but cannot enforce field-level authorization inside a packed String. Backups and restore tests must preserve the bytes and the application schema version needed to interpret them.

9. Summary and next step

BITFIELD views slices of a String as signed or unsigned integers. Width, bit offset, # indexing, bit order, and overflow mode are part of the schema. WRAP is default; SAT clamps; FAIL rejects the overflowing subcommand. Packing reduces metadata and can be extremely memory-efficient, but concentrates lifecycle, routing, contention, and schema complexity into one key. Chapter 04 now moves from byte-packed records to Redis Hashes, where fields become named and object-like rather than positional bits.

Authoritative references

• BITFIELD command — encodings, offsets, overflow modes, bit order, and performance notes
• BITFIELD_RO command — read-only packed-field access
• SETBIT command — shared String allocation/offset constraints
• Redis Bitmaps — bit-oriented String mental model
• Redis 8.10 command reference — target-version command surface