Partitions, Rows, Clustering Columns, Column Families/Tables, and Sparse Data

Learning outcomes

AtlasMart's telemetry service has outgrown a row-per-object mental model. A device emits thousands of measurements, many measurements share the same access path, and most reads ask for a bounded slice such as “device d7 for one day, newest first.” Wide-column systems make the routing and ordering decisions explicit. This lesson builds the vocabulary before using it: a partition key chooses the logical partition and therefore the routing unit; clustering columns order rows inside that partition; a row is identified by the full primary key; and a cell/column is an attribute stored for that row. Historical systems use terms such as column family; modern Cassandra exposes a similar modeling surface through tables.

1. A wide-column table is laid out for a query path

Relational normalization asks which facts belong to which entity and relationship. Wide-column design begins one step later: which exact query must be served without scanning the cluster? The answer determines which values must be co-located. For AtlasMart telemetry, the logical query is “all samples for one tenant/device/day ordered by event time.” A natural partition key is therefore (tenant_id, device_id, day); event_time becomes the clustering column. Rows with that same partition key land together and are ordered locally by the clustering value.

The storage engine may persist that logical partition across immutable SSTables, caches, memtables, replicas, and compaction generations. “One partition” is therefore a logical routing/locality statement, not a promise that one contiguous disk extent exists forever.

2. Sparse rows are useful only when their meaning is explicit

Sensor firmware versions do not always emit the same measurements. One row may contain temperature, another only humidity. Wide-column ancestry was built to handle sparse, large logical tables efficiently. But schema flexibility still needs contracts: operators must know which cells are optional, which are required for an invariant, how missing differs from an explicit sentinel, and which indexes or derived tables depend on a cell existing.

3. Trace the request: route first, then scan a bounded ordered slice

A client asks for the newest three measurements for tenant=t1, device=d7, day=2026-08-29. The client/coordinator serializes the partition key, hashes or otherwise maps it through routing metadata, contacts the replica set that owns the partition, and reads the clustering slice in descending time order. Replication policy and consistency level decide how many replicas participate; this chapter is focusing on the data-layout side of that path rather than re-teaching the quorum semantics from Chapter 06.

from collections import defaultdict
import hashlib

nodes = ["n1", "n2", "n3", "n4"]

def owner(partition_key):
    digest = hashlib.sha256("|".join(map(str, partition_key)).encode()).digest()
    return nodes[int.from_bytes(digest[:4], "big") % len(nodes)]

events = [
    {"tenant":"t1", "device":"d7", "day":"2026-08-29", "ts":300, "temp":24.7},
    {"tenant":"t1", "device":"d7", "day":"2026-08-29", "ts":100, "temp":24.1, "humidity":41},
    {"tenant":"t1", "device":"d7", "day":"2026-08-29", "ts":200, "humidity":43},
    {"tenant":"t1", "device":"d8", "day":"2026-08-29", "ts":110, "temp":21.9},
]

# Wide-column-shaped table: partition key groups rows; clustering key orders them.
partitions = defaultdict(list)
for e in events:
    pk = (e["tenant"], e["device"], e["day"])
    partitions[pk].append(e)
for rows in partitions.values():
    rows.sort(key=lambda r: r["ts"], reverse=True)

pk = ("t1", "d7", "2026-08-29")
print("partition:", pk, "owner:", owner(pk))
print("clustering order:", [r["ts"] for r in partitions[pk]])
print("sparse cells:", [{k:v for k,v in r.items() if k not in {"tenant","device","day","ts"}} for r in partitions[pk]])

# Bad design: event_id is the partition key. A device-day query cannot route to one partition.
bad_partitions = {}
for i,e in enumerate(events):
    bad_partitions[(f"evt-{i}",)] = e
bad_touched = len(bad_partitions)
good_touched = 1
print("device-day query partitions touched: bad=", bad_touched, "good=", good_touched)

Expected evidence: the three d7 rows share one logical partition, clustering timestamps appear in descending order, cells are sparse, and a device-day query touches one partition in the query-first model. The deliberately bad event_id-as-partition-key design forces the same business query to inspect every event partition in this tiny dataset. The output proves routing consequences in the deterministic model; it does not benchmark Cassandra or establish a universal hash function.

4. Wrong approach: maximize distribution by giving every event its own partition

Unique event keys distribute beautifully but destroy the access locality AtlasMart actually needs. “More evenly distributed” is not automatically “better modeled.” If the application must discover every relevant key before it can answer a normal request, it has moved the index problem into application scatter-gather. The repair is to choose a partition key that is both selective enough to route and coarse enough to co-locate the rows normally read together, then bound its growth in the next lesson.

5. Production judgment

Use partition-local wide-column layouts when access patterns are stable enough to encode into primary keys and when rows sharing a key have a bounded, operationally acceptable lifetime. Partitioning is not replication: replicas protect copies of the same partition, while the partition key decides which data belongs together. Monitor partition-size distribution, read partitions/request, clustering-slice width, hot-key traffic, replica imbalance, compaction backlog, tombstone scans, and repair age. Tenant identity must be part of authorization as well as key design; putting tenant_id in a key does not by itself enforce isolation.

Authoritative references

• Chang et al. — Bigtable: A Distributed Storage System for Structured Data (OSDI 2006) — primary research for the wide-column lineage, ordered row keys, column families, and sparse structured data.
• Apache Cassandra 5.0 — Data definition — current implementation reference for partition keys, clustering columns, composite primary keys, and clustering order.
• Apache Cassandra — Logical data modeling — query-first table design and primary-key reasoning.
• Apache Cassandra 5.0 — Tombstones — deletion markers, grace/reconciliation, repair interaction, and resurrection risk.
• Apache Cassandra 5.0 — Compaction overview — immutable SSTables, compaction, TTL/deletes, and read/space effects.
• Apache Cassandra releases — Cassandra 5.0.9 is the latest GA release as of the August 2026 review.