Duplicate Data Across Tables and Keep It Consistent with Application / Streaming Workflows

Learning outcomes

Denormalized query tables are useful only if AtlasMart can explain who keeps them aligned. Cassandra does not enforce referential integrity across tables, so a product name or order status duplicated into three projections can drift after timeout, process crash or partial deployment. The consistency mechanism belongs to the application/data pipeline and must be observable and replayable.

1. RF/CL consistency is not cross-table projection consistency

With RF=3 and QUORUM, Cassandra can successfully replicate an update to products_by_id while products_by_category never receives the corresponding logical projection update. RF and consistency level govern replicas of a partitioned table write; they do not create a multi-table business invariant.

AtlasMart therefore attaches an application-owned source_version to every product projection. A business event such as product-version=42 is applied idempotently to each read shape. A reconciler can detect a projection at version 41, rebuild the expected row and safely replay version 42.

2. Three common ownership patterns

The chapter does not require Kafka, a proprietary managed stream or Cassandra CDC. The mandatory lab simulates the same state machine locally with explicit CQL so the invariant is visible. Production streaming choices must be evaluated separately.

3. Reproducible partial-failure and reconciliation lab

# Reuse the course cluster if it exists; otherwise create only these disposable resources.docker network inspect atlasmart-cassandra >/dev/null 2>&1 || docker network create atlasmart-cassandradocker volume create atlasmart-cass-1-datadocker volume create atlasmart-cass-2-datadocker volume create atlasmart-cass-3-data# Start node 1 only if it does not already exist.docker inspect atlasmart-cass-1 >/dev/null 2>&1 || docker run -d --name atlasmart-cass-1 --hostname atlasmart-cass-1 --network atlasmart-cassandra -e CASSANDRA_CLUSTER_NAME=atlasmart-course -e CASSANDRA_DC=dc1 -e CASSANDRA_RACK=rack1 -e CASSANDRA_ENDPOINT_SNITCH=GossipingPropertyFileSnitch -e CASSANDRA_NUM_TOKENS=16 -v atlasmart-cass-1-data:/var/lib/cassandra cassandra:5.0.9# Wait until node 1 answers before starting peers:docker exec atlasmart-cass-1 nodetool statusdocker inspect atlasmart-cass-2 >/dev/null 2>&1 || docker run -d --name atlasmart-cass-2 --hostname atlasmart-cass-2 --network atlasmart-cassandra -e CASSANDRA_CLUSTER_NAME=atlasmart-course -e CASSANDRA_DC=dc1 -e CASSANDRA_RACK=rack2 -e CASSANDRA_ENDPOINT_SNITCH=GossipingPropertyFileSnitch -e CASSANDRA_NUM_TOKENS=16 -e CASSANDRA_SEEDS=atlasmart-cass-1 -v atlasmart-cass-2-data:/var/lib/cassandra cassandra:5.0.9docker inspect atlasmart-cass-3 >/dev/null 2>&1 || docker run -d --name atlasmart-cass-3 --hostname atlasmart-cass-3 --network atlasmart-cassandra -e CASSANDRA_CLUSTER_NAME=atlasmart-course -e CASSANDRA_DC=dc1 -e CASSANDRA_RACK=rack3 -e CASSANDRA_ENDPOINT_SNITCH=GossipingPropertyFileSnitch -e CASSANDRA_NUM_TOKENS=16 -e CASSANDRA_SEEDS=atlasmart-cass-1 -v atlasmart-cass-3-data:/var/lib/cassandra cassandra:5.0.9# Wait until all three nodes report UN before creating RF=3 data.docker exec atlasmart-cass-1 nodetool statusdocker exec atlasmart-cass-1 cqlsh -e "SELECT release_version,cluster_name,data_center,rack FROM system.local;"docker exec atlasmart-cass-1 cqlsh -e "CREATE KEYSPACE IF NOT EXISTS atlasmart_model WITH replication = {'class':'NetworkTopologyStrategy','dc1':3};"docker exec atlasmart-cass-1 cqlsh -e "DESCRIBE KEYSPACE atlasmart_model"

On Windows, run these Docker commands from WSL/Git Bash or adapt the existence checks to PowerShell. If three Cassandra JVMs are too heavy for the machine, a single cassandra:5.0.9 node with RF=1 is sufficient for the chapter's query-shape exercises, but it is not equivalent evidence for RF=3/QUORUM behavior.

CREATE TABLE IF NOT EXISTS atlasmart_model.product_authority (  product_id text PRIMARY KEY, name text, category text, price_cents int, active boolean, source_version bigint) WITH compaction={'class':'UnifiedCompactionStrategy'};CREATE TABLE IF NOT EXISTS atlasmart_model.products_by_id (  product_id text PRIMARY KEY, name text, category text, price_cents int, active boolean, source_version bigint) WITH compaction={'class':'UnifiedCompactionStrategy'};CREATE TABLE IF NOT EXISTS atlasmart_model.products_by_category (  category text, price_cents int, product_id text, name text, active boolean, source_version bigint,  PRIMARY KEY ((category),price_cents,product_id)) WITH CLUSTERING ORDER BY (price_cents ASC,product_id ASC)  AND compaction={'class':'UnifiedCompactionStrategy'};CONSISTENCY QUORUM;INSERT INTO atlasmart_model.product_authority (product_id,name,category,price_cents,active,source_version) VALUES ('p-701','AtlasCam Mini','cameras',39900,true,12);INSERT INTO atlasmart_model.products_by_id (product_id,name,category,price_cents,active,source_version) VALUES ('p-701','AtlasCam Mini','cameras',39900,true,12);INSERT INTO atlasmart_model.products_by_category (category,price_cents,product_id,name,active,source_version) VALUES ('cameras',39900,'p-701','AtlasCam Mini',true,12);

Now simulate event version 13 reaching the authority and ID projection but not the category projection.

UPDATE atlasmart_model.product_authority SET name='AtlasCam Mini 2',source_version=13 WHERE product_id='p-701';UPDATE atlasmart_model.products_by_id SET name='AtlasCam Mini 2',source_version=13 WHERE product_id='p-701';SELECT product_id,name,source_version FROM atlasmart_model.product_authority WHERE product_id='p-701';SELECT product_id,name,source_version FROM atlasmart_model.products_by_id WHERE product_id='p-701';SELECT product_id,name,source_version FROM atlasmart_model.products_by_category WHERE category='cameras' AND price_cents=39900 AND product_id='p-701';

Your deterministic drift predicate is projection.source_version < authority.source_version (plus content validation where versions match). Repair by replaying version 13 to the stale projection.

UPDATE atlasmart_model.products_by_category SET name='AtlasCam Mini 2',source_version=13 WHERE category='cameras' AND price_cents=39900 AND product_id='p-701';SELECT product_id,name,source_version FROM atlasmart_model.products_by_category WHERE category='cameras' AND price_cents=39900 AND product_id='p-701';

4. Reconciliation must be a product feature, not a one-off script

A real reconciler needs a bounded work queue or scan strategy, rate limiting, checkpoints, metrics and an escalation path. At minimum record drift count, oldest drift age, projection retry backlog, replay success/failure and version regression attempts. A source version also protects against out-of-order delivery: a projection consumer should not blindly overwrite version 13 with stale version 12.

5. Safety and rollback

Projection schema changes need the same staged thinking as application changes. Add a new query table, dual-write or backfill it, reconcile against the authority, shadow-read it, switch traffic, retain the old path through a rollback window, then retire old writes/data deliberately. A migration that drops the old table immediately after a backfill has no safe rollback story.

Production judgment

A Cassandra table is an operational commitment, not just a schema object. Before approving the read shape, record expected partition cardinality and byte growth, retention/TTL behavior, read and write rates, tail-latency objectives, RF/CL, failure domains, write fan-out, retry/idempotency rules, reconciliation ownership, compaction and tombstone consequences, repair/backup requirements, observability signals, and the migration/rollback path. The same denormalization that removes a read-time join can multiply writes, storage, repair traffic and opportunities for projection drift.

Do not derive universal size or latency thresholds from this local lab. Production decisions require measurements with the real key distribution, payloads, concurrency, disk/network/JVM behavior and failure modes. SAI/vector features are intentionally not used to rescue a poor primary access model here; those mechanisms have their own later chapters and costs.

Lesson 5 consolidates the chapter by taking a normalized relational AtlasMart model and systematically deriving Cassandra query tables, update fan-out and migration controls from the workload.

Summary and next bridge

Denormalized data does not keep itself consistent. Projection ownership requires event identity/versioning, idempotent writes, observable retries and a reconciler that can detect and repair drift. The final lesson applies those controls to a complete normalized-to-Cassandra redesign and documents the costs that moved from reads to writes and operations.

Authoritative references

• Apache Cassandra downloads — release baseline; 5.0.9 is the current GA 5.0 patch at chapter generation time.
• Data modeling introduction — official query-driven modeling, partition-minimization and denormalization guidance.
• RDBMS design versus Cassandra design — official join, referential-integrity and denormalization contrasts.
• CQL data definition — primary-key, partition-key and clustering-order syntax used by the chapter tables.
• CQL data manipulation — SELECT restrictions and single-table query semantics.
• Unified Compaction Strategy — current Cassandra guidance for the explicit lab compaction baseline.