Chapter 18 · Secondary Indexing with Storage-Attached Indexing (SAI)

Create SAI Indexes for Textual / Numeric Columns and Combine Multiple Predicates

Create current Cassandra 5.0 SAI indexes for textual and numeric columns, combine supported predicates, and use rejected queries to expose real boundaries.

Intermediate → Advanced105–145 minutesText/numeric predicate labApache Cassandra 5.0.9 · SAI · cqlsh/nodetool · Java Driver 4.19.3 optional · RF=3 · UCSLast reviewed: September 2026

Learning outcomes

AtlasMart wants an operational catalog filter that combines brand, status, price band, and rating without maintaining a new table for every support-screen combination. This is a good SAI candidate only if the predicates are supported, the result sets are bounded, and the workload does not quietly become free-form relational search. This lesson tests the supported CQL surface instead of memorizing an index syntax.

01

Create text and numeric SAI indexes with current Cassandra 5.0 syntax and verify schema/build state.

02

Use equality and numeric range predicates and combine multiple indexed expressions with AND.

03

Explain text analyzer options and why they are not equivalent to relevance-ranked full-text search.

04

Identify current type/operator restrictions such as counters, non-frozen UDTs, LIKE, and single-column partition-key indexing.

05

Use rejected queries as evidence for the boundary rather than adding ALLOW FILTERING reflexively.

Chapter 18 lab baseline

The mandatory labs continue the disposable AtlasMart course cluster: Apache Cassandra 5.0.9 in the pinned cassandra:5.0.9 image, Java 17 inside the image, cluster atlasmart-course, Docker network atlasmart-cassandra, nodes atlasmart-cass-1..3, datacenter dc1, racks rack1..rack3, 16 virtual nodes per node, NetworkTopologyStrategy with replication factor (RF) 3, and consistency level (CL) LOCAL_QUORUM unless an experiment explicitly changes it. New tables use UnifiedCompactionStrategy (UCS), gc_grace_seconds = 864000, and no default Time To Live (TTL). Authentication, client Transport Layer Security (TLS), internode TLS, and remote Java Management Extensions (JMX) are disabled only inside this isolated local learning network. Apache Cassandra Java Driver 4.19.3 is optional for latency/routing exercises; all mandatory SAI evidence is available with cqlsh, virtual tables, tracing, nodetool, and read-only filesystem inventory. Storage-Attached Indexing (SAI) is a Cassandra 5.0 feature; do not assume the same syntax or operational behavior on Cassandra 4.x, commercial distributions, or Cassandra-compatible services.

Execution and safety note

Run commands only against the disposable Apache Cassandra course lab or another explicitly approved non-production environment. Confirm node, keyspace, table, container, volume, path, and datacenter targets before destructive, failure-injection, cleanup, repair, restore, security, or topology operations. Capture current state and expected rollback/recovery evidence first; output and timings can differ by host, operating system, Java runtime, Docker/runtime, driver, and Cassandra configuration.

Core terms for this chapter

Apache Cassandra is a peer-to-peer distributed database. CQL is the Cassandra Query Language. A partition key determines a partition and is hashed into a token; token ownership and the keyspace replication strategy determine the replicas storing that partition. A request's coordinator is the Cassandra node that accepts that client request and coordinates work with replicas. A primary-key access path identifies partitions directly from the primary key and is Cassandra's most efficient lookup path. A secondary index provides an additional way to find rows by non-primary-key values. Storage-Attached Indexing (SAI) is Cassandra 5.0's storage-integrated secondary-index capability: it indexes Memtables in memory and attaches on-disk index components to SSTables. An SSTable is an immutable on-disk Cassandra storage file set; a Memtable is the mutable in-memory write buffer before flush. Selectivity describes how narrowly a predicate reduces candidate rows; a predicate matching nearly every row is low-selectivity. Fanout is the amount of token-range/replica work a distributed query must touch. Build state records whether an index is still being constructed and whether it is queryable. Streaming moves SSTable components between nodes during operations such as bootstrap, repair, rebuild, or decommission. SAI is a filtering engine, not a general relational join engine or full-text relevance/search platform.

1. Supported does not mean “arbitrary SQL WHERE clause”

Current Cassandra 5.0 SAI supports indexes on most CQL columns, including many primary-key, collection, static, text, numeric, temporal, and UUID-like types. Important restrictions remain: counter columns and non-frozen user-defined types are not SAI-indexable, and a table whose partition key is a single column does not need/allow an SAI index on that partition-key column. Numeric indexes support equality/ranges; text indexes support equality and collection-specific containment patterns. LIKE is not a supported SAI operator. Current SAI concepts also describe richer logical behavior such as IN/OR in some contexts; because operator support can evolve, this lab uses the conservative command-reference surface—equality, numeric range, collection containment when explicitly indexed, and AND—and asks you to re-check your target patch before relying on newer expressions.

Need SAI pattern Boundary
Exact text filter brand='Northstar' not relevance ranking or fuzzy search
Numeric band price_cents >= X AND price_cents <= Y selectivity still determines distributed work
Multiple filters brand=... AND price... AND status=... more indexed expressions process more components; post-filtering may occur
Text wildcard LIKE ... not supported by SAI
Counter search index counter column not supported
Primary-key lookup product_id=... use primary-key access; no SAI needed

2. Create a practical index set, not an index set for every column

bash · verify or recreate the disposable three-node lab
# Verify the existing course cluster first.docker exec atlasmart-cass-1 nodetool versiondocker exec atlasmart-cass-1 nodetool statusdocker exec atlasmart-cass-1 java -version# If the shared course cluster does not exist, recreate the same local topology.docker network inspect atlasmart-cassandra >/dev/null 2>&1 || docker network create atlasmart-cassandradocker volume create atlasmart-cass-1-datadocker volume create atlasmart-cass-2-datadocker volume create atlasmart-cass-3-datadocker inspect atlasmart-cass-1 >/dev/null 2>&1 || docker run -d --name atlasmart-cass-1 --hostname atlasmart-cass-1 --network atlasmart-cassandra -e CASSANDRA_CLUSTER_NAME=atlasmart-course -e CASSANDRA_DC=dc1 -e CASSANDRA_RACK=rack1 -e CASSANDRA_ENDPOINT_SNITCH=GossipingPropertyFileSnitch -e CASSANDRA_NUM_TOKENS=16 -v atlasmart-cass-1-data:/var/lib/cassandra cassandra:5.0.9# Wait until node 1 is UN before starting peers.docker exec atlasmart-cass-1 nodetool statusdocker inspect atlasmart-cass-2 >/dev/null 2>&1 || docker run -d --name atlasmart-cass-2 --hostname atlasmart-cass-2 --network atlasmart-cassandra -e CASSANDRA_CLUSTER_NAME=atlasmart-course -e CASSANDRA_DC=dc1 -e CASSANDRA_RACK=rack2 -e CASSANDRA_ENDPOINT_SNITCH=GossipingPropertyFileSnitch -e CASSANDRA_NUM_TOKENS=16 -e CASSANDRA_SEEDS=atlasmart-cass-1 -v atlasmart-cass-2-data:/var/lib/cassandra cassandra:5.0.9docker inspect atlasmart-cass-3 >/dev/null 2>&1 || docker run -d --name atlasmart-cass-3 --hostname atlasmart-cass-3 --network atlasmart-cassandra -e CASSANDRA_CLUSTER_NAME=atlasmart-course -e CASSANDRA_DC=dc1 -e CASSANDRA_RACK=rack3 -e CASSANDRA_ENDPOINT_SNITCH=GossipingPropertyFileSnitch -e CASSANDRA_NUM_TOKENS=16 -e CASSANDRA_SEEDS=atlasmart-cass-1 -v atlasmart-cass-3-data:/var/lib/cassandra cassandra:5.0.9# Continue only when all three nodes are UN in dc1.docker exec atlasmart-cass-1 nodetool statusdocker exec -it atlasmart-cass-1 cqlsh
CQL · create the Chapter 18 base and query-table fixtures
CREATE KEYSPACE IF NOT EXISTS atlasmart_saiWITH replication = {'class':'NetworkTopologyStrategy','dc1':3};CREATE TABLE IF NOT EXISTS atlasmart_sai.product_catalog (    product_id uuid PRIMARY KEY,    category text,    brand text,    status text,    price_cents int,    rating int,    warehouse_region text,    updated_at timestamp) WITH compaction = {'class':'UnifiedCompactionStrategy'};CREATE TABLE IF NOT EXISTS atlasmart_sai.products_by_category_bucket (    category text,    bucket tinyint,    product_id uuid,    brand text,    status text,    price_cents int,    rating int,    PRIMARY KEY ((category,bucket),product_id)) WITH compaction = {'class':'UnifiedCompactionStrategy'};CONSISTENCY LOCAL_QUORUM;
CQL · load a small deterministic AtlasMart catalog
INSERT INTO atlasmart_sai.product_catalog (product_id,category,brand,status,price_cents,rating,warehouse_region,updated_at) VALUES (00000000-0000-0000-0000-000000001801,'laptop','Northstar','ACTIVE',129900,5,'eu-west',toTimestamp(now()));INSERT INTO atlasmart_sai.product_catalog (product_id,category,brand,status,price_cents,rating,warehouse_region,updated_at) VALUES (00000000-0000-0000-0000-000000001802,'laptop','Northstar','ACTIVE',89900,4,'eu-west',toTimestamp(now()));INSERT INTO atlasmart_sai.product_catalog (product_id,category,brand,status,price_cents,rating,warehouse_region,updated_at) VALUES (00000000-0000-0000-0000-000000001803,'laptop','Contoso','ACTIVE',149900,5,'us-east',toTimestamp(now()));INSERT INTO atlasmart_sai.product_catalog (product_id,category,brand,status,price_cents,rating,warehouse_region,updated_at) VALUES (00000000-0000-0000-0000-000000001804,'monitor','Northstar','ACTIVE',39900,4,'eu-west',toTimestamp(now()));INSERT INTO atlasmart_sai.product_catalog (product_id,category,brand,status,price_cents,rating,warehouse_region,updated_at) VALUES (00000000-0000-0000-0000-000000001805,'monitor','Fabrikam','DISCONTINUED',29900,3,'us-east',toTimestamp(now()));INSERT INTO atlasmart_sai.product_catalog (product_id,category,brand,status,price_cents,rating,warehouse_region,updated_at) VALUES (00000000-0000-0000-0000-000000001806,'keyboard','Fabrikam','ACTIVE',9900,4,'eu-west',toTimestamp(now()));INSERT INTO atlasmart_sai.product_catalog (product_id,category,brand,status,price_cents,rating,warehouse_region,updated_at) VALUES (00000000-0000-0000-0000-000000001807,'keyboard','Northstar','ACTIVE',15900,5,'us-east',toTimestamp(now()));INSERT INTO atlasmart_sai.product_catalog (product_id,category,brand,status,price_cents,rating,warehouse_region,updated_at) VALUES (00000000-0000-0000-0000-000000001808,'mouse','Contoso','ACTIVE',4900,4,'eu-west',toTimestamp(now()));INSERT INTO atlasmart_sai.products_by_category_bucket (category,bucket,product_id,brand,status,price_cents,rating) VALUES ('laptop',0,00000000-0000-0000-0000-000000001801,'Northstar','ACTIVE',129900,5);INSERT INTO atlasmart_sai.products_by_category_bucket (category,bucket,product_id,brand,status,price_cents,rating) VALUES ('laptop',0,00000000-0000-0000-0000-000000001802,'Northstar','ACTIVE',89900,4);INSERT INTO atlasmart_sai.products_by_category_bucket (category,bucket,product_id,brand,status,price_cents,rating) VALUES ('laptop',0,00000000-0000-0000-0000-000000001803,'Contoso','ACTIVE',149900,5);
CQL · create selected text and numeric SAI indexes
CREATE INDEX IF NOT EXISTS product_brand_saiON atlasmart_sai.product_catalog (brand) USING 'sai'WITH OPTIONS = {'case_sensitive':'false','normalize':'true','ascii':'true'};CREATE INDEX IF NOT EXISTS product_status_saiON atlasmart_sai.product_catalog (status) USING 'sai';CREATE INDEX IF NOT EXISTS product_price_saiON atlasmart_sai.product_catalog (price_cents) USING 'sai';CREATE INDEX IF NOT EXISTS product_rating_saiON atlasmart_sai.product_catalog (rating) USING 'sai';DESCRIBE TABLE atlasmart_sai.product_catalog;SELECT index_name,column_name,analyzer,is_building,is_queryable,       indexed_sstable_count,per_column_disk_size,per_table_disk_sizeFROM system_views.indexesWHERE keyspace_name='atlasmart_sai';

The brand index uses case/normalization options so equality comparisons can normalize text according to the configured analyzer. That still does not produce stemming, relevance ranking, fuzzy typo tolerance, synonym expansion, phrase search, or highlighting. If those are product requirements, evaluate an external search engine rather than layering many assumptions onto SAI.

3. Combine indexed predicates and inspect actual candidate work

CQL · supported equality/range/AND examples
CONSISTENCY LOCAL_QUORUM;TRACING ON;SELECT product_id,category,brand,status,price_cents,ratingFROM atlasmart_sai.product_catalogWHERE brand='northstar'  AND status='ACTIVE'  AND price_cents >= 10000  AND price_cents <= 140000LIMIT 20;TRACING OFF;SELECT product_id,category,brand,price_cents,ratingFROM atlasmart_sai.product_catalogWHERE rating >= 4 AND price_cents < 100000LIMIT 20;

Record the final row count and the trace's index/segment/partition/post-filter counts. Multiple indexed clauses are not free: current documentation notes that processing more indexed columns increases component work, and some additional expressions can be applied through post-filtering after initial indexed candidate generation. Therefore “four indexes narrowed the result to two rows” is incomplete without the trace and dataset cardinality.

4. Deliberately rejected queries teach the boundary

CQL · deliberate boundary probes; capture the actual errors
-- LIKE is not an SAI query operator in the current command reference.SELECT product_id,brand FROM atlasmart_sai.product_catalogWHERE brand LIKE 'North%';-- product_id is the single-column partition key: direct lookup is already the primary index.CREATE INDEX product_id_saiON atlasmart_sai.product_catalog (product_id) USING 'sai';-- Counter columns cannot be SAI-indexed. Use a disposable schema probe.CREATE TABLE IF NOT EXISTS atlasmart_sai.counter_probe (  k text PRIMARY KEY,  c counter);CREATE INDEX counter_probe_saiON atlasmart_sai.counter_probe (c) USING 'sai';

Do not “repair” an unsupported expression by adding ALLOW FILTERING until the data model and boundedness are understood. ALLOW FILTERING is permission for server filtering, not a feature that converts unsupported index semantics into a scalable query. The safer repairs are: direct primary-key routing, a query-specific denormalized table, a supported SAI predicate, or an external search system whose feature contract matches the product requirement.

Wrong approach: “If four indexes work, ten must be better.”

Current Cassandra includes a per-table SAI index-count guardrail and query-time SSTable-index guardrails. Even below those limits, every index adds synchronous write/index work and disk state. Add an index because a measured query/SLO justifies it, not because a column exists.

5. Verification and cleanup

  • Four intended indexes are queryable; rejected probe indexes were not created.
  • The combined query returns the expected logical subset from the small fixture.
  • You captured trace evidence rather than inferring selectivity from only the final row count.
  • You can distinguish analyzer normalization from full-text relevance search.
  • You did not use unsupported queries as evidence that SAI is broken; they are contract boundaries.
CQL · clean only the disposable counter probe
DROP TABLE IF EXISTS atlasmart_sai.counter_probe;SELECT index_name,column_name,is_queryableFROM system_views.indexesWHERE keyspace_name='atlasmart_sai';

Check your understanding

  1. Can SAI index a counter column?
  2. Why is an SAI index on a single-column partition key rejected?
  3. Does a case-insensitive normalized text index provide full-text relevance ranking?
  4. What is the safest operator set for this reproducible lab?
  5. Why can many indexed predicates still be expensive?
Review the answers

1. No. Counter columns are explicitly unsupported for SAI indexing.

2. The primary key already provides the direct partition access path; SAI does not add a meaningful secondary path there.

3. No. It changes equality/token handling according to configured analyzer options; it is not a full search/relevance engine.

4. Text equality, numeric equality/ranges, AND, and explicitly documented collection containment patterns; re-check target-version docs for newer operators.

5. They add index-component processing and can require post-filtering/materialization after candidate discovery.

Production judgment

Adopt SAI only after the primary-key model and business query have been written down. Record expected cardinality/selectivity, result limit, partitions/token ranges touched, replica/DC scope, RF/CL, p50/p95/p99 latency, index build state, SSTables/indexes touched per query, SAI disk bytes, base-table disk bytes, write amplification, compaction strategy/backlog, tombstones/TTL, streaming and repair behavior, node/index failure behavior, driver timeout/retry/idempotency policy, and whether authorization is enforced before or as part of retrieval. SAI adds synchronous write work and on-disk components; it does not make a low-selectivity cluster-wide predicate free. More SAI indexes also mean more components and more write/disk cost, so index count is a workload decision rather than a schema decoration.

For security, remember that index components may contain user-derived values and must be included in the data-at-rest threat model. For managed Cassandra services, verify exact SAI version, allowed index options, query guardrails, observability surfaces, backup/restore behavior, encryption, and billing model instead of assuming open-source defaults. Do not claim SAI provides cross-row relational constraints, global linearizability, arbitrary joins, relevance scoring, fuzzy full-text semantics, or authorization isolation. Lesson 4 scales the fixture and measures selectivity, per-index disk bytes, SSTable-index fanout, guardrails, write/read latency, and one-node failure behavior.

Summary and next step

This lesson’s concepts, evidence path, failure boundaries, and production judgment should now be explicit enough to verify rather than assume. Re-run the check-your-understanding prompts and preserve any lab evidence you need before changing or cleaning up the environment.

Next, continue to Index Selectivity, Range Queries, Disk/Write Cost, Query Limits, and Operational Monitoring.

Authoritative references

These are the version-sensitive source of truth for this lesson. Re-check them when regenerating the course because SAI capabilities, guardrails, virtual tables, and query semantics can evolve.

Keep knowledge open

Help the academy stay free and grow.

If these tutorials save you time, a small donation supports new lessons, technical review, diagrams, examples, and long-term maintenance.

ETHEthereum / ERC-20 only
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0

Send only Ethereum or ERC-20 compatible assets to this address.