Capacity, Index Build / Streaming, Embedding Updates, Evaluation, and AI / RAG Architecture Tradeoffs

Learning outcomes

AtlasMart's prototype is ready for a production review. The final questions are operational: how much index capacity is required, what happens during bootstrap/rebuild, how embeddings are updated without mixing spaces, how recall/relevance regressions block rollout, and where Cassandra stops in a RAG architecture.

Terms and mental model

A vector is an ordered fixed-length array of numbers. An embedding is a vector produced by a model or deterministic transformation to represent an object in a geometric space. Its dimension is the number of elements. Embedding provenance means the model name/version, preprocessing, normalization policy, and source revision that explain how the vector was created. A similarity function or distance rule decides what “near” means. Cosine similarity compares direction, dot product combines direction and magnitude unless vectors are normalized, and Euclidean is based on geometric distance. Normalization usually means scaling a vector to unit length. Approximate Nearest Neighbor (ANN) retrieval searches an index for likely nearest vectors without guaranteeing the exact top-k ordering; exact retrieval evaluates every candidate under the chosen metric. Recall@k is the fraction of exact top-k items recovered by ANN. It measures retrieval fidelity, not whether the documents are useful to a user.

Storage-Attached Indexing (SAI) is Cassandra 5.0's storage-integrated secondary indexing framework; vector ANN search uses an SAI index on a vector<float,n> column. A coordinator is the Cassandra node handling a particular client request. A replica stores data for a token range according to the keyspace replication strategy. A partition is the Cassandra storage/routing unit selected by a partition key; the partitioner hashes that key to a token. An immutable SSTable is an on-disk Sorted String Table. A consistency level (CL) states the replica acknowledgments/responses required by an operation. RAG (Retrieval-Augmented Generation) is an application architecture that retrieves evidence and supplies it to a generative model; Cassandra can be the retrieval store but does not generate embeddings or model answers.

1. Capacity: vectors, replicas, indexes, and transient headroom

A raw vector<float,d> has roughly 4 × d bytes of float payload before row/storage/index overhead. A 1536-dimensional float vector therefore carries about 6 KiB of raw coordinates per row, while the lab's four-dimensional vector carries only 16 bytes. Production capacity must add primary-key/row metadata, text/metadata columns, SAI vector/index structures, RF replication, SSTable compression, compaction/rebuild headroom, snapshots/backups, and free-space reserve.

CREATE KEYSPACE IF NOT EXISTS atlasmart_vectorWITH replication = {'class':'NetworkTopologyStrategy','dc1':3};CREATE TABLE IF NOT EXISTS atlasmart_vector.documents (    tenant_id text,    corpus_bucket tinyint,    document_id uuid,    title text,    body text,    locale text,    doc_type text,    embedding_model text,    embedding_version text,    embedding vector<float,4>,    updated_at timestamp,    PRIMARY KEY ((tenant_id,corpus_bucket),document_id)) WITH compaction = {'class':'UnifiedCompactionStrategy'};CONSISTENCY LOCAL_QUORUM;CREATE INDEX IF NOT EXISTS documents_embedding_annON atlasmart_vector.documents (embedding) USING 'sai'WITH OPTIONS = {'similarity_function':'COSINE'};CREATE INDEX IF NOT EXISTS documents_locale_saiON atlasmart_vector.documents (locale) USING 'sai';CREATE INDEX IF NOT EXISTS documents_type_saiON atlasmart_vector.documents (doc_type) USING 'sai';

INSERT INTO atlasmart_vector.documents (tenant_id,corpus_bucket,document_id,title,body,locale,doc_type,embedding_model,embedding_version,embedding,updated_at) VALUES ('tenant-a',0,00000000-0000-0000-0000-000000001901,'Return policy','How to return an unopened product within the return window.','en','support','atlasmart-toy','v1',[1.0,0.0,0.0,0.0],'2026-09-08T07:00:00Z');INSERT INTO atlasmart_vector.documents (tenant_id,corpus_bucket,document_id,title,body,locale,doc_type,embedding_model,embedding_version,embedding,updated_at) VALUES ('tenant-a',0,00000000-0000-0000-0000-000000001902,'Refund delay','Why approved refunds can take several days to appear.','en','support','atlasmart-toy','v1',[0.9,0.2,0.1,0.0],'2026-09-08T07:01:00Z');INSERT INTO atlasmart_vector.documents (tenant_id,corpus_bucket,document_id,title,body,locale,doc_type,embedding_model,embedding_version,embedding,updated_at) VALUES ('tenant-a',0,00000000-0000-0000-0000-000000001903,'Shipping tracker','Track a parcel after warehouse dispatch.','en','support','atlasmart-toy','v1',[0.0,1.0,0.0,0.0],'2026-09-08T07:02:00Z');INSERT INTO atlasmart_vector.documents (tenant_id,corpus_bucket,document_id,title,body,locale,doc_type,embedding_model,embedding_version,embedding,updated_at) VALUES ('tenant-a',0,00000000-0000-0000-0000-000000001904,'Password reset','Recover access to an AtlasMart account.','en','security','atlasmart-toy','v1',[0.0,0.0,1.0,0.0],'2026-09-08T07:03:00Z');INSERT INTO atlasmart_vector.documents (tenant_id,corpus_bucket,document_id,title,body,locale,doc_type,embedding_model,embedding_version,embedding,updated_at) VALUES ('tenant-a',1,00000000-0000-0000-0000-000000001905,'Invoice copy','Download a VAT invoice for an order.','en','billing','atlasmart-toy','v1',[0.0,0.0,0.0,1.0],'2026-09-08T07:04:00Z');INSERT INTO atlasmart_vector.documents (tenant_id,corpus_bucket,document_id,title,body,locale,doc_type,embedding_model,embedding_version,embedding,updated_at) VALUES ('tenant-a',1,00000000-0000-0000-0000-000000001906,'Exchange item','Exchange an eligible item for another size.','en','support','atlasmart-toy','v1',[0.8,0.1,0.0,0.2],'2026-09-08T07:05:00Z');INSERT INTO atlasmart_vector.documents (tenant_id,corpus_bucket,document_id,title,body,locale,doc_type,embedding_model,embedding_version,embedding,updated_at) VALUES ('tenant-a',1,00000000-0000-0000-0000-000000001907,'Cancel order','Cancel before warehouse fulfillment begins.','en','support','atlasmart-toy','v1',[0.7,0.0,0.2,0.1],'2026-09-08T07:06:00Z');INSERT INTO atlasmart_vector.documents (tenant_id,corpus_bucket,document_id,title,body,locale,doc_type,embedding_model,embedding_version,embedding,updated_at) VALUES ('tenant-a',1,00000000-0000-0000-0000-000000001908,'Delivery delay','Investigate a shipment that missed its delivery date.','en','support','atlasmart-toy','v1',[0.1,0.9,0.0,0.0],'2026-09-08T07:07:00Z');INSERT INTO atlasmart_vector.documents (tenant_id,corpus_bucket,document_id,title,body,locale,doc_type,embedding_model,embedding_version,embedding,updated_at) VALUES ('tenant-b',0,00000000-0000-0000-0000-000000001909,'Tenant B private return note','Private tenant B support content.','en','support','atlasmart-toy','v1',[0.99,0.01,0.0,0.0],'2026-09-08T07:08:00Z');

DESCRIBE TABLE atlasmart_vector.documents;SELECT keyspace_name,index_name,column_name,cell_count,indexed_sstable_count,       is_building,is_queryable,per_column_disk_size,per_table_disk_sizeFROM system_views.indexesWHERE keyspace_name='atlasmart_vector';

docker exec atlasmart-cass-1 nodetool tablestats atlasmart_vector.documentsdocker exec atlasmart-cass-1 nodetool compactionstatsdocker exec atlasmart-cass-1 nodetool netstatsdocker exec atlasmart-cass-1 sh -lc "du -sh /var/lib/cassandra/data/atlasmart_vector/* 2>/dev/null || true"

per_column_disk_size and per_table_disk_size are node-local SAI component evidence. Measure after representative flush/compaction state; do not multiply a tiny-table ratio into a petabyte forecast.

2. Build, rebuild, streaming, and readiness

Creating an SAI index on existing SSTables launches a build. During a drop/recreate, queries that do not depend on the index can continue, but the vector index itself is not usable until it reports queryable. Cassandra 5.0 SAI is compatible with zero-copy streaming: when that path is eligible during topology streaming, index components can move with SSTables rather than being rebuilt independently on the receiver. If an index must be rebuilt, nodetool rebuild_index is the supported operator command.

SELECT index_name,is_building,is_queryable,indexed_sstable_count,       per_column_disk_size,per_table_disk_sizeFROM system_views.indexesWHERE keyspace_name='atlasmart_vector';

# Run only in the disposable lab when intentionally testing rebuild cost.docker exec atlasmart-cass-1 nodetool rebuild_index atlasmart_vector documents documents_embedding_ann# Observe build and topology/network activity in separate terminals.docker exec atlasmart-cass-1 nodetool netstatsdocker exec atlasmart-cass-1 cqlsh -e "SELECT index_name,is_building,is_queryable,indexed_sstable_count FROM system_views.indexes WHERE keyspace_name='atlasmart_vector';"

3. Embedding migration: dual space, evaluate, cut over, rollback

Updating every vector in place to a new model can mix old/new spaces while traffic is live and causes sustained vector-index update work. A safer pattern writes v2 into a separate versioned table/index (or otherwise isolates the new space), backfills, waits for queryability, evaluates the same fixed query/ground-truth set, shadows production queries, then cuts over. Keep v1 until rollback criteria expire.

CREATE TABLE IF NOT EXISTS atlasmart_vector.documents_v2 (  tenant_id text,  corpus_bucket tinyint,  document_id uuid,  title text,  locale text,  doc_type text,  embedding_model text,  embedding_version text,  embedding vector<float,4>,  updated_at timestamp,  PRIMARY KEY ((tenant_id,corpus_bucket),document_id)) WITH compaction = {'class':'UnifiedCompactionStrategy'};CREATE INDEX IF NOT EXISTS documents_v2_embedding_annON atlasmart_vector.documents_v2 (embedding) USING 'sai'WITH OPTIONS = {'similarity_function':'COSINE'};CREATE INDEX IF NOT EXISTS documents_v2_locale_sai ON atlasmart_vector.documents_v2 (locale) USING 'sai';CREATE INDEX IF NOT EXISTS documents_v2_type_sai ON atlasmart_vector.documents_v2 (doc_type) USING 'sai';

4. RAG architecture: Cassandra retrieves; the application governs and generates

A robust Retrieval-Augmented Generation pipeline has separate stages: authenticate the caller; derive authorized corpus scope; embed the query with the same compatible model space; retrieve ANN candidates from Cassandra with structured filters; optionally rerank; fetch source text/metadata; construct a prompt/context with provenance; call a local or hosted generative model; validate/cite the answer; log evaluation/feedback. Cassandra handles durable distributed storage, structured metadata, SAI filtering, and vector ANN retrieval. It does not produce embeddings, rerank with a cross-encoder, enforce arbitrary row-level policy, create prompts, or generate the final answer.

request  -> authenticate user/service  -> derive allowed tenant/corpus/buckets  -> generate query embedding (same model/version/normalization)  -> Cassandra filtered ANN retrieval  -> optional rerank / exact rescoring  -> fetch source passages + provenance  -> prompt/context construction  -> local or hosted LLM generation  -> grounding/citation/policy checks  -> response + telemetry + evaluation

5. Final Chapter 19 acceptance test

docker exec atlasmart-cass-1 nodetool versiondocker exec atlasmart-cass-1 nodetool statusdocker exec atlasmart-cass-1 cqlsh -e "SELECT index_name,column_name,is_building,is_queryable,indexed_sstable_count,per_column_disk_size,per_table_disk_size FROM system_views.indexes WHERE keyspace_name='atlasmart_vector';"docker exec atlasmart-cass-1 nodetool tablestats atlasmart_vector.documentsdocker exec atlasmart-cass-1 nodetool compactionstatsdocker exec atlasmart-cass-1 nodetool netstats

• Version/topology/RF/CL are recorded.
• Embedding model/version/normalization contract is documented.
• ANN index metric and readiness are recorded.
• Recall@k uses fixed exact ground truth, while relevance is evaluated separately.
• p50/p95/p99 benchmark context is documented.
• Authorization constrains retrieval before ANN results leave the data layer.
• v2 migration has a rollback plan; v1 is not destroyed before acceptance.

Production judgment

Decide whether Cassandra vector search fits by measuring the whole retrieval system: corpus size and growth; vector dimension and bytes; embedding generation/update rate; embedding provenance; tenant and authorization model; query filters and bucket fanout; RF/CL and node/DC failures; ANN recall@k against a fixed exact baseline; application relevance metrics; p50/p95/p99 retrieval latency; result payload size; write amplification; SAI disk and memory footprint; SSTable count/compaction; vector overwrite/delete rate; index build/rebuild/streaming time; repair/backup/restore behavior; JVM/off-heap/chunk-cache pressure; driver timeouts/retries/idempotency; guardrails; observability; and operator skill. Do not report ANN latency without recall, and do not call recall “answer quality.”

Security is not a similarity metric. Cassandra role permissions are table/keyspace oriented, not row-level authorization; an application must derive allowed tenant/corpus scope before retrieval and encode that scope in the data model/query. Post-filtering unauthorized ANN results can leak information or reduce useful top-k results. Managed services and Cassandra-compatible APIs may differ in vector syntax, limits, index lifecycle, metrics, encryption, billing, and topology. Chapter 20 leaves vector search and examines materialized views and explicit derived-table alternatives—another case where automatic derivation must be weighed against ownership, failure, and operational control.

Summary and next bridge

Cassandra 5.0 vector search is a storage-integrated ANN capability, not an AI stack by itself. Production readiness requires fixed embedding provenance, measurable recall and latency, authorized structured scope, capacity/build/streaming evidence, and a versioned migration strategy. Chapter 20 applies the same mechanism-first discipline to materialized views and derived-table choices.

Authoritative references

Vector search and SAI are version-sensitive. Re-check these sources when regenerating the lesson rather than freezing a 2026 assumption.

• Apache Cassandra downloads / current GA baseline
• Vector CQL data type
• Vector Search overview
• Vector Search concepts
• Vector Search data modeling
• Working with Vector Search
• CREATE INDEX / SAI similarity_function
• SAI virtual tables
• SAI monitoring
• Java Driver vector support
• nodetool rebuild_index
• SAI concepts / zero-copy streaming