Chapter 09 · Redis Search: Indexes, Text/Numeric/Tag/Geo Fields, Querying, and Aggregation
Index Maintenance, Memory Cost, Query Profiling, and Choosing Search vs Key-Based Access
Measure index memory/write/query cost, profile and migrate indexes safely, and choose Search only when secondary retrieval justifies its operational cost.
Learning outcomes
A Search index is only valuable when its query capability justifies its memory, indexing work, operational lifecycle, and application complexity. AtlasMart therefore finishes Chapter 09 by measuring index state, profiling representative queries, testing schema evolution and index removal, and comparing Search with direct key access. The goal is not “optimize Redis”; it is to choose the simplest access path that satisfies the workload.
Read FT.INFO memory/indexing statistics without confusing them with source-key memory or process RSS.
Use FT.PROFILE and FT.EXPLAINCLI to separate parser correctness from runtime query cost.
Measure p50/p95/p99 query and write paths with representative fixtures and clearly disclosed methodology.
Plan index schema changes with FT.ALTER or versioned indexes/aliases according to what actually changes.
Choose direct key access versus Search from access pattern, freshness, cost, security, topology, and migration requirements.
All Chapter 09 mandatory labs reuse the disposable Chapter 01
environment: Redis Open Source 8.10.1 from Docker
Official Image redis:8.10.1, container
atlasmart-redis-ch01, standalone topology, host
publication 127.0.0.1:6379, TLS disabled only
because traffic stays on loopback, default ACL user disabled,
named users atlasmart-app and
academy-admin, logical database 0, AOF with
appendfsync everysec plus RDB snapshots,
persistent /data, and no explicit
maxmemory limit or eviction policy. Redis 8
integrates Search and JSON into Redis Open Source; the
mandatory path needs no separate historical Redis Stack image
or paid service. Source writes use the restricted application
user where possible; Search/index administration and evidence
commands use the disposable academy-admin user
because the Chapter 01 application ACL was intentionally not
broadened for Search administration. Fixtures stay under
atlasmart:ch09:*. No benchmark result is
fabricated here. The environment has no live Docker daemon, so
this lesson supplies commands and a local measurement harness
and labels expected qualitative evidence rather than invented
timings.
1. Re-create a known index before measuring it
docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app DEL atlasmart:ch09:product:1001 atlasmart:ch09:product:1002 atlasmart:ch09:product:1003 atlasmart:ch09:outside:product:9001docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app JSON.SET atlasmart:ch09:product:1001 '$' '{"schemaVersion":2,"sku":"SKU-1001","name":"Trail Running Backpack","description":"Lightweight running backpack for trail and city travel","category":["outdoor","travel"],"brand":"AtlasPeak","priceCents":12990,"stock":12,"location":"49.8671,40.4093"}'docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app JSON.SET atlasmart:ch09:product:1002 '$' '{"schemaVersion":2,"sku":"SKU-1002","name":"City Runner Pack","description":"Compact runner backpack for commuting and urban travel","category":["urban","travel"],"brand":"AtlasPeak","priceCents":9990,"stock":0,"location":"49.8920,40.3777"}'docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app JSON.SET atlasmart:ch09:product:1003 '$' '{"schemaVersion":2,"sku":"SKU-1003","name":"Trail Bottle","description":"Insulated bottle for hiking and trail running","category":["outdoor","hydration"],"brand":"NorthSpring","priceCents":3490,"stock":40,"location":"49.8500,40.4000"}'docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app JSON.SET atlasmart:ch09:outside:product:9001 '$' '{"schemaVersion":2,"sku":"SKU-9001","name":"Outside Prefix Backpack","description":"This key proves prefix scope","category":["outdoor"],"brand":"ScopeTest","priceCents":1,"stock":1,"location":"49.8671,40.4093"}'docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin FT.CREATE atlasmart-ch09-products-json-idx ON JSON PREFIX 1 atlasmart:ch09:product: SCHEMA '$.sku' AS sku TAG SORTABLE '$.name' AS name TEXT WEIGHT 2.0 '$.description' AS description TEXT '$.category[*]' AS category TAG SORTABLE '$.brand' AS brand TAG SORTABLE '$.priceCents' AS priceCents NUMERIC SORTABLE '$.stock' AS stock NUMERIC SORTABLE '$.location' AS location GEO '$.schemaVersion' AS schemaVersion NUMERICMeasurement only means something when the source dataset and schema are recorded. This fixture is intentionally tiny for correctness; capacity conclusions require a separately generated representative dataset.
2. FT.INFO is the first index health/capacity ledger
Current Redis Search exposes index configuration plus metrics including document/term/record counts, indexing progress, failures, number of uses, and memory components such as document table, inverted index, key table, offset vectors, sortable values, and vector index size where applicable. Exact field names can evolve; treat the target server reply as the schema for your monitoring parser.
docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin FT.INFO atlasmart-ch09-products-json-idxdocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin INFO memorydocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app MEMORY USAGE atlasmart:ch09:product:1001docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin FT.SEARCH atlasmart-ch09-products-json-idx '*' NOCONTENT LIMIT 0 0 DIALECT 2
MEMORY USAGE for one source document is not the
index size. FT.INFO index components are not total
Redis process RSS. INFO memory includes
allocator/process effects beyond source and Search structures.
Capacity planning needs all three perspectives.
3. Avoid over-indexing: each searchable/sortable feature has a cost
Every TEXT field can add terms, records, offsets, and frequency data; every TAG/NUMERIC/GEO field adds its own index structures; SORTABLE keeps values available for efficient sort/aggregation; schema breadth increases update work and migration surface. Redis guidance explicitly recommends indexing only fields required by planned queries.
| Schema choice | Benefit | Cost/risk to measure |
|---|---|---|
| TEXT description | full-text retrieval | term/offset/frequency memory; update CPU |
| TAG category | exact filter | tag index memory; cardinality |
| NUMERIC SORTABLE price | range + fast sort/aggregate | numeric index + sortable value memory |
| GEO location | radius filter | geo index memory and coordinate quality |
| Index every internal field | none unless queried | memory/write amplification and larger migration surface |
4. FT.EXPLAINCLI proves parser intent; FT.PROFILE proves runtime work
Use FT.EXPLAINCLI first when results are logically
surprising. Use FT.PROFILE after the parsed query
matches the business question. Otherwise you can spend time
optimizing the wrong predicate.
docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin FT.EXPLAINCLI atlasmart-ch09-products-json-idx '@category:{outdoor} @priceCents:[0 15000] @description:(running)' DIALECT 2docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin FT.PROFILE atlasmart-ch09-products-json-idx SEARCH LIMITED QUERY '@category:{outdoor} @priceCents:[0 15000] @description:(running)' RETURN 3 sku name priceCents LIMIT 0 10 DIALECT 2Capture query text, index schema/version, Redis version, dataset size/distribution, and profiler output together. Planner/iterator names are diagnostic implementation details and can change across releases.
5. p50/p95/p99 needs a persistent-client method for serious numbers
The Lesson 2 subprocess harness includes Docker/CLI startup overhead. For more representative query latency, use a maintained persistent client. The following standard-library-only harness talks RESP directly to the loopback server, but authentication and RESP parsing are intentionally minimal for teaching; production code should use redis-py or another maintained client. Because the Chapter 01 ACL requires username/password, the harness sends AUTH, then repeated FT.SEARCH commands on one TCP connection.
import socket, timedef enc(*parts): out = f"*{len(parts)}\r\n".encode() for p in parts: b = str(p).encode() out += f"${len(b)}\r\n".encode() + b + b"\r\n" return outdef read_resp(f): lead = f.read(1) line = f.readline().rstrip(b"\r\n") if lead in (b"+", b"-", b":"): return (lead, line) if lead == b"$": n = int(line); data = f.read(n) if n >= 0 else None if n >= 0: f.read(2) return data if lead == b"*": return [read_resp(f) for _ in range(int(line))] if int(line) >= 0 else None raise RuntimeError((lead, line))with socket.create_connection(("127.0.0.1",6379), timeout=3) as s: f = s.makefile("rb") s.sendall(enc("AUTH","academy-admin","AtlasMart-Admin-Lab-Only-2026")); read_resp(f) cmd = enc("FT.SEARCH","atlasmart-ch09-products-json-idx","@category:{outdoor} @stock>0","NOCONTENT","DIALECT","2") for _ in range(10): s.sendall(cmd); read_resp(f) samples=[] for _ in range(100): t0=time.perf_counter_ns(); s.sendall(cmd); read_resp(f) samples.append((time.perf_counter_ns()-t0)/1_000_000)samples.sort()def pct(p): return samples[min(len(samples)-1, int((p/100)*len(samples))-1)]print({"p50_ms":pct(50),"p95_ms":pct(95),"p99_ms":pct(99),"n":len(samples)})The parser above covers only the RESP types needed by this exercise and should not become an application client. Report network RTT, payload size, warmup, concurrency, Search query, index size, persistence, CPU, and topology with any measured percentile.
6. Measure write amplification separately from query latency
An indexed source-field update can require both source mutation and index maintenance. A non-indexed source-field update can have a different cost. Compare them on representative documents rather than inferring from command complexity alone. Do not use the three-document correctness fixture to estimate production write throughput.
docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app JSON.SET atlasmart:ch09:product:1001 '$.stock' '13'docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app JSON.SET atlasmart:ch09:product:1001 '$.internalExperimentNote' '"not indexed"'docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin FT.INFO atlasmart-ch09-products-json-idxdocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin FT.SEARCH atlasmart-ch09-products-json-idx '@stock:[13 13]' RETURN 2 sku stock DIALECT 2The Search query verifies the indexed-field update became visible through the secondary access path. It does not quantify write cost; Chapter 24 will use a proper workload generator, concurrency, and distributions.
7. Indexing failures are correctness signals, not harmless counters
If a source field cannot be indexed according to its schema—for example a malformed numeric/GEO value—Search can report indexing failures. Monitor failure counts and inspect source/schema mismatch during migrations. A “mostly indexed” catalog is a data-quality incident if omitted documents affect user or business decisions.
Before an index alias/cutover, require expected document count, indexing complete, zero unexplained indexing failures, representative query parity, memory headroom, and rollback path.
8. FT.ALTER adds fields; it is not a universal schema-migration engine
Redis index-management guidance recommends
FT.ALTER when adding fields without a full rebuild.
It cannot remove or modify existing fields. For incompatible
schema changes, create a versioned new index, let it populate,
validate it, then switch an alias. Keep the old index until
rollback criteria expire.
docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin FT.ALTER atlasmart-ch09-products-json-idx SCHEMA ADD '$.rating' AS rating NUMERIC SORTABLEdocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app JSON.SET atlasmart:ch09:product:1001 '$.rating' '4.8'docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin FT.SEARCH atlasmart-ch09-products-json-idx '@rating:[4.5 5]' RETURN 2 sku rating DIALECT 2The new sortable field adds memory. Adding it does not validate or migrate old source documents into a new application schema; source evolution and index evolution remain separate.
9. Versioned indexes and aliases give cleaner rollback for incompatible changes
When field types, analyzers, prefixes, or removed fields change,
build v2 alongside v1. Validate
counts/quality/latency, then point a stable alias to the chosen
version with
FT.ALIASADD/FT.ALIASUPDATE. Alias
behavior and commands are operational features, not tenant
security boundaries.
Keep enough capacity for parallel indexes during migration. Record the source schema version, index definition, creation time, readiness metrics, and application version compatible with each alias target.
10. Direct key access is usually the simplest path when the key is known
If AtlasMart already has
atlasmart:ch09:product:1001, a direct
JSON.GET is simpler than asking Search to
rediscover the document. Search earns its cost when the
application does not know keys and needs text, structured, geo,
aggregation, or later vector retrieval. Do not route every read
through Search for architectural uniformity.
docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app JSON.GET atlasmart:ch09:product:1001 '$.name' '$.priceCents'docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin FT.SEARCH atlasmart-ch09-products-json-idx '@sku:{SKU-1001}' RETURN 2 name priceCents DIALECT 2Both can return the same business fields. The direct read requires the key; Search requires an index and query. Choose based on caller knowledge and required predicates.
11. Search visibility and source visibility must be reconciled during cutovers
A direct source write succeeding does not by itself prove that
an index build is complete, every source document conformed to
the Search schema, or a newly created index is ready for
traffic. Conversely, dropping an index without
DD should not delete source documents. Migration
tests should compare source counts/fixtures with Search
counts/results explicitly.
docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin FT.DROPINDEX atlasmart-ch09-products-json-idxdocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app JSON.GET atlasmart:ch09:product:1001 '$.sku'docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app TYPE atlasmart:ch09:product:1001
The product remains after the index is dropped.
FT.DROPINDEX ... DD has different destructive
semantics and is intentionally not used in this course cleanup.
12. Security, tenancy, persistence, and topology remain separate design axes
An index can reveal matching keys/content to an authorized Search caller, so Redis ACLs and application authorization need deliberate design. TLS protects transport; it does not decide which product a user may see. Persistence/replication protect/reproduce Redis state but are not tested backups. In Cluster or managed deployments, Search/index placement, query routing, replica/failover behavior, and service limits are topology/product-specific; verify them before carrying standalone assumptions into production.
13. Reproducible capstone lab for Chapter 09
Re-create the source/index, capture FT.INFO, run one explain/profile, execute the persistent latency probe on your machine, perform one indexed and one non-indexed source update, add the rating field, then prove dropping the index leaves source data intact.
| Artifact to record | Why |
|---|---|
| Index definition + FT.INFO | schema, count, readiness, memory/failure evidence |
| Representative query + FT.EXPLAINCLI | logical parser contract |
| Same query + FT.PROFILE | runtime diagnostic evidence |
| p50/p95/p99 with environment | performance distribution, not anecdote |
| Direct JSON.GET after FT.DROPINDEX | proof source and secondary index lifecycles are distinct |
docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app DEL atlasmart:ch09:product:1001 atlasmart:ch09:product:1002 atlasmart:ch09:product:1003 atlasmart:ch09:outside:product:9001docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app DEL atlasmart:ch09:offer:1 atlasmart:ch09:offer:214. Production decision matrix
| Requirement | Direct key | Search | Production note |
|---|---|---|---|
| Known product ID | Best default | Usually unnecessary | Keep the shortest correctness path. |
| Full text / category / range / geo discovery | Insufficient alone | Strong fit | Index only required fields and measure quality/cost. |
| Schema-enforced business object | Application/schema layer | Not provided | Search index is not validation. |
| Tenant authorization | Not sufficient by itself | Not sufficient by itself | Enforce application + Redis security boundaries. |
| Auditable financial aggregate | Authoritative ledger/store | Interactive query only | Do not turn Search aggregates into accounting truth. |
| Vector similarity | Not provided | Chapter 10 extends retrieval | Separate ANN recall from task relevance and authorization. |
Prefer the simplest correct primitive. Search is valuable when its query capability offsets its extra derived state, memory, write cost, operational lifecycle, and client/query complexity.
15. Production judgment
A production Search platform needs explicit SLOs for query latency and indexing readiness, capacity headroom for source + index + replication/persistence + migrations, p99 monitoring, failure counters, index version/alias rollback, workload-specific timeouts, predictable result-size limits, schema/data-quality tests, ACL/tenant controls, restore/rebuild drills, and upgrade regression tests for dialect/analyzer/planner behavior. Do not copy universal index-memory ratios, query timeout values, or “always Search” architecture advice. Measure the actual AtlasMart workload.
16. Summary and bridge to Chapter 10
Chapter 09 established Redis Search as a measurable secondary-index subsystem: explicit source scope, typed fields, language/query dialect semantics, bounded deterministic queries, aggregation pipelines, memory/write cost, profiling, and migration discipline. Chapter 10 adds vector retrieval. The same principle will hold: an approximate nearest-neighbor result is another derived retrieval mechanism, not an authorization decision or proof of task relevance.
Check your understanding
- Is MEMORY USAGE on one product the Search index size?
- What should you do before FT.PROFILE when results are logically surprising?
- Can FT.ALTER remove or change an existing field type?
- Why prefer JSON.GET when the product key is already known?
- Does FT.DROPINDEX necessarily delete source keys?
Review the answers
No. Source-key memory, index-memory components, and total process RSS are different measurements.
Use FT.EXPLAIN/FT.EXPLAINCLI and fixture checks to make sure the query expresses the intended predicate.
No. It is primarily additive; incompatible changes need a versioned new index and controlled cutover.
It is the simpler direct source access path and avoids unnecessary secondary-index query machinery.
No. Without the destructive DD option, the index is removed while source keys remain.
Authoritative references
These references were re-checked for the Redis 8.10.1 course snapshot. Command output and planner details can change between Redis releases, clients, RESP modes, and topologies; prefer the target-version reference when reproducing the lab.