Chapter 10 · Vector Sets, Vector Search, Hybrid Retrieval, and AI Workloads
Hybrid Retrieval with Structured Filters and Vector Similarity
Combine vector similarity with structured constraints without leaking unauthorized candidates or confusing score semantics.
Learning outcomes
AtlasMart now needs “similar outdoor products for tenant-a that are active and below a price ceiling.” Similarity alone is insufficient; structured constraints must participate in retrieval before results are exposed.
Build hybrid retrieval where semantic similarity and structured constraints are both explicit.
Compare Vector Set FILTER with Redis Search metadata predicates plus KNN.
Keep tenant/security filtering inside the retrieval boundary.
Explain pre-filter/post-filter tradeoffs and why top-k after filtering can change recall.
Verify Search vector index metadata and dimension/metric contracts before querying.
All Chapter 10 mandatory labs reuse the disposable Chapter 01
environment: Redis Open Source 8.10.1 from Docker
Official Image redis:8.10.1, container
atlasmart-redis-ch01, standalone topology, host
publication 127.0.0.1:6379, TLS disabled only
because traffic stays on loopback, default ACL user disabled,
named users atlasmart-app and
academy-admin, logical database 0, AOF with
appendfsync everysec plus RDB snapshots,
persistent /data, and no explicit
maxmemory limit or eviction policy. Redis 8
integrates Vector Sets and the Redis Query Engine into Redis
Open Source. Mandatory examples use synthetic numeric vectors
created locally—Redis stores/searches vectors but does not
generate embeddings. Fixtures stay under
atlasmart:ch10:*. Vector Set commands use the
restricted application user where allowed; Search index
administration uses the disposable
academy-admin user. No paid embedding API,
managed service, production endpoint, or real credential is
required.
Redis 8.0 introduced Vector Sets as a beta data type. The
current Redis Open Source 8.10 command reference documents
VADD, VSIM, VINFO,
filtering, quantization, and related commands as available
since 8.0, with standard Redis Software/Redis Cloud
compatibility. The official sources checked for this lesson do
not provide a separate explicit “Vector Sets became GA on
version X” declaration. Treat Vector Set API/product status,
client coverage, managed-service support, and Active-Active
compatibility as version-sensitive and verify the exact target
rather than inventing a GA date.
1. Hybrid retrieval means two relevance systems cooperate
Hybrid retrieval combines vector similarity with structured or textual predicates. Vector similarity answers “what is nearby in embedding space?” Structured predicates answer “what is allowed/eligible?” These concerns are related but not interchangeable.
| Constraint | Best represented as | Reason |
|---|---|---|
| tenant | TAG/attribute/security scope | exact authorization identity |
| active status | TAG/boolean-like attribute | exact eligibility |
| price ceiling | NUMERIC/attribute predicate | ordered structured constraint |
| semantic intent | vector similarity | continuous representation from embedding model |
| keyword phrase | TEXT | lexical evidence, not vector geometry |
2. Vector Set FILTER: lightweight in-structure hybrid retrieval
docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app DEL atlasmart:ch10:hybrid:vsetdocker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app VADD atlasmart:ch10:hybrid:vset VALUES 3 1 0 0 p1001 SETATTR '{"tenant":"tenant-a","category":"outdoor","active":true,"priceCents":12990}'docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app VADD atlasmart:ch10:hybrid:vset VALUES 3 0.95 0.10 0.02 p1002 SETATTR '{"tenant":"tenant-b","category":"outdoor","active":true,"priceCents":10990}'docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app VADD atlasmart:ch10:hybrid:vset VALUES 3 0.88 0.20 0.05 p1003 SETATTR '{"tenant":"tenant-a","category":"outdoor","active":true,"priceCents":9990}'docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app VSIM atlasmart:ch10:hybrid:vset VALUES 3 0.99 0.05 0.01 WITHSCORES WITHATTRIBS COUNT 10 FILTER '.tenant == "tenant-a" && .active == true && .priceCents <= 12000'The most similar overall candidate might belong to tenant-b or exceed price. Correct hybrid retrieval returns the best candidates from the allowed subset, not “global top-k then hide disallowed rows.”
3. Why post-filtering top-k can leak and under-fill
Suppose global top-3 contains two tenant-b products and one tenant-a product. If your application fetches those IDs and removes tenant-b afterward, it already observed unauthorized identifiers and returns only one result even if many valid tenant-a neighbors ranked 4–20. Retrieval-time filtering addresses both security exposure and candidate under-fill.
A tenant predicate is not a relevance preference. Treat it as mandatory eligibility and test that unfiltered retrieval would contain adversarial cross-tenant fixtures.
4. Search vector fields support richer hybrid predicates
When AtlasMart needs vector KNN plus TEXT/TAG/NUMERIC/GEO logic over JSON/Hash documents, use a Redis Search vector field. The source record remains a JSON/Hash key; the vector field is one indexed attribute among others.
docker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin FT.DROPINDEX atlasmart-ch10-hybrid-idx # ignore unknown-index error on first rundocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin FT.CREATE atlasmart-ch10-hybrid-idx ON JSON PREFIX 1 atlasmart:ch10:doc: SCHEMA '$.tenant' AS tenant TAG '$.category' AS category TAG '$.active' AS active TAG '$.priceCents' AS price NUMERIC '$.embedding' AS embedding VECTOR HNSW 6 TYPE FLOAT32 DIM 3 DISTANCE_METRIC COSINEdocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin FT.INFO atlasmart-ch10-hybrid-idxThe schema evidence should show HNSW, FLOAT32, DIM 3, and COSINE for the vector field. JSON vector ingestion details and query parameters must match the exact Redis release/client. The mandatory Vector Set path remains simpler and fully redis-cli friendly.
5. Search KNN returns distance; Vector Set WITHSCORES returns similarity
Do not compare raw numbers from two APIs without understanding
their convention. Vector Set
VSIM WITHSCORES currently reports similarity where
1 is identical. Search KNN exposes a vector score/distance field
where ordering follows the configured distance metric and lower
distance is generally closer. Name metrics explicitly in
telemetry.
| Surface | Typical returned quantity | Interpretation |
|---|---|---|
| VSIM WITHSCORES | similarity 1→0 | higher is more similar |
| Search KNN alias | distance | lower is closer |
| Business reranker | task-specific score | definition owned by application/model |
6. Exact structured filters do not make semantic retrieval “correct”
A query can be perfectly tenant-safe and still retrieve semantically poor products because the embedding model is wrong for the domain. Conversely, a great embedding can violate business constraints if filters are missing. Track security/eligibility correctness, ANN recall, and task relevance as separate dimensions.
7. Candidate depth and filter selectivity interact
Highly selective filters can require more search effort to find
enough eligible neighbors. Vector Set provides
FILTER-EF to bound filtering attempts, and Search
vector queries have their own runtime/planner controls. Measure
under realistic selectivity distributions rather than one
happy-path query.
docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app VSIM atlasmart:ch10:hybrid:vset VALUES 3 0.99 0.05 0.01 COUNT 3 EF 50 FILTER '.tenant == "tenant-a"' FILTER-EF 100docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app VSIM atlasmart:ch10:hybrid:vset VALUES 3 0.99 0.05 0.01 COUNT 3 EF 200 FILTER '.tenant == "tenant-a"' FILTER-EF 1000Do not assume the larger effort is always better; compare recall, latency, and CPU impact.
8. Key/tenant isolation is stronger than filter-only isolation
If tenant data is highly sensitive, one shared vector structure with filters may be unacceptable even if queries are correctly written. Alternatives include per-tenant keys/index prefixes, database/process isolation, or separate managed databases depending on threat model and scale. Redis logical databases are not a security isolation boundary, and Cluster supports only DB 0.
9. Hybrid evaluation fixture
candidates=[ {"id":"p1","tenant":"a","active":True,"sim":0.97,"relevant":1}, {"id":"p2","tenant":"b","active":True,"sim":0.99,"relevant":1}, {"id":"p3","tenant":"a","active":True,"sim":0.91,"relevant":0}, {"id":"p4","tenant":"a","active":False,"sim":0.96,"relevant":1},]eligible=[x for x in candidates if x["tenant"]=="a" and x["active"]]print([x["id"] for x in sorted(eligible,key=lambda x:x["sim"], reverse=True)])# Then score task relevance only over the authorized/eligible result set.10. Wrong approach: query all tenants, then post-filter in UI
This can leak identifiers/scores through logs, traces, caches, errors, or client memory and can return fewer than k eligible neighbors. Repair by enforcing tenant eligibility before results leave Redis/retrieval service, testing adversarial fixtures, and designing ACL/key isolation consistent with the threat model.
11. Reproducible cleanup
docker exec -e REDISCLI_AUTH=AtlasMart-App-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user atlasmart-app DEL atlasmart:ch10:hybrid:vsetdocker exec -e REDISCLI_AUTH=AtlasMart-Admin-Lab-Only-2026 atlasmart-redis-ch01 redis-cli --user academy-admin FT.DROPINDEX atlasmart-ch10-hybrid-idx
If the Search index was not created,
FT.DROPINDEX can report an unknown-index error;
that is harmless in the disposable cleanup.
12. Production judgment
Choose Vector Set FILTER for compact similarity workflows with lightweight attributes; choose Redis Search when hybrid predicates, document schemas, text, geo, sorting, or richer query planning matter. Keep tenant constraints non-optional, benchmark selectivity distributions, record top-k under-fill, use deterministic sort/tie handling where needed, and instrument both similarity/distance semantics. Cluster and managed-service behavior can alter routing and candidate fan-out; validate the exact topology.
13. Summary and next step
Hybrid retrieval is not “vector search plus a WHERE clause.” It is a correctness boundary where semantic candidates meet exact eligibility and security. Lesson 4 quantifies another boundary: memory and accuracy changes from quantization, dimensionality reduction, and graph tuning.
Check your understanding
- Why is tenant post-filtering risky?
- When is Vector Set FILTER a good fit?
- When is Redis Search preferable?
- Are VSIM scores and Search KNN distances numerically interchangeable?
- What should be evaluated besides ANN recall?
Review the answers
Unauthorized candidates can leave the retrieval boundary and top-k can under-fill.
When lightweight JSON attributes and similarity are enough.
When richer structured/text/geo predicates or document indexing is needed.
No; one is similarity-oriented and the other commonly distance-oriented.
Eligibility/security correctness and task relevance.
Authoritative references
- Redis Vector Sets — native Vector Set data type, commands, filtering, and examples
- VADD — Vector Set insertion, quantization, REDUCE, EF, M, and attributes
- VSIM — similarity queries, scores, filters, EF, TRUTH, and NOTHREAD
- VINFO — Vector Set implementation and configuration evidence
- VEMB — stored/reconstructed vector evidence
- VSETATTR — JSON attributes attached to Vector Set elements
- Vector Set memory optimization — Q8/BIN/NOQUANT, dimensions, graph links, and memory tradeoffs
- Vector Set performance — quantization and vector-set performance considerations
- Redis vector search concepts — Search FLAT/HNSW/SVS-VAMANA vector indexes and runtime parameters
- Vector field options — Search vector field types, metrics, and index algorithms
- FT.CREATE — Search schema and vector field creation
- FT.SEARCH — KNN/hybrid query syntax and parameters
- Redis 8.10 commands — target-version command surface
- Redis 8.10 release notes — 8.10.1 security baseline including Vector Set fixes
- Redis 8 GA announcement — historical 8.0 Vector Set beta status and integrated Redis 8 capabilities