Filtering Vector Search, Multi-Vector/Nested Patterns, Quantization/Compression, and Hardware/Memory Planning

Learning outcomes

1. AtlasMart problem: the unfiltered benchmark passes, production fails

Unfiltered ANN recall is excellent, but production queries always include tenant_id, category, stock region, and visibility filters. A selective filter changes the eligible set and may change how the engine traverses or falls back to exact work. AtlasMart therefore benchmarks the filter distributions users actually send.

2. Efficient filter versus post-filter

A post-filter can retrieve k vector neighbors first and discard ineligible documents afterward, potentially returning fewer than k useful hits. A vector-aware efficient filter constrains search so the engine tries to return k neighbors from the eligible set when enough exist. Exact mechanics differ by engine and selectivity. OpenSearch documents dynamic exact/approximate decisions for supported Lucene/Faiss/JVector filtering; Elasticsearch kNN filters are part of the kNN request/query mechanism.

POST /atlasmart-vector-es-v25/_search
{
  "size":3,
  "knn":{
    "field":"embedding",
    "query_vector":[0.70,0.42,0.04,0.05,0.05,0.55,0.08,0.20],
    "k":3,
    "num_candidates":20,
    "filter":{"bool":{"filter":[
      {"term":{"tenant_id":"tenant-a"}},
      {"term":{"category":"outdoor"}}
    ]}}
  }
}
# Eligible fixture docs are P-1001, P-1002, P-1005.

POST /atlasmart-vector-os-v25/_search
{
  "size":3,
  "query":{"knn":{"embedding":{
    "vector":[0.70,0.42,0.04,0.05,0.05,0.55,0.08,0.20],
    "k":3,
    "filter":{"bool":{"filter":[
      {"term":{"tenant_id":"tenant-a"}},
      {"term":{"category":"outdoor"}}
    ]}}
  }}}
}
# Verify supported filter behavior for the exact engine/version; do not move this to a generic post_filter and assume equivalence.

3. Tenant metadata is not optional

Vector similarity is not authorization. Store and enforce tenant/visibility metadata through roles/DLS/application boundaries appropriate to the product, then use vector filters for retrieval semantics. Do not rely on “different tenants probably have dissimilar vectors.” That is data leakage by design.

4. Multi-vector and nested patterns

Long products, manuals, reviews, or passages often need more than one vector per logical product. Treat each passage vector as an addressable retrieval unit with its own text, model metadata, and position. Elasticsearch supports nested kNN patterns; OpenSearch supports nested vector structures with product-specific query syntax. Validate the returned document/passages semantics, not only vector distance.

product_id: P-2001
passages:
  - passage_id: description
    text: "waterproof shell for alpine rain"
    embedding: [ ... 8 floats ... ]
  - passage_id: care
    text: "wash cold; reactivate DWR"
    embedding: [ ... 8 floats ... ]
embedding_model: atlasmart-product-v1
# Implement with the product's documented nested/multi-vector mapping; one dense_vector value is not an array-of-vectors shortcut.

5. Quantization and compression

Elasticsearch and OpenSearch expose different quantization families and defaults. Do not infer equal recall from similar compression ratios.

6. First-order capacity math

A raw float32 vector uses approximately 4 × dims bytes before document, graph, segment, metadata, native-library, cache, and replication overhead. For 768 dimensions that is about 3072 bytes/vector before overhead. This is a lower-bound planning number, not a node-sizing formula.

raw_vector_bytes = vector_count * dimensions * bytes_per_dimension
replicated_raw_bytes = raw_vector_bytes * (1 + replica_count)

# Then MEASURE, do not guess:
# - graph/index bytes
# - segment/store bytes
# - native/off-heap/vector memory
# - filesystem cache behavior
# - JVM heap and breakers
# - merge/recovery temporary headroom
# - quantized representation + raw vector/rescore overhead

7. Hardware and lifecycle stage

Vector search can stress memory bandwidth, CPU SIMD paths, filesystem cache, native memory, and storage differently from lexical search. Local SSD can help storage-bound modes, but benchmark the actual engine/mode and failure/recovery path. Quantization that makes steady-state search fit in memory can still leave index builds, merges, recovery, and rescoring as peak-resource events.

8. Production judgment

Choose filters, nested layout, and compression together. A tenant filter that is highly selective may make exact scoring practical; a low-selectivity filter at billion-vector scale may favor an ANN engine with efficient in-search filtering. Re-evaluate when tenant sizes, model dimensions, passage counts, or retention change.

References and version checks

• Elastic dense_vector field — dimensions, similarity, exact/script scoring, HNSW and quantized index options.
• Elastic kNN search — approximate versus brute-force kNN, filters, nested vector search, and query forms.
• Elastic sparse_vector field — weighted sparse features, pruning, and specialized query boundaries.
• OpenSearch vector index creation — knn_vector, dimensions, workload modes, and storage choices.
• OpenSearch methods and engines — HNSW/IVF and Lucene/Faiss/NMSLIB/JVector capability boundaries.
• OpenSearch k-NN query — k, filters, and method parameters such as ef_search.
• OpenSearch efficient k-NN filtering — filter-aware exact/approximate behavior.
• OpenSearch vector quantization — byte/binary, scalar, product, and engine-specific compression choices.
• OpenSearch sparse_vector — neural sparse ANN support introduced in OpenSearch 3.3.
• Elastic Stack 9.5.3 release and OpenSearch version history — pinned September/August 2026 baselines.