Query/Request/Fielddata/Shard Request Caches: Eligibility, Invalidation, and Memory Tradeoffs

Learning outcomes

AtlasMart dashboards repeat a size-zero sales aggregation and become very fast after the first run. A product-search page repeats the same lexical filter but still performs work after every refresh. Another engineer enables fielddata on name because a terms aggregation failed, and heap consumption spikes. All three observations mention “cache,” but they are different mechanisms with different eligibility and invalidation rules.

1. A cache taxonomy that prevents category errors

These mechanisms can all improve latency, but they cache different things. Clearing a request cache does not evict the OS page cache. Warming Lucene file pages does not make an ineligible request-cache query eligible. A fielddata problem cannot be repaired by increasing request-cache size.

2. Query cache: filter context, repeated structure, segment lifecycle

In Elasticsearch, the node query cache caches eligible queries used in filter context. Queries used for scoring are not automatically interchangeable with cached filters. Eligibility also depends on repetition and segment characteristics. Because results are tied to segments, merges can invalidate entries.

OpenSearch also exposes query-cache statistics/settings and uses cacheable query/filter execution as an optimization. Treat the exact cache policy as product/version behavior, not an API guarantee that every repeated filter will be cached.

GET _nodes/stats/indices/query_cache?human
GET atlasmart-runtime-lab-v1/_stats/query_cache?human

GET atlasmart-runtime-lab-v1/_search
{
  "size": 0,
  "query": {"bool":{"filter":[
    {"term":{"available":true}},
    {"term":{"category":"electronics"}}
  ]}},
  "aggs":{"price":{"stats":{"field":"price"}}}
}

3. Shard/index request cache: whole-result reuse

Elasticsearch calls this the shard request cache. OpenSearch documentation calls the corresponding feature the index request cache, although the cached result is still shard-local. The cache is especially valuable for repeatable aggregation/dashboard requests because a shard can reuse a prior local result and the coordinating node can reduce those cached shard results.

By default, size-zero aggregation-like requests are the common cacheable case. Product/version rules differ for non-zero hit sizes and non-deterministic constructs. Relative time such as now, profiling, scroll/DFS modes or script nondeterminism can remove or reduce cacheability. Preserve the exact request body when measuring because the body participates in cache identity.

GET atlasmart-runtime-lab-v1/_search?request_cache=true
{
  "size": 0,
  "query":{"term":{"available":true}},
  "aggs":{"by_category":{"terms":{"field":"category","size":10}}}
}

GET atlasmart-runtime-lab-v1/_stats/request_cache?human
GET _nodes/stats/indices/request_cache?human

Repeat without indexing between runs, then index one document and wait for/issue the refresh required by your experiment. Observe that data-changing refresh semantics can invalidate request-cache entries. Do not claim that a cache hit returns stale search data; the cache lifecycle is integrated with index refresh/change visibility.

4. Fielddata and global ordinals: not a generic “make text aggregatable” switch

Analyzed text is optimized for full-text matching through an inverted index. Sorting or terms aggregation needs efficient per-document values. Enabling fielddata on high-cardinality text asks the engine to build an on-heap structure by uninverting text terms, which can consume substantial heap. Both products therefore default fielddata off for text and recommend modeling an exact-value keyword subfield for sorting/aggregation.

Global ordinals map per-segment term ordinals into a shard-wide view for terms-like operations. They can be built lazily after refresh or eagerly when the mapping requests it. Eager construction can move latency from the first aggregation into refresh/indexing work; it is a latency-placement decision, not free speed.

"name": {
  "type": "text",
  "fields": {
    "raw": {"type": "keyword", "ignore_above": 256}
  }
}

# Aggregate/sort on name.raw, search full text on name.
# Do NOT set fielddata:true on name merely to make a demo aggregation pass.

GET _cat/fielddata?v&bytes=mb
GET _nodes/stats/indices/fielddata?human
GET atlasmart-runtime-lab-v1/_stats/fielddata?human

5. Wrong approach: optimize cache hit rate as the goal

A team disables refreshes and canonicalizes every dashboard request until request-cache hit rate is excellent. Unfortunately, users now see stale data beyond the product requirement. Another team increases cache sizes until heap pressure causes GC and breaker risk. Both optimized an internal metric while degrading the service.

Cache metrics are diagnostic inputs. The service objective is correct results within freshness, latency, throughput, cost and failure-isolation constraints. A lower hit rate may be acceptable for highly dynamic queries; a high hit rate is useless if the cached work is cheap or if it displaces more valuable memory.

6. AtlasMart cache experiment

1. Capture _stats/query_cache,request_cache,fielddata and node JVM stats.
2. Run a fixed filter+aggregation request N times without writes; record client latency distribution and cache deltas.
3. Run a semantically similar request with an intentionally non-cacheable/changed element and compare.
4. Index one known document and refresh; rerun the first request and inspect request-cache behavior.
5. Compare aggregation on keyword to the prohibited idea of fielddata on text; do not enable fielddata in the shared fixture.
6. Interpret whether any latency difference is due to logical result caching, filesystem warmness, both, or neither.

7. Elastic vs OpenSearch details that matter

Production judgment

If a cache is the only thing keeping a query within the latency objective, verify what happens after refresh, relocation, node restart and eviction. That is the real miss path users will eventually experience. Capacity planning must include both warm and recovery/cold behavior.

A cache that avoids expensive repeated work can be valuable, but the next lesson covers the safety layer underneath: circuit breakers exist because some requests can allocate more heap than caches and ordinary GC can safely absorb.

Summary and next step

You can now say exactly which cache you mean, what it reuses, when it invalidates and which resource it consumes. Lesson 3 uses that vocabulary to explain circuit breakers, aggregation memory, in-flight requests and why safety limits should not be disabled to make one query succeed.

Authoritative references

• Elastic JVM settings — Automatic heap sizing, filesystem-cache headroom, container memory, and compressed ordinary object pointer guidance.
• Elastic node query cache settings — Filter-context query-cache eligibility, segment-level caching and invalidation behavior.
• Elastic shard request cache — Shard-level request-result caching and request/index controls.
• Elastic field data cache settings — On-heap fielddata/global-ordinal cache behavior and breaker interaction.
• Elastic circuit breaker settings — Parent, fielddata, request and in-flight breaker semantics.
• Elastic thread pool settings — Current thread-pool types, sizing and queue behavior.
• Elastic indexing pressure — Outstanding indexing-byte accounting and rejection behavior.
• Elastic Nodes Stats API — JVM, process, caches, breakers, thread pools and indexing-pressure observations.
• OpenSearch circuit breaker settings — Parent/child breaker settings and failure-protection intent.
• OpenSearch caching overview — On-heap request/query/fielddata caching concepts.
• OpenSearch index request cache — Shard-level request-result caching, invalidation and metrics.
• OpenSearch field data cache — Fielddata/global ordinals and the keyword-subfield alternative.
• OpenSearch thread pool settings — Thread pools, queue sizing and monitoring guidance.
• OpenSearch search backpressure — OpenSearch-specific search task resource tracking and cancellation.
• OpenSearch shard indexing backpressure — OpenSearch-specific per-shard indexing pressure and rejection mechanism.
• OpenSearch Nodes Stats API — JVM, process, caches, indexing pressure and backpressure statistics.
• OpenSearch Nodes Hot Threads API — CPU/wait/block thread samples for diagnosis.