Is your feature request related to a problem? Please describe

There are 2 settings related to field data cache size:

indices.breaker.fielddata.limit (default 40% of heap). Incoming queries that would cause the FD cache to grow larger than this size cause a CircuitBreakingException.
indices.fielddata.cache.size (default -1, aka no maximum). Once the FD cache is larger than this, entries are evicted in LRU order.

This is confusing behavior. In general if a cache grows too large, as a user I'd just expect entries to be evicted, not for future queries to be blocked.

I imagine the intention here was to prevent execution for queries that construct single field data entries that are very large, like ~40% of the cache. That seems reasonable to me, but the same effect is triggered even for many small field data entries, which in my opinion should cause eviction instead.

(The returned error is also confusing because to me it reads like the data for that particular query is too large, even though the actual problem is the size of the whole cache: CircuitBreakingException[[fielddata] Data too large, data for [text42] would be [209716286/200mb], which is larger than the limit of [209715200/200mb])

Describe the solution you'd like

We can allow normal eviction for small FD entries, while keeping the behavior of preventing queries that would create extremely large FD entries, by setting the default for indices.fielddata.cache.size to a few percentage points lower than indices.breaker.fielddata.limit. For example, cache.size = 35% and fielddata.limit = 40% seems reasonable. Then if the cache is near 35% of heap, incoming requests that would create >5% of heap worth of field data would be blocked, but smaller ones can continue to run and just evict least-recently-used entries.

Related component

Search

Describe alternatives you've considered

No response

Additional context

No response