Pinging @elastic/es-search (Team:Search)

One consideration @cbuescher raised is that inspecting index stats in this way will have some inaccuracies when it comes to deleted docs. The worst-case scenario is that an index with a rogue multi-valued doc that was changed to be single-valued will still report the whole index as multi-valued until the old deleted doc is merged out.

Performance feedback

@original-brownbear was kind enough to perform some benchmarking on a large installation (15k shards) and was not able to measure any performance difference on the field caps api between this PR and previous versions.

@original-brownbear was kind enough to perform some benchmarking on a large installation (15k shards) and was not able to measure any performance difference on the field caps api between this PR and previous versions.

Out of curiosity how many mapped fields per shard did the benchmark have, and did the index have actual data for each of these fields?

Out of curiosity how many mapped fields per shard did the benchmark have, and did the index have actual data for each of these fields?

This was Beats indices (mix of Audit, Metric- etc.) so I'd say on average ~2k fields per index.

index have actual data for each of these fields?

This was benchmarked using a variation of the Observability logging track. I do not believe that there was data present for all fields. Looking at the code again now that you mention it (sorry completely missed checking the number field mapper implementation here), whether or not we have data (and also how much of it?) seems to be massively important here.
We don't have a benchmark ready to go for this kind of thing right now (and it's not entirely trivial to set up because of the raw amount of data it would have to index) but maybe it's time to set that up to be safe here?

Pinging @elastic/es-search-foundations (Team:Search Foundations)

Closing here since in this form the PR is outdated.