It turns out that sometimes my beautiful date_histogram optimizations aren't always optimizations. Sometimes they make the date_histogram super ultra slow. We've known that and have heuristics that disable the optimization. Those heuristics generally work because, like, we worked on them. But it turns out that they don't work for everyone. Sometimes folks have queries or data or something that doesn't work well with the heuristics we have. I'll be filing a separate issue with the cases we've heard about and fix the heuristics. But I think this is a forever problem. We just won't cover all cases in our tests. So we're going to need an escape hatch to disable the optimization entirely. I believe a cluster level setting is the most sensible thing here. Its a "big hammer", but its one that you can easily modify if you hit the issue without restarting anything and you can disable once you've upgraded to a version that includes the fix.
It turns out that sometimes my beautiful date_histogram optimizations aren't always optimizations. Sometimes they make the
date_histogramsuper ultra slow. We've known that and have heuristics that disable the optimization. Those heuristics generally work because, like, we worked on them. But it turns out that they don't work for everyone. Sometimes folks have queries or data or something that doesn't work well with the heuristics we have. I'll be filing a separate issue with the cases we've heard about and fix the heuristics. But I think this is a forever problem. We just won't cover all cases in our tests. So we're going to need an escape hatch to disable the optimization entirely. I believe a cluster level setting is the most sensible thing here. Its a "big hammer", but its one that you can easily modify if you hit the issue without restarting anything and you can disable once you've upgraded to a version that includes the fix.