-
Notifications
You must be signed in to change notification settings - Fork 25.8k
Response filtering using negative filter_path is removing too many fields by skipping empty arrays #63842
Description
Elasticsearch version 8.0 snapshot
Description of the problem including expected versus actual behavior:
I am trying to reduce the JSON payload from ES to Kibana by up to 50%. One part of this is that I'm trying to use a negative filter_path parameter to exclude one particular path, -**.key_as_string. The bug is that the response filtering is not as targeted as it should be, and it's removing other empty fields.
Steps to reproduce:
This reproduction works for any index, and it will exclude the hits: [] response property which I was expecting to have.
a. Query showing full response
curl -u elastic -X POST http://localhost:9200/kibana_sample_data_logs/_search -H 'Content-Type: application/json' -d '{
"size": 0
}'
{"took":0,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":10000,"relation":"gte"},"max_score":null,"hits":[]}}
b. Same query with response filtering
curl -u elastic -X POST http://localhost:9200/kibana_sample_data_logs/_search?filter_path=**,-**.key_as_string -H 'Content-Type: application/json' -d '{
"size": 0
}'
{"took":2,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":10000,"relation":"gte"},"max_score":null}}
Notice that the hits: [] is no longer returned.