Skip to content

Response filtering using negative filter_path is removing too many fields by skipping empty arrays #63842

@wylieconlon

Description

@wylieconlon

Elasticsearch version 8.0 snapshot

Description of the problem including expected versus actual behavior:

I am trying to reduce the JSON payload from ES to Kibana by up to 50%. One part of this is that I'm trying to use a negative filter_path parameter to exclude one particular path, -**.key_as_string. The bug is that the response filtering is not as targeted as it should be, and it's removing other empty fields.

Steps to reproduce:

This reproduction works for any index, and it will exclude the hits: [] response property which I was expecting to have.

a. Query showing full response

curl -u elastic -X POST http://localhost:9200/kibana_sample_data_logs/_search -H 'Content-Type: application/json' -d '{
  "size": 0
}'

{"took":0,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":10000,"relation":"gte"},"max_score":null,"hits":[]}}

b. Same query with response filtering

curl -u elastic -X POST http://localhost:9200/kibana_sample_data_logs/_search?filter_path=**,-**.key_as_string -H 'Content-Type: application/json' -d '{
  "size": 0
}'

{"took":2,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":10000,"relation":"gte"},"max_score":null}}

Notice that the hits: [] is no longer returned.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions