Emulate "fields" option on older versions #77749
Merged
cbuescher merged 42 commits intoelastic:7.xfrom Sep 21, 2021
Merged
Conversation
We introduced the new "fields" option in search with version 7.10. With this change we are trying to do a best-effort attempt at emulating this new behaviour in mixed cluster or CCS scenarios where search requests using the "fields" option also target older nodes or clusters. In that case, currently we don't return anything in the "fields" section of the response. This change tried to emulate the fields behaviour by modifying the request to include the respective "_source" fields and then parsing them back into the "fields" section on return. This will not be fully equivalent to the post-7.10 "fields" functionality but at least try to include whatever we find in "_source" in earlier versions. Currently Draft only, needs more testing in CCS scenarios but I'm opening this here already to get some test coverage on the modifications so far.
added 9 commits
September 10, 2021 16:29
Collaborator
|
Pinging @elastic/es-search (Team:Search) |
cbuescher
commented
Sep 15, 2021
| * use the parameter on the rest request. The bwc emulation will not be necessary on 8.0 any more, | ||
| * so this setter will not be availabe there. | ||
| */ | ||
| @Deprecated |
Member
Author
There was a problem hiding this comment.
I added this to warn that this option will be gone in 8.0 already. Unfortunately we need a public setter in the RestSearchAction when parsing the parameter from the request because the classes are in different packages.
jimczi
approved these changes
Sep 20, 2021
Contributor
jimczi
left a comment
There was a problem hiding this comment.
LGTM
Also the emulation behaviour will be limited to searches using the
default query_then_fetch search type
Can we add a validation and throw an error if dfs is used ?
Member
Author
|
@elasticmachine run elasticsearch-ci/part-2 |
This was referenced Oct 4, 2021
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We introduced the "fields" option in search with version 7.10. With this
change we are trying to do a best-effort attempt at emulating this option
in mixed cluster or CCS scenarios where older nodes or clusters are targeted.
This change emulates the fields behaviour by modifying the search request
to include the respective parts of the "_source" field and then parses them back
into the "fields" section returned with the response. This will not be fully equivalent
to the post-7.10 "fields" functionality (e.g. we cannot get multi-field values which are
not part of “_source” from pre-7.10 nodes), but will include values present in “_source”
In order to use the emulation, clients need to set the
enable_fields_emulationflag onthe search request. Also the emulation behaviour will be limited to searches using the
default
query_then_fetchsearch type. If both the “fields” option and source filteringare mixed in the same search request, we will throw an error.
This PR is a continuation of #75745 with changes to the feature flag serialization.