Search - return ignored field values from fields api.#78697
Merged
markharwood merged 9 commits intoelastic:masterfrom Oct 13, 2021
Merged
Search - return ignored field values from fields api.#78697markharwood merged 9 commits intoelastic:masterfrom
markharwood merged 9 commits intoelastic:masterfrom
Conversation
Collaborator
|
Pinging @elastic/es-search (Team:Search) |
b487ac6 to
d8cd701
Compare
jimczi
reviewed
Oct 11, 2021
Contributor
jimczi
left a comment
There was a problem hiding this comment.
Looks good to me, thanks @markharwood . I left some comments regarding the mutation of the ignored values list.
docs/reference/search/search-your-data/retrieve-selected-fields.asciidoc
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/common/document/DocumentField.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/common/document/DocumentField.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/common/document/DocumentField.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/search/fetch/subphase/FetchDocValuesPhase.java
Outdated
Show resolved
Hide resolved
1f82ab3 to
40d23d1
Compare
Contributor
Author
|
@jimczi good to go with this one now? |
jimczi
reviewed
Oct 12, 2021
server/src/main/java/org/elasticsearch/common/document/DocumentField.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/common/document/DocumentField.java
Outdated
Show resolved
Hide resolved
jimczi
approved these changes
Oct 12, 2021
Contributor
jimczi
left a comment
There was a problem hiding this comment.
I left one final comment, LGTM otherwise
server/src/main/java/org/elasticsearch/common/document/DocumentField.java
Outdated
Show resolved
Hide resolved
Since Kibana's Discover switched to retrieving values via the fields API rather than source there have been gaps in the display caused by "ignored" fields (those that fall foul of ignore_above and ignore_malformed size and formatting rules). This PR returns ignored values from source when a user-requested field fails to be parsed for a document. In these cases the corresponding hit adds a new ignored_field_values section in the response. Closes elastic#74121
This reverts commit 1f82ab33d96d9e6ed247f75892362b9faba387cb.
…ctor" This reverts commit 13a5062cfd081d823635eac87f6ed062c126a9f6.
2053c74 to
f2188f7
Compare
jpountz
reviewed
Oct 13, 2021
| [discrete] | ||
| [[Ignored-field values]] | ||
| ==== Ignored field values | ||
| The `fields` option only returns values that were valid when indexed. |
Contributor
There was a problem hiding this comment.
This sentence is a bit confusing since we say that we return invalid values a few lines below? (just under a different path)
Contributor
Author
There was a problem hiding this comment.
How about changing "the fields option..." to "the fields section of the response..."?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Since Kibana's Discover switched to retrieving values via the fields API rather than source there have been gaps in the display caused by "ignored" fields (those that fall foul of ignore_above and ignore_malformed size and formatting rules).
This PR returns ignored values from source when a user-requested field fails to be parsed for a document. In these cases the corresponding hit adds a new ignored_field_values section in the response. The DocumentField class is changed to collect ignored values as well as the valid ones it already did.
Closes #74121