Skip to content

Docvalueformat errors#73121

Merged
not-napoleon merged 6 commits intoelastic:masterfrom
not-napoleon:docvalueformat-errors
Jun 7, 2021
Merged

Docvalueformat errors#73121
not-napoleon merged 6 commits intoelastic:masterfrom
not-napoleon:docvalueformat-errors

Conversation

@not-napoleon
Copy link
Copy Markdown
Member

Resolves #72040

When an aggregation gets run across multiple indices with conflicting mappings (e.g. one maps the field being aggregated as an IP address and another maps it as a keyword), the situation can go undetected until the formatting stage. Formatting can then fail with an unhelpful error (e.g. Array index out of bounds). This PR catches many such errors and wraps them with a clearer error message, which will hopefully steer users towards a resolution.

@not-napoleon
Copy link
Copy Markdown
Member Author

@elasticmachine update branch

@not-napoleon not-napoleon added >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v7.14.0 v8.0.0 :Analytics/Aggregations Aggregations labels May 18, 2021
@not-napoleon not-napoleon marked this pull request as ready for review May 18, 2021 16:37
@elasticmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-analytics-geo (Team:Analytics)

Copy link
Copy Markdown
Member

@nik9000 nik9000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me.

can resolve the issue by coercing the unmapped field into the correct type.

[source,console,id=terms-aggregation-value_type-example]
--------------------------------------------------
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Four - is enough for these. I've been trying to stick to it out of sheer stubbornness.

@not-napoleon
Copy link
Copy Markdown
Member Author

@elasticmachine update branch

@not-napoleon not-napoleon merged commit 2d4d3d4 into elastic:master Jun 7, 2021
@not-napoleon not-napoleon deleted the docvalueformat-errors branch June 7, 2021 19:24
not-napoleon added a commit that referenced this pull request Jun 7, 2021
Improve the error message when inconsistent mappings cause doc value formatting errors.  For example, trying to format a binary encoded IP address as a UTF8 string often fails with something unexpected, like `ArrayIndexOutOfBounds`.  This change catches that and wraps it with a message suggesting the user check their mappings.  Also gets rid of anonymous instances for doc value formatters, which made it hard to see what format was failing to be applied.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/Aggregations Aggregations >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v7.14.0 v8.0.0-alpha1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Using IP in date histogram aggregate causes array_index_out_of_bounds_exception

4 participants