Date fields with ignore malformed do not ignore object and array values#143533
Merged
parkertimmins merged 7 commits intoelastic:mainfrom Mar 4, 2026
Merged
Conversation
Date and date_nanos fields failed to index documents containing object or array values even when ignore_malformed was enabled. The parser.text() call throws an exception type not caught by the ignore_malformed handler. Handle these tokens explicitly before attempting to parse text. Closes elastic#109539
Collaborator
|
Hi @parkertimmins, I've created a changelog YAML for you. |
Collaborator
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
parkertimmins
commented
Mar 3, 2026
| throw e; | ||
| throw new IllegalArgumentException("Cannot parse object or array as a date value"); | ||
| } | ||
| } else { |
Contributor
Author
There was a problem hiding this comment.
I really hate the indentation forced by spotless here. Unless I'm misunderstanding, this } else { and the subsequent block are indented because || context.parser().currentToken() == XContentParser.Token.START_ARRAY is indented. Looks weird to me though.
Kubik42
approved these changes
Mar 3, 2026
Kubik42
reviewed
Mar 3, 2026
| throw new IllegalArgumentException("Cannot parse object or array as a date value"); | ||
| } | ||
| } else { | ||
| try { |
Contributor
There was a problem hiding this comment.
[nit] can you add a comment into the else clause indicating the date is being treated as a string?
// Otherwise, attempt to parse the date as if its a string
burqen
pushed a commit
to burqen/elasticsearch
that referenced
this pull request
Mar 5, 2026
…es (elastic#143533) Date and date_nanos fields fail to index documents when the field value is an object or array, even with ignore_malformed: true. This happens because parseCreateField falls through to parser.text(), which throws an exception type not in the catch list (IllegalArgumentException | ElasticsearchParseException | DateTimeException | ArithmeticException), so the ignore_malformed handling is bypassed entirely. The fix adds an explicit check for START_OBJECT and START_ARRAY tokens before attempting to parse text. When ignore_malformed is enabled, the malformed value is skipped and the field is marked as ignored. When disabled, a clear IllegalArgumentException is thrown.
jfreden
pushed a commit
to jfreden/elasticsearch
that referenced
this pull request
Mar 5, 2026
…es (elastic#143533) Date and date_nanos fields fail to index documents when the field value is an object or array, even with ignore_malformed: true. This happens because parseCreateField falls through to parser.text(), which throws an exception type not in the catch list (IllegalArgumentException | ElasticsearchParseException | DateTimeException | ArithmeticException), so the ignore_malformed handling is bypassed entirely. The fix adds an explicit check for START_OBJECT and START_ARRAY tokens before attempting to parse text. When ignore_malformed is enabled, the malformed value is skipped and the field is marked as ignored. When disabled, a clear IllegalArgumentException is thrown.
spinscale
pushed a commit
to spinscale/elasticsearch
that referenced
this pull request
Mar 6, 2026
…es (elastic#143533) Date and date_nanos fields fail to index documents when the field value is an object or array, even with ignore_malformed: true. This happens because parseCreateField falls through to parser.text(), which throws an exception type not in the catch list (IllegalArgumentException | ElasticsearchParseException | DateTimeException | ArithmeticException), so the ignore_malformed handling is bypassed entirely. The fix adds an explicit check for START_OBJECT and START_ARRAY tokens before attempting to parse text. When ignore_malformed is enabled, the malformed value is skipped and the field is marked as ignored. When disabled, a clear IllegalArgumentException is thrown.
This was referenced Mar 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Date and date_nanos fields fail to index documents when the field value is an object or array, even with ignore_malformed: true. This happens because parseCreateField falls through to parser.text(), which throws an exception type not in the catch list (IllegalArgumentException | ElasticsearchParseException | DateTimeException | ArithmeticException), so the ignore_malformed handling is bypassed entirely. The fix adds an explicit check for START_OBJECT and START_ARRAY tokens before attempting to parse text. When ignore_malformed is enabled, the malformed value is skipped and the field is marked as ignored. When disabled, a clear IllegalArgumentException is thrown.
Closes #109539