Disable OCC in update/delete-by-query for seq_no-less indices#143465
Merged
fcofdez merged 10 commits intoelastic:mainfrom Mar 5, 2026
Merged
Disable OCC in update/delete-by-query for seq_no-less indices#143465fcofdez merged 10 commits intoelastic:mainfrom
fcofdez merged 10 commits intoelastic:mainfrom
Conversation
When an index has `index.disable_sequence_numbers` enabled, the update-by-query and delete-by-query actions now skip setting ifSeqNo/ifPrimaryTerm on bulk write requests and stop requesting seq_no in the search source. A new resolver validates that all target indices (or their parent data streams) agree on the setting, rejecting mixed configurations. Within a single data stream, if any backing index disables seq_no the entire stream is treated as seq_no-disabled.
Collaborator
|
Pinging @elastic/es-distributed (Team:Distributed) |
Collaborator
|
Hi @fcofdez, I've created a changelog YAML for you. |
19 tasks
tlrx
reviewed
Mar 4, 2026
modules/reindex/src/main/java/org/elasticsearch/reindex/AsyncDeleteByQueryAction.java
Outdated
Show resolved
Hide resolved
modules/reindex/src/main/java/org/elasticsearch/reindex/BulkByScrollOCCResolver.java
Show resolved
Hide resolved
| Map<String, Boolean> seqNoDisabledPerSource = new HashMap<>(); | ||
| for (Index concreteIndex : concreteIndices) { | ||
| IndexMetadata indexMetadata = projectMetadata.index(concreteIndex); | ||
| boolean seqNoDisabled = hasSequenceNumbersDisabled(indexMetadata); |
Member
There was a problem hiding this comment.
I have the impression that we could early exit in this for loop (maybe with 2 booleans: "anySeqNoDisabled" and "anySeqNoEnabled" or something along those lines and check if they are both true).
I'm mentioning this because it's not unusual to resolve hundreds of concrete indices in requests...
Contributor
Author
There was a problem hiding this comment.
The issue is that if the index expression is targeting multiple data streams, you kind of need to evaluate all the indices before doing the validation, otherwise you risk throwing an exception too early, i.e. if DS-2 latest index is the only one with seq_no disabled. I took this route because it's likely that a data stream would have a mix of seq_no enabled and disabled and we want to accept that scenario.
Contributor
Author
There was a problem hiding this comment.
But happy to change this if we think that there are other ways to optimize this.
Member
There was a problem hiding this comment.
Makes sense, let's keep it as it is.
modules/reindex/src/main/java/org/elasticsearch/reindex/TransportDeleteByQueryAction.java
Outdated
Show resolved
Hide resolved
…ez/elasticsearch into disable-occ-update-delete-by-query
tlrx
approved these changes
Mar 4, 2026
| Map<String, Boolean> seqNoDisabledPerSource = new HashMap<>(); | ||
| for (Index concreteIndex : concreteIndices) { | ||
| IndexMetadata indexMetadata = projectMetadata.index(concreteIndex); | ||
| boolean seqNoDisabled = hasSequenceNumbersDisabled(indexMetadata); |
Member
There was a problem hiding this comment.
Makes sense, let's keep it as it is.
...src/internalClusterTest/java/org/elasticsearch/index/reindex/BulkByScrollSeqNoSettingIT.java
Show resolved
Hide resolved
...src/internalClusterTest/java/org/elasticsearch/index/reindex/BulkByScrollSeqNoSettingIT.java
Outdated
Show resolved
Hide resolved
...src/internalClusterTest/java/org/elasticsearch/index/reindex/BulkByScrollSeqNoSettingIT.java
Show resolved
Hide resolved
tlrx
reviewed
Mar 5, 2026
...src/internalClusterTest/java/org/elasticsearch/index/reindex/BulkByScrollSeqNoSettingIT.java
Show resolved
Hide resolved
Contributor
Author
|
@elasticmachine update branch |
jfreden
pushed a commit
to jfreden/elasticsearch
that referenced
this pull request
Mar 5, 2026
…c#143465) When an index has `index.disable_sequence_numbers` enabled, the update-by-query and delete-by-query actions now skip setting ifSeqNo/ifPrimaryTerm on bulk write requests and stop requesting seq_no in the search source. A new resolver validates that all target indices (or their parent data streams) agree on the setting, rejecting mixed configurations. Within a single data stream, if any backing index disables seq_no the entire stream is treated as seq_no-disabled. Related to elastic#136305
spinscale
pushed a commit
to spinscale/elasticsearch
that referenced
this pull request
Mar 6, 2026
…c#143465) When an index has `index.disable_sequence_numbers` enabled, the update-by-query and delete-by-query actions now skip setting ifSeqNo/ifPrimaryTerm on bulk write requests and stop requesting seq_no in the search source. A new resolver validates that all target indices (or their parent data streams) agree on the setting, rejecting mixed configurations. Within a single data stream, if any backing index disables seq_no the entire stream is treated as seq_no-disabled. Related to elastic#136305
This was referenced Mar 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When an index has
index.disable_sequence_numbersenabled,the update-by-query and delete-by-query actions now skip setting
ifSeqNo/ifPrimaryTerm on bulk write requests and stop requesting seq_no
in the search source. A new resolver validates that all target indices
(or their parent data streams) agree on the setting, rejecting mixed configurations.
Within a single data stream, if any backing index disables seq_no
the entire stream is treated as seq_no-disabled.
Related to #136305