Skip to content

[DiskBBQ] avoid EsAcceptDocs bug by calling cost before building iterator#138127

Merged
iverase merged 1 commit intoelastic:mainfrom
iverase:avoidAcceptDocsBug
Nov 15, 2025
Merged

[DiskBBQ] avoid EsAcceptDocs bug by calling cost before building iterator#138127
iverase merged 1 commit intoelastic:mainfrom
iverase:avoidAcceptDocsBug

Conversation

@iverase
Copy link
Copy Markdown
Contributor

@iverase iverase commented Nov 15, 2025

Nightly benchmarks are failing with the following error:

Caused by: java.lang.IllegalStateException: This operation only works with an unpositioned iterator, got current position = 2147483647
        Suppressed: java.lang.IllegalStateException: This operation only works with an unpositioned iterator, got current position = 2147483647

This happens whenever we call EsAcceptDocs#iterator and then EsAcceptDocs#bits and the underlaying structure is not a bitset.

More over if you try the combination EsAcceptDocs#bits and then EsAcceptDocs#iterator you get a different error:

 Exception in thread "main" java.lang.IllegalArgumentException: cost must be >= 0, got -1
        at org.apache.lucene.util.BitSetIterator.<init>(BitSetIterator.java:61)

Therefore in order to run those two methods you need to call EsAcceptDocs#cost. I am hacking this in this PR so we can get some nightly benchmarks data points.

@elasticsearchmachine elasticsearchmachine added the Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch label Nov 15, 2025
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-search-relevance (Team:Search Relevance)

@iverase iverase merged commit e34cc11 into elastic:main Nov 15, 2025
34 checks passed
@iverase iverase deleted the avoidAcceptDocsBug branch November 15, 2025 13:38
weizijun added a commit to weizijun/elasticsearch that referenced this pull request Nov 16, 2025
* main: (135 commits)
  Mute org.elasticsearch.upgrades.IndexSortUpgradeIT testIndexSortForNumericTypes {upgradedNodes=1} elastic#138130
  Mute org.elasticsearch.upgrades.IndexSortUpgradeIT testIndexSortForNumericTypes {upgradedNodes=2} elastic#138129
  Mute org.elasticsearch.search.basic.SearchWithRandomDisconnectsIT testSearchWithRandomDisconnects elastic#138128
  [DiskBBQ] avoid EsAcceptDocs bug by calling cost before building iterator (elastic#138127)
  Log NOT_PREFERRED shard movements (elastic#138069)
  Improve bulk loading of binary doc values (elastic#137995)
  Add internal action for getting inference fields and inference results for those fields (elastic#137680)
  Address issue with DateFieldMapper#isFieldWithinQuery(...) (elastic#138032)
  WriteLoadConstraintDecider: Have separate rate limiting for canRemain and canAllocate decisions (elastic#138067)
  Adding NodeContext to TransportBroadcastByNodeAction (elastic#138057)
  Mute org.elasticsearch.simdvec.ESVectorUtilTests testSoarDistanceBulk elastic#138117
  Mute org.elasticsearch.xpack.esql.qa.single_node.GenerativeIT test elastic#137909
  Backport batched_response_might_include_reduction_failure version to 8.19 (elastic#138046)
  Add summary metrics for tdigest fields (elastic#137982)
  Add gp-llm-v2 model ID and inference endpoint (elastic#138045)
  Various tracing fixes (elastic#137908)
  [ML] Fixing KDE evaluate() to return correct ValueAndMagnitude object (elastic#128602)
  Mute org.elasticsearch.xpack.shutdown.NodeShutdownIT testStalledShardMigrationProperlyDetected elastic#115697
  [ML] Fix Flaky Audit Message Assertion in testWithDatastream for RegressionIT and ClassificationIT (elastic#138065)
  [ML] Fix Non-Deterministic Training Set Selection in RegressionIT testTwoJobsWithSameRandomizeSeedUseSameTrainingSet (elastic#138063)
  ...

# Conflicts:
#	rest-api-spec/src/yamlRestTest/resources/rest-api-spec/test/search.vectors/200_dense_vector_docvalue_fields.yml
elasticsearchmachine pushed a commit that referenced this pull request Nov 17, 2025
This commit fixes the computation of docs per centroid added in
#138127.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>non-issue :Search Relevance/Vectors Vector search Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch v9.3.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants