pass in order terms as sorted to TermInSetQuery() by mkhludnev · Pull Request #17714 · opensearch-project/OpenSearch

mkhludnev · 2025-03-27T18:50:54Z

Pass in-order terms as Sorted into TermInSet

Check List

Functionality includes testing.
becnhmark. Not yet.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

spin off from discussion https://forum.opensearch.org/t/avoid-re-sorting-when-initializing-terminsetquery/23865

github-actions · 2025-03-27T20:53:15Z

❌ Gradle check result for aa495eb:

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

harshavamsi · 2025-03-28T17:39:22Z

@mkhludnev thanks for adding this change! This looks right, but TermInSetQuery expects that we pass a Collection of ByteRef, but it seems we're using a list here

github-actions · 2025-03-30T08:16:33Z

❌ Gradle check result for 4aba2a0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-30T19:50:08Z

❌ Gradle check result for 5808ce4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-30T21:07:26Z

✅ Gradle check result for 329035a: SUCCESS

codecov · 2025-03-30T21:07:47Z

Codecov Report

Attention: Patch coverage is 98.30508% with 1 line in your changes missing coverage. Please review.

Project coverage is 72.42%. Comparing base (8182bb0) to head (c76663e).
Report is 73 commits behind head on main.

Files with missing lines	Patch %	Lines
...earch/index/mapper/BytesRefsCollectionBuilder.java	97.50%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #17714      +/-   ##
============================================
+ Coverage     72.29%   72.42%   +0.13%     
- Complexity    65900    66031     +131     
============================================
  Files          5350     5351       +1     
  Lines        306185   306229      +44     
  Branches      44373    44374       +1     
============================================
+ Hits         221347   221789     +442     
+ Misses        66670    66306     -364     
+ Partials      18168    18134      -34

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

github-actions · 2025-03-31T07:40:17Z

❌ Gradle check result for ec7707c: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

mkhludnev · 2025-04-03T14:34:27Z

one more optimization in Lucene 10.3 apache/lucene#14425

jainankitk

Thanks @mkhludnev for fixing this issue. Few comments to help improve the readability

server/src/main/java/org/opensearch/index/mapper/BytesRefsCollectionBuilder.java

Signed-off-by: Mikhail Khludnev <mkhl@apache.org>

github-actions · 2025-04-08T14:05:01Z

✅ Gradle check result for c76663e: SUCCESS

opensearch-trigger-bot · 2025-04-11T01:01:15Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-17714-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 032f4095cf797059e718d74fd2d337d95f8a09a9
# Push it to GitHub
git push --set-upstream origin backport/backport-17714-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-17714-to-2.x.

mkhludnev · 2025-04-11T06:44:32Z

@jainankitk thanks for merging this. Let's remove these redundant test #17902 (pardon, my intent wasn't clear), and then I'll proceed with backporting.

Followup for #17714: Remove redundant tests (#17902) * Remove redundant tests Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Fix empty collection test Signed-off-by: Mikhail Khludnev <mkhl@apache.org> --------- Signed-off-by: Mikhail Khludnev <mkhl@apache.org>

…#17714) * pass in order terms as sorted to TermInSetQuery() Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * slightly more elegant solution Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Attempting mocking TermInSetQ constructor. Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Handle ids as well. Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * forbidden api Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * make unnecessary method slow but correct. Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * make unnecessary method slow but correct. Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Polish test coverage Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * CHANGELOG.md Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * assertThrows Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * spotlessApply Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * coverage tests and refactoring Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * javadoc Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * javadoc Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * mark nocommit Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * one more nocommit test Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * forbidden api Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * no commit for out of line tests Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Review Signed-off-by: Mikhail Khludnev <mkhl@apache.org> --------- Signed-off-by: Mikhail Khludnev <mkhl@apache.org> Signed-off-by: Mikhail Khludnev <mkhludnev@users.noreply.github.com>

Followup for opensearch-project#17714: Remove redundant tests (opensearch-project#17902) * Remove redundant tests Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Fix empty collection test Signed-off-by: Mikhail Khludnev <mkhl@apache.org> --------- Signed-off-by: Mikhail Khludnev <mkhl@apache.org>

mkhludnev · 2025-04-12T17:59:26Z

backport pr #17916

…#17714) * pass in order terms as sorted to TermInSetQuery() Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * slightly more elegant solution Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Attempting mocking TermInSetQ constructor. Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Handle ids as well. Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * forbidden api Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * make unnecessary method slow but correct. Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * make unnecessary method slow but correct. Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Polish test coverage Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * CHANGELOG.md Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * assertThrows Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * spotlessApply Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * coverage tests and refactoring Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * javadoc Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * javadoc Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * mark nocommit Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * one more nocommit test Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * forbidden api Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * no commit for out of line tests Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Review Signed-off-by: Mikhail Khludnev <mkhl@apache.org> --------- Signed-off-by: Mikhail Khludnev <mkhl@apache.org> Signed-off-by: Mikhail Khludnev <mkhludnev@users.noreply.github.com> Signed-off-by: Sriram Ganesh <srignsh22@gmail.com>

Followup for opensearch-project#17714: Remove redundant tests (opensearch-project#17902) * Remove redundant tests Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Fix empty collection test Signed-off-by: Mikhail Khludnev <mkhl@apache.org> --------- Signed-off-by: Mikhail Khludnev <mkhl@apache.org> Signed-off-by: Sriram Ganesh <srignsh22@gmail.com>

* Pass in order terms as sorted to TermInSetQuery() (#17714) * pass in order terms as sorted to TermInSetQuery() Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * slightly more elegant solution Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Attempting mocking TermInSetQ constructor. Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Handle ids as well. Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * forbidden api Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * make unnecessary method slow but correct. Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * make unnecessary method slow but correct. Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Polish test coverage Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * CHANGELOG.md Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * assertThrows Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * spotlessApply Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * coverage tests and refactoring Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * javadoc Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * javadoc Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * mark nocommit Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * one more nocommit test Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * forbidden api Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * no commit for out of line tests Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Review Signed-off-by: Mikhail Khludnev <mkhl@apache.org> --------- Signed-off-by: Mikhail Khludnev <mkhl@apache.org> Signed-off-by: Mikhail Khludnev <mkhludnev@users.noreply.github.com> * Fix test Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Fix BytesRefsCollectionBuilderTests.testBuildSortedNotSorted Followup for #17714: Remove redundant tests (#17902) * Remove redundant tests Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Fix empty collection test Signed-off-by: Mikhail Khludnev <mkhl@apache.org> --------- Signed-off-by: Mikhail Khludnev <mkhl@apache.org> --------- Signed-off-by: Mikhail Khludnev <mkhl@apache.org> Signed-off-by: Mikhail Khludnev <mkhludnev@users.noreply.github.com>

…#17714) * pass in order terms as sorted to TermInSetQuery() Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * slightly more elegant solution Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Attempting mocking TermInSetQ constructor. Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Handle ids as well. Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * forbidden api Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * make unnecessary method slow but correct. Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * make unnecessary method slow but correct. Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Polish test coverage Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * CHANGELOG.md Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * assertThrows Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * spotlessApply Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * coverage tests and refactoring Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * javadoc Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * javadoc Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * mark nocommit Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * one more nocommit test Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * forbidden api Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * no commit for out of line tests Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Review Signed-off-by: Mikhail Khludnev <mkhl@apache.org> --------- Signed-off-by: Mikhail Khludnev <mkhl@apache.org> Signed-off-by: Mikhail Khludnev <mkhludnev@users.noreply.github.com> Signed-off-by: Harsh Kothari <techarsh@amazon.com>

Followup for opensearch-project#17714: Remove redundant tests (opensearch-project#17902) * Remove redundant tests Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Fix empty collection test Signed-off-by: Mikhail Khludnev <mkhl@apache.org> --------- Signed-off-by: Mikhail Khludnev <mkhl@apache.org> Signed-off-by: Harsh Kothari <techarsh@amazon.com>

…#17714) * pass in order terms as sorted to TermInSetQuery() Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * slightly more elegant solution Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Attempting mocking TermInSetQ constructor. Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Handle ids as well. Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * forbidden api Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * make unnecessary method slow but correct. Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * make unnecessary method slow but correct. Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Polish test coverage Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * CHANGELOG.md Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * assertThrows Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * spotlessApply Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * coverage tests and refactoring Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * javadoc Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * javadoc Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * mark nocommit Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * one more nocommit test Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * forbidden api Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * no commit for out of line tests Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Review Signed-off-by: Mikhail Khludnev <mkhl@apache.org> --------- Signed-off-by: Mikhail Khludnev <mkhl@apache.org> Signed-off-by: Mikhail Khludnev <mkhludnev@users.noreply.github.com> Signed-off-by: Harsh Kothari <techarsh@amazon.com>

Followup for opensearch-project#17714: Remove redundant tests (opensearch-project#17902) * Remove redundant tests Signed-off-by: Mikhail Khludnev <mkhl@apache.org> * Fix empty collection test Signed-off-by: Mikhail Khludnev <mkhl@apache.org> --------- Signed-off-by: Mikhail Khludnev <mkhl@apache.org> Signed-off-by: Harsh Kothari <techarsh@amazon.com>

opensearch-ci-bot mentioned this pull request Mar 30, 2025

[AUTOCUT] Gradle Check Flaky Test Report for DedicatedClusterSnapshotRestoreIT #15806

Open

jainankitk reviewed Apr 8, 2025

View reviewed changes

Review

c76663e

Signed-off-by: Mikhail Khludnev <mkhl@apache.org>

jainankitk approved these changes Apr 11, 2025

View reviewed changes

jainankitk merged commit 032f409 into opensearch-project:main Apr 11, 2025
32 of 38 checks passed

jainankitk added the backport 2.x Backport to 2.x branch label Apr 11, 2025

opensearch-trigger-bot bot added the backport-failed label Apr 11, 2025

mkhludnev mentioned this pull request Apr 11, 2025

FIX BytesRefsCollectionBuilderTests.testBuildSortedNotSorted; Followup for #17714: Remove redundant tests #17902

Merged

This was referenced Apr 14, 2025

[AUTOCUT] Gradle Check Flaky Test Report for ResourceAwareTasksTests #14293

Closed

[AUTOCUT] Gradle Check Flaky Test Report for RangeAggregatorTests #17928

Closed

mkhludnev mentioned this pull request Apr 22, 2025

backport sorted BytesRef to TermInSet #17714 to 2.x #17916

Merged

BrewTestBot mentioned this pull request May 6, 2025

opensearch 3.0.0 Homebrew/homebrew-core#222665

Merged

opensearch-ci-bot mentioned this pull request May 14, 2025

[AUTOCUT] Gradle Check Flaky Test Report for WorkloadManagementIT #17154

Open

mkhludnev mentioned this pull request Sep 20, 2025

Pack terms once when terms query is created for indexed and docValues… #19350

Merged

This was referenced Jan 11, 2026

docs: add Terms Query Optimization report for v3.4.0 tkykenmt/opensearch-feature-explorer#1788

Merged

[feature] Terms Query Optimization tkykenmt/opensearch-feature-explorer#1694

Closed

Conversation

mkhludnev commented Mar 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pass in-order terms as Sorted into TermInSet

Check List

Uh oh!

github-actions bot commented Mar 27, 2025

Uh oh!

harshavamsi commented Mar 28, 2025

Uh oh!

github-actions bot commented Mar 30, 2025

Uh oh!

github-actions bot commented Mar 30, 2025

Uh oh!

github-actions bot commented Mar 30, 2025

Uh oh!

codecov bot commented Mar 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Mar 31, 2025

Uh oh!

mkhludnev commented Apr 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jainankitk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Apr 8, 2025

Uh oh!

Uh oh!

opensearch-trigger-bot bot commented Apr 11, 2025

Uh oh!

mkhludnev commented Apr 11, 2025

Uh oh!

mkhludnev commented Apr 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mkhludnev commented Mar 27, 2025 •

edited

Loading

codecov bot commented Mar 30, 2025 •

edited

Loading

mkhludnev commented Apr 3, 2025 •

edited

Loading