Skip to content

[Concurrent Segment Search][BUG] org.opensearch.search.profile.query.QueryProfilerIT.testProfileQuery {p0={"search.concurrent_segment_search.enabled":"true"} p1=true} is flaky #9787

@reta

Description

@reta

Describe the bug
The test case org.opensearch.search.profile.query.QueryProfilerIT.testProfileQuery {p0={"search.concurrent_segment_search.enabled":"true"} p1=true} is flaky

Failed to execute phase [query], all shards failed; shardFailures {[cWz5tn-wSD6HkZtOCYeenQ][test][0]: RemoteTransportException[[node_s1][127.0.0.1:42349][indices:data/read/search[phase/query]]]; nested: QueryPhaseExecutionException[Query Failed [Failed to execute main query]]; nested: QueryPhaseExecutionException[Query Failed [Failed to execute concurrent segment search thread]]; nested: AssertionError; }
	at __randomizedtesting.SeedInfo.seed([B1A9A3487FC5243D:4A8940A482451BDC]:0)
	at app//org.opensearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:677)
	at app//org.opensearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:373)
	at app//org.opensearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:716)
	at app//org.opensearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:485)
	at app//org.opensearch.action.search.AbstractSearchAsyncAction$1.onFailure(AbstractSearchAsyncAction.java:295)
	at app//org.opensearch.action.search.SearchExecutionStatsCollector.onFailure(SearchExecutionStatsCollector.java:104)
	at app//org.opensearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:75)
	at app//org.opensearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:749)
	at app//org.opensearch.transport.TransportService$6.handleException(TransportService.java:875)
	at app//org.opensearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1483)
	at app//org.opensearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1597)
	at app//org.opensearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1571)
	at app//org.opensearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:80)
	at app//org.opensearch.transport.TransportChannel.sendErrorResponse(TransportChannel.java:72)
	at app//org.opensearch.action.support.ChannelActionListener.onFailure(ChannelActionListener.java:70)
	at app//org.opensearch.action.ActionRunnable.onFailure(ActionRunnable.java:104)
	at app//org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:54)
	at app//org.opensearch.threadpool.TaskAwareRunnable.doRun(TaskAwareRunnable.java:78)
	at app//org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52)
	at app//org.opensearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:59)
	at app//org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:908)
	at app//org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52)
	at java.base@20.0.2/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base@20.0.2/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base@20.0.2/java.lang.Thread.run(Thread.java:1623)
Caused by: OpenSearchException; nested: AssertionError;
	at app//org.opensearch.OpenSearchException.guessRootCauses(OpenSearchException.java:708)
	at app//org.opensearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:371)
	... 23 more
Caused by: java.lang.AssertionError
	at org.opensearch.search.profile.query.AbstractQueryProfileTree.startRewriteTime(AbstractQueryProfileTree.java:48)
	at org.opensearch.search.profile.query.QueryProfiler.startRewriteTime(QueryProfiler.java:78)
	at org.opensearch.search.internal.ContextIndexSearcher.rewrite(ContextIndexSearcher.java:189)
	at org.apache.lucene.search.AbstractMultiTermQueryConstantScoreWrapper$RewritingWeight.rewrite(AbstractMultiTermQueryConstantScoreWrapper.java:174)
	at org.apache.lucene.search.AbstractMultiTermQueryConstantScoreWrapper$RewritingWeight$1.get(AbstractMultiTermQueryConstantScoreWrapper.java:269)
	at org.opensearch.search.profile.query.ProfileWeight$1.get(ProfileWeight.java:96)
	at org.opensearch.search.profile.query.ProfileWeight.scorer(ProfileWeight.java:72)
	at org.apache.lucene.search.Weight.bulkScorer(Weight.java:166)
	at org.opensearch.search.profile.query.ProfileWeight.bulkScorer(ProfileWeight.java:123)
	at org.opensearch.search.internal.ContextIndexSearcher$1.bulkScorer(ContextIndexSearcher.java:380)
	at org.opensearch.search.internal.ContextIndexSearcher.searchLeaf(ContextIndexSearcher.java:321)
	at org.opensearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:280)
	at org.apache.lucene.search.IndexSearcher.lambda$search$1(IndexSearcher.java:723)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	... 8 more

To Reproduce

./gradlew ':server:internalClusterTest' --tests "org.opensearch.search.profile.query.QueryProfilerIT" -Dtests.method="testProfileQuery {p0={"search.concurrent_segment_search.enabled":"true"} p1=true}" -Dtests.seed=B1A9A3487FC5243D 

Expected behavior
Test must always pass

Plugins
Standard

Screenshots
N/A

Host/Environment (please complete the following information):

  • CI

Additional context
https://build.ci.opensearch.org/job/gradle-check/24531/testReport/junit/org.opensearch.search.profile.query/QueryProfilerIT/testProfileQuery__p0___search_concurrent_segment_search_enabled___true___p1_true_/

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingflaky-testRandom test failure that succeeds on second runv2.12.0Issues and PRs related to version 2.12.0v3.0.0Issues and PRs related to version 3.0.0

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions