-
Notifications
You must be signed in to change notification settings - Fork 2.5k
[BUG] org.opensearch.action.admin.indices.create.RemoteSplitIndexIT is flaky #11091
Description
Describe the bug
The whole test suite org.opensearch.action.admin.indices.create.RemoteSplitIndexI seems to be flaky:
org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndex
org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testCreateSplitIndexToN
org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testSplitFromOneToN
org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.testSplitIndexPrimaryTerm
org.opensearch.action.admin.indices.create.RemoteSplitIndexIT.classMethod
java.lang.AssertionError: All incoming requests on node [node_s0] should have finished. Expected 0 but got 4580; pending tasks [[{
"node" : "node_s0",
"id" : 610,
"type" : "transport",
"action" : "internal:index/shard/recovery/start_recovery",
"description" : "",
"start_time" : "2023-11-04T02:12:00.316Z",
"start_time_in_millis" : 1699063920316,
"running_time" : "4.1m",
"running_time_in_nanos" : 247039486299,
"cancellable" : false,
"cancelled" : false,
"headers" : { },
"resource_stats" : {
"average" : {
"cpu_time_in_nanos" : 0,
"memory_in_bytes" : 0
},
"total" : {
"cpu_time_in_nanos" : 0,
"memory_in_bytes" : 0
},
"min" : {
"cpu_time_in_nanos" : 0,
"memory_in_bytes" : 0
},
"max" : {
"cpu_time_in_nanos" : 0,
"memory_in_bytes" : 0
},
"thread_info" : {
"thread_executions" : 0,
"active_threads" : 0
}
}
}]]
nov 03, 2023 7:12:00 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
AVVERTENZA: Uncaught exception in thread: Thread[#637,opensearch[node_s0][generic][T#4],5,TGRP-RemoteSplitIndexIT]
java.lang.AssertionError: shard [target][0] is not locked
at __randomizedtesting.SeedInfo.seed([35E40FF221179300]:0)
at org.opensearch.env.NodeEnvironment.deleteShardDirectoryUnderLock(NodeEnvironment.java:579)
at org.opensearch.indices.IndicesService.deleteShardStore(IndicesService.java:1215)
at org.opensearch.index.IndexService.onShardClose(IndexService.java:675)
at org.opensearch.index.IndexService$StoreCloseListener.accept(IndexService.java:798)
at org.opensearch.index.IndexService$StoreCloseListener.accept(IndexService.java:785)
at org.opensearch.index.store.Store.closeInternal(Store.java:573)
at org.opensearch.index.store.Store$1.closeInternal(Store.java:193)
at org.opensearch.common.util.concurrent.AbstractRefCounted.decRef(AbstractRefCounted.java:78)
at org.opensearch.index.store.Store.decRef(Store.java:546)
at org.opensearch.action.ActionRunnable$1.doRun(ActionRunnable.java:61)
at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:908)
at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
To Reproduce
./gradlew ':server:internalClusterTest' --tests "org.opensearch.action.admin.indices.create.RemoteSplitIndexIT" -Dtests.seed=35E40FF221179300
Expected behavior
The test should always pass
Plugins
Standard
Screenshots
If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):
- CI
Additional context