Skip to content

CI: SearchableSnapshotsIntegTests failures #66730

@costin

Description

@costin

Two set of tests are failing - from the logs they seem somewhat related.

org.elasticsearch.xpack.searchablesnapshots.SearchableSnapshotsIntegTests testSnapshotMountedIndexLeavesBlobsUntouched
org.elasticsearch.xpack.searchablesnapshots.SearchableSnapshotRecoveryStateIntegrationTests testRecoveryStateRecoveredBytesMatchPhysicalCacheState

Build scan:
https://gradle-enterprise.elastic.co/s/mespip5gogcmg
https://gradle-enterprise.elastic.co/s/p5uuwims7a7f2

Repro line:

Execution failed for task ':x-pack:plugin:searchable-snapshots:internalClusterTest'.

REPRODUCE WITH: gradlew ':x-pack:plugin:searchable-snapshots:internalClusterTest' --tests "org.elasticsearch.xpack.searchablesnapshots.SearchableSnapshotsIntegTests.testSnapshotMountedIndexLeavesBlobsUntouched" \
  -Dtests.seed=BB70024FC3F106BB \
  -Dtests.security.manager=true \
  -Dtests.locale=en-SG \
  -Dtests.timezone=America/St_Johns \
  -Druntime.java=11

REPRODUCE WITH: gradlew ':x-pack:plugin:searchable-snapshots:internalClusterTest' --tests "org.elasticsearch.xpack.searchablesnapshots.SearchableSnapshotsIntegTests.testSnapshotMountedIndexLeavesBlobsUntouched" \
  -Dtests.seed=BB70024FC3F106BB \
  -Dtests.security.manager=true \
  -Dtests.locale=en-SG \
  -Dtests.timezone=America/St_Johns \
  -Druntime.java=11
Execution failed for task ':x-pack:plugin:searchable-snapshots:internalClusterTest'.

REPRODUCE WITH: gradlew ':x-pack:plugin:searchable-snapshots:internalClusterTest' --tests "org.elasticsearch.xpack.searchablesnapshots.SearchableSnapshotRecoveryStateIntegrationTests.testRecoveryStateRecoveredBytesMatchPhysicalCacheState" \
  -Dtests.seed=D4540C05A4826074 \
  -Dtests.security.manager=true \
  -Dtests.locale=ar-AE \
  -Dtests.timezone=Asia/Atyrau \
  -Druntime.java=8

REPRODUCE WITH: gradlew ':x-pack:plugin:searchable-snapshots:internalClusterTest' --tests "org.elasticsearch.xpack.searchablesnapshots.SearchableSnapshotRecoveryStateIntegrationTests.testRecoveryStateRecoveredBytesMatchPhysicalCacheState" \
  -Dtests.seed=D4540C05A4826074 \
  -Dtests.security.manager=true \
  -Dtests.locale=ar-AE \
  -Dtests.timezone=Asia/Atyrau \
  -Druntime.java=8

Applicable branches:
7.11 and master

Failure excerpt:

com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=380, name=elasticsearch[node_s0][generic][T#2], state=RUNNABLE, group=TGRP-SearchableSnapshotRecoveryStateIntegrationTests]
	at __randomizedtesting.SeedInfo.seed([D4540C05A4826074:9AB6D6941D759D7C]:0)
Caused by: java.lang.AssertionError: shard eviction should be successful: [snapshotUUID=xlTcRHS3TvKMZ0ThoYJJtg, snapshotIndexName=twpfmpmquh, shardId=[twpfmpmquh][0]]
	at __randomizedtesting.SeedInfo.seed([D4540C05A4826074]:0)
	at org.elasticsearch.xpack.searchablesnapshots.cache.CacheService.runIfShardMarkedAsEvictedInCache(CacheService.java:412)
	at org.elasticsearch.xpack.searchablesnapshots.cache.CacheService.access$100(CacheService.java:62)
	at org.elasticsearch.xpack.searchablesnapshots.cache.CacheService$1.doRun(CacheService.java:351)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:684)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=868, name=elasticsearch[node_s1][generic][T#6], state=RUNNABLE, group=TGRP-SearchableSnapshotsIntegTests]
	at __randomizedtesting.SeedInfo.seed([BB70024FC3F106BB:5D02DB6F37A22D01]:0)
Caused by: java.lang.AssertionError: shard eviction should be successful: [snapshotUUID=zjm5kXbKSHisEnXGh-76Lw, snapshotIndexName=oxejtlweoj, shardId=[oxejtlweoj][1]]
	at __randomizedtesting.SeedInfo.seed([BB70024FC3F106BB]:0)
	at org.elasticsearch.xpack.searchablesnapshots.cache.CacheService.runIfShardMarkedAsEvictedInCache(CacheService.java:412)
	at org.elasticsearch.xpack.searchablesnapshots.cache.CacheService$1.doRun(CacheService.java:351)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:680)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

Metadata

Metadata

Assignees

Labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions