Skip to content

[CI] GoogleCloudStorageBlobStoreRepositoryTests.testWriteReadLarge (and others) causes assertion error in HTTP server #52906

@dakrone

Description

@dakrone

On https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+7.x+matrix-java-periodic/ES_RUNTIME_JAVA=corretto8,nodes=general-purpose/538/console / https://gradle-enterprise.elastic.co/s/3f3lkoii653zq

The test fails due to

09:42:14     java.lang.AssertionError: Only index blobs should remain in repository but found [foobar]
09:42:14     Expected: a collection with size <0>
09:42:14          but: collection size was <1>
09:42:14         at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
09:42:14         at org.junit.Assert.assertThat(Assert.java:956)
09:42:14         at org.elasticsearch.repositories.blobstore.ESMockAPIBasedRepositoryIntegTestCase.tearDownHttpServer(ESMockAPIBasedRepositoryIntegTestCase.java:112)

However, this is largely due in part to the HTTP server dying and causing a bunch of

09:42:14   2> com.google.cloud.storage.StorageException: Read timed out
09:42:14         at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:227)
09:42:14         at com.google.cloud.storage.spi.v1.HttpStorageRpc.write(HttpStorageRpc.java:762)
09:42:14         at com.google.cloud.storage.BlobWriteChannel$1.run(BlobWriteChannel.java:60)
...
09:42:14         Caused by:
09:42:14         java.net.SocketTimeoutException: Read timed out
09:42:14             at java.net.SocketInputStream.socketRead0(Native Method)
09:42:14             at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)

The server seems to die due to a very unhelpful assertion failure in the actual ServerImpl:

09:42:14   2> ADVERTENCIA: Uncaught exception in thread: Thread[Thread-6,5,TGRP-GoogleCloudStorageBlobStoreRepositoryTests]
09:42:14   2> java.lang.AssertionError
09:42:14   2> 	at __randomizedtesting.SeedInfo.seed([E55E3CFA3325D9BE]:0)
09:42:14   2> 	at sun.net.httpserver.ServerImpl.responseCompleted(ServerImpl.java:795)
09:42:14   2> 	at sun.net.httpserver.ServerImpl$Dispatcher.handleEvent(ServerImpl.java:284)
09:42:14   2> 	at sun.net.httpserver.ServerImpl$Dispatcher.run(ServerImpl.java:343)
09:42:14   2> 	at java.lang.Thread.run(Thread.java:748)

I was unable to reproduce this on the 7.x branch.

Metadata

Metadata

Labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions