[CI] WatchBackwardsCompatibilityIT failures #40841
Description
Several bwc tests fail, because of connection timeouts. In the es logs of the upgraded node the following fatal error can be found:
[2019-04-03T22:02:01,777][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [upgraded-node-0] fatal error in thread [elasticsearch[upgraded-node-0][write][T#4]], exi
ting
java.lang.AssertionError: setIfMatch [3], currentDocTem [1]
at org.elasticsearch.action.index.IndexRequest.writeTo(IndexRequest.java:657) ~[elasticsearch-6.7.1-SNAPSHOT.jar:6.7.1-SNAPSHOT]
at org.elasticsearch.action.DocWriteRequest.writeDocumentRequest(DocWriteRequest.java:244) ~[elasticsearch-6.7.1-SNAPSHOT.jar:6.7.1-SNAPSHOT]
at org.elasticsearch.action.bulk.BulkItemRequest.writeTo(BulkItemRequest.java:110) ~[elasticsearch-6.7.1-SNAPSHOT.jar:6.7.1-SNAPSHOT]
at org.elasticsearch.action.bulk.BulkShardRequest.writeTo(BulkShardRequest.java:73) ~[elasticsearch-6.7.1-SNAPSHOT.jar:6.7.1-SNAPSHOT]
at org.elasticsearch.action.support.replication.TransportReplicationAction$ConcreteShardRequest.writeTo(TransportReplicationAction.java:1276) ~[elasticsearch-6
.7.1-SNAPSHOT.jar:6.7.1-SNAPSHOT]
at org.elasticsearch.action.support.replication.TransportReplicationAction$ConcreteReplicaRequest.writeTo(TransportReplicationAction.java:1332) ~[elasticsearch
-6.7.1-SNAPSHOT.jar:6.7.1-SNAPSHOT]
at org.elasticsearch.transport.OutboundMessage.writeMessage(OutboundMessage.java:70) ~[elasticsearch-6.7.1-SNAPSHOT.jar:6.7.1-SNAPSHOT]
at org.elasticsearch.transport.OutboundMessage.serialize(OutboundMessage.java:53) ~[elasticsearch-6.7.1-SNAPSHOT.jar:6.7.1-SNAPSHOT]
at org.elasticsearch.transport.OutboundHandler$MessageSerializer.get(OutboundHandler.java:107) ~[elasticsearch-6.7.1-SNAPSHOT.jar:6.7.1-SNAPSHOT]
at org.elasticsearch.transport.OutboundHandler$MessageSerializer.get(OutboundHandler.java:93) ~[elasticsearch-6.7.1-SNAPSHOT.jar:6.7.1-SNAPSHOT]
at org.elasticsearch.transport.OutboundHandler$SendContext.get(OutboundHandler.java:140) ~[elasticsearch-6.7.1-SNAPSHOT.jar:6.7.1-SNAPSHOT]
at org.elasticsearch.transport.OutboundHandler.internalSendMessage(OutboundHandler.java:78) ~[elasticsearch-6.7.1-SNAPSHOT.jar:6.7.1-SNAPSHOT]
at org.elasticsearch.transport.OutboundHandler.sendMessage(OutboundHandler.java:70) ~[elasticsearch-6.7.1-SNAPSHOT.jar:6.7.1-SNAPSHOT]
at org.elasticsearch.transport.TcpTransport.sendRequestToChannel(TcpTransport.java:690) ~[elasticsearch-6.7.1-SNAPSHOT.jar:6.7.1-SNAPSHOT]
at org.elasticsearch.transport.TcpTransport.sendRequestToChannel(TcpTransport.java:679) ~[elasticsearch-6.7.1-SNAPSHOT.jar:6.7.1-SNAPSHOT]
at org.elasticsearch.transport.TcpTransport.access$300(TcpTransport.java:100) ~[elasticsearch-6.7.1-SNAPSHOT.jar:6.7.1-SNAPSHOT]
at org.elasticsearch.transport.TcpTransport$NodeChannels.sendRequest(TcpTransport.java:272) ~[elasticsearch-6.7.1-SNAPSHOT.jar:6.7.1-SNAPSHOT]
at org.elasticsearch.transport.TransportService.sendRequestInternal(TransportService.java:626) ~[elasticsearch-6.7.1-SNAPSHOT.jar:6.7.1-SNAPSHOT]
at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor.sendWithUser(SecurityServerTransportInterceptor.java:155) ~[?:?]
at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor.access$400(SecurityServerTransportInterceptor.java:52) ~[?:?]
at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$1.lambda$sendRequest$2(SecurityServerTransportInterceptor.java:129) ~[?:?]
at org.elasticsearch.xpack.core.security.SecurityContext.executeAfterRewritingAuthentication(SecurityContext.java:111) ~[?:?]
I suspect that this assertion error is caused by watcher trying to update a watch status, because the watcher tests was the first one to fail with connection timeout. Also Watcher's execution service does not take into account nodes before 6.6.0 that don't support if_seq_no and if_primary_term parameters on write requests.
build url: https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+6.7+bwc-tests/226/console
Metadata
Metadata
Assignees
Labels
Type
Fields
Give feedbackNo fields configured for issues without a type.