Skip to content

[CI] ShardFollowTaskReplicationTests#testRetryBulkShardOperations #39093

@martijnvg

Description

@martijnvg

This test failed, because the history between leader shard and follower shard is inconsistent.

FAILURE 12.3s | ShardFollowTaskReplicationTests.testRetryBulkShardOperations <<< FAILURES!
   > Throwable #1: java.lang.AssertionError: [test][0], node[s1], [P], s[STARTED], a[id=onA9S6QtTeehn3eQTofAxw]
   > Expected: <[Tuple [v1=140, v2=DELETE], Tuple [v1=22, v2=INDEX], Tuple [v1=107, v2=DELETE], Tuple [v1=80, v2=INDEX], Tuple [v1=47, v2=INDEX], Tuple [v1=148, v2=DELETE], Tuple [v1=14, v2=INDEX], Tuple [v1=115, v2=DELETE], Tuple [v1=72, v2=INDEX], Tuple [v1=39, v2=INDEX], Tuple [v1=105, v2=NO_OP], Tuple [v1=38, v2=INDEX], Tuple [v1=123, v2=DELETE], Tuple [v1=5, v2=INDEX], Tuple [v1=96, v2=INDEX], Tuple [v1=63, v2=INDEX], Tuple [v1=30, v2=INDEX], Tuple [v1=131, v2=DELETE], Tuple [v1=88, v2=INDEX], Tuple [v1=55, v2=INDEX], Tuple [v1=139, v2=DELETE], Tuple [v1=48, v2=INDEX], Tuple [v1=15, v2=INDEX], Tuple [v1=81, v2=INDEX], Tuple [v1=147, v2=DELETE], Tuple [v1=114, v2=DELETE], Tuple [v1=40, v2=INDEX], Tuple [v1=7, v2=INDEX], Tuple [v1=104, v2=NO_OP], Tuple [v1=73, v2=INDEX], Tuple [v1=6, v2=INDEX], Tuple [v1=122, v2=DELETE], Tuple [v1=64, v2=INDEX], Tuple [v1=31, v2=INDEX], Tuple [v1=97, v2=INDEX], Tuple [v1=130, v2=DELETE], Tuple [v1=56, v2=INDEX], Tuple [v1=23, v2=INDEX], Tuple [v1=89, v2=INDEX], Tuple [v1=20, v2=INDEX], Tuple [v1=86, v2=INDEX], Tuple [v1=53, v2=INDEX], Tuple [v1=138, v2=DELETE], Tuple [v1=113, v2=DELETE], Tuple [v1=12, v2=INDEX], Tuple [v1=78, v2=INDEX], Tuple [v1=45, v2=INDEX], Tuple [v1=146, v2=DELETE], Tuple [v1=103, v2=NO_OP], Tuple [v1=121, v2=DELETE], Tuple [v1=36, v2=INDEX], Tuple [v1=3, v2=INDEX], Tuple [v1=69, v2=INDEX], Tuple [v1=129, v2=DELETE], Tuple [v1=28, v2=INDEX], Tuple [v1=94, v2=INDEX], Tuple [v1=61, v2=INDEX], Tuple [v1=137, v2=DELETE], Tuple [v1=54, v2=INDEX], Tuple [v1=21, v2=INDEX], Tuple [v1=145, v2=DELETE], Tuple [v1=79, v2=INDEX], Tuple [v1=112, v2=DELETE], Tuple [v1=46, v2=INDEX], Tuple [v1=13, v2=INDEX], Tuple [v1=102, v2=NO_OP], Tuple [v1=71, v2=INDEX], Tuple [v1=120, v2=DELETE], Tuple [v1=4, v2=INDEX], Tuple [v1=70, v2=INDEX], Tuple [v1=37, v2=INDEX], Tuple [v1=95, v2=INDEX], Tuple [v1=128, v2=DELETE], Tuple [v1=62, v2=INDEX], Tuple [v1=29, v2=INDEX], Tuple [v1=87, v2=INDEX], Tuple [v1=136, v2=DELETE], Tuple [v1=84, v2=INDEX], Tuple [v1=51, v2=INDEX], Tuple [v1=144, v2=DELETE], Tuple [v1=18, v2=INDEX], Tuple [v1=111, v2=DELETE], Tuple [v1=76, v2=INDEX], Tuple [v1=43, v2=INDEX], Tuple [v1=101, v2=NO_OP], Tuple [v1=152, v2=DELETE], Tuple [v1=10, v2=INDEX], Tuple [v1=119, v2=DELETE], Tuple [v1=67, v2=INDEX], Tuple [v1=34, v2=INDEX], Tuple [v1=127, v2=DELETE], Tuple [v1=1, v2=INDEX], Tuple [v1=92, v2=INDEX], Tuple [v1=59, v2=INDEX], Tuple [v1=26, v2=INDEX], Tuple [v1=135, v2=DELETE], Tuple [v1=52, v2=INDEX], Tuple [v1=19, v2=INDEX], Tuple [v1=85, v2=INDEX], Tuple [v1=143, v2=DELETE], Tuple [v1=110, v2=DELETE], Tuple [v1=44, v2=INDEX], Tuple [v1=11, v2=INDEX], Tuple [v1=100, v2=NO_OP], Tuple [v1=77, v2=INDEX], Tuple [v1=151, v2=DELETE], Tuple [v1=118, v2=DELETE], Tuple [v1=68, v2=INDEX], Tuple [v1=35, v2=INDEX], Tuple [v1=2, v2=INDEX], Tuple [v1=126, v2=DELETE], Tuple [v1=60, v2=INDEX], Tuple [v1=27, v2=INDEX], Tuple [v1=93, v2=INDEX], Tuple [v1=134, v2=DELETE], Tuple [v1=109, v2=DELETE], Tuple [v1=16, v2=INDEX], Tuple [v1=82, v2=INDEX], Tuple [v1=49, v2=INDEX], Tuple [v1=142, v2=DELETE], Tuple [v1=99, v2=NO_OP], Tuple [v1=117, v2=DELETE], Tuple [v1=8, v2=INDEX], Tuple [v1=74, v2=INDEX], Tuple [v1=41, v2=INDEX], Tuple [v1=150, v2=DELETE], Tuple [v1=125, v2=DELETE], Tuple [v1=32, v2=INDEX], Tuple [v1=98, v2=INDEX], Tuple [v1=65, v2=INDEX], Tuple [v1=133, v2=DELETE], Tuple [v1=24, v2=INDEX], Tuple [v1=90, v2=INDEX], Tuple [v1=57, v2=INDEX], Tuple [v1=141, v2=DELETE], Tuple [v1=83, v2=INDEX], Tuple [v1=108, v2=DELETE], Tuple [v1=50, v2=INDEX], Tuple [v1=17, v2=INDEX], Tuple [v1=149, v2=DELETE], Tuple [v1=75, v2=INDEX], Tuple [v1=116, v2=DELETE], Tuple [v1=42, v2=INDEX], Tuple [v1=9, v2=INDEX], Tuple [v1=106, v2=NO_OP], Tuple [v1=124, v2=DELETE], Tuple [v1=0, v2=INDEX], Tuple [v1=66, v2=INDEX], Tuple [v1=33, v2=INDEX], Tuple [v1=91, v2=INDEX], Tuple [v1=132, v2=DELETE], Tuple [v1=58, v2=INDEX], Tuple [v1=25, v2=INDEX]]>
   >      but: was <[Tuple [v1=140, v2=DELETE], Tuple [v1=22, v2=INDEX], Tuple [v1=107, v2=DELETE], Tuple [v1=80, v2=INDEX], Tuple [v1=47, v2=INDEX], Tuple [v1=148, v2=DELETE], Tuple [v1=14, v2=INDEX], Tuple [v1=115, v2=DELETE], Tuple [v1=72, v2=INDEX], Tuple [v1=39, v2=INDEX], Tuple [v1=105, v2=NO_OP], Tuple [v1=38, v2=INDEX], Tuple [v1=123, v2=DELETE], Tuple [v1=5, v2=INDEX], Tuple [v1=96, v2=INDEX], Tuple [v1=63, v2=INDEX], Tuple [v1=30, v2=INDEX], Tuple [v1=131, v2=DELETE], Tuple [v1=88, v2=INDEX], Tuple [v1=55, v2=INDEX], Tuple [v1=139, v2=DELETE], Tuple [v1=48, v2=INDEX], Tuple [v1=15, v2=INDEX], Tuple [v1=81, v2=INDEX], Tuple [v1=147, v2=DELETE], Tuple [v1=114, v2=DELETE], Tuple [v1=40, v2=INDEX], Tuple [v1=7, v2=INDEX], Tuple [v1=104, v2=NO_OP], Tuple [v1=73, v2=INDEX], Tuple [v1=6, v2=INDEX], Tuple [v1=122, v2=DELETE], Tuple [v1=64, v2=INDEX], Tuple [v1=31, v2=INDEX], Tuple [v1=97, v2=INDEX], Tuple [v1=130, v2=DELETE], Tuple [v1=56, v2=INDEX], Tuple [v1=23, v2=INDEX], Tuple [v1=89, v2=INDEX], Tuple [v1=20, v2=INDEX], Tuple [v1=86, v2=INDEX], Tuple [v1=53, v2=INDEX], Tuple [v1=138, v2=DELETE], Tuple [v1=113, v2=DELETE], Tuple [v1=12, v2=INDEX], Tuple [v1=78, v2=INDEX], Tuple [v1=45, v2=INDEX], Tuple [v1=146, v2=DELETE], Tuple [v1=103, v2=NO_OP], Tuple [v1=121, v2=DELETE], Tuple [v1=36, v2=INDEX], Tuple [v1=3, v2=INDEX], Tuple [v1=69, v2=INDEX], Tuple [v1=129, v2=DELETE], Tuple [v1=28, v2=INDEX], Tuple [v1=94, v2=INDEX], Tuple [v1=61, v2=INDEX], Tuple [v1=137, v2=DELETE], Tuple [v1=54, v2=INDEX], Tuple [v1=21, v2=INDEX], Tuple [v1=145, v2=DELETE], Tuple [v1=79, v2=INDEX], Tuple [v1=112, v2=DELETE], Tuple [v1=46, v2=INDEX], Tuple [v1=13, v2=INDEX], Tuple [v1=102, v2=NO_OP], Tuple [v1=71, v2=INDEX], Tuple [v1=120, v2=DELETE], Tuple [v1=4, v2=INDEX], Tuple [v1=70, v2=INDEX], Tuple [v1=37, v2=INDEX], Tuple [v1=95, v2=INDEX], Tuple [v1=128, v2=DELETE], Tuple [v1=62, v2=INDEX], Tuple [v1=29, v2=INDEX], Tuple [v1=87, v2=INDEX], Tuple [v1=136, v2=DELETE], Tuple [v1=84, v2=INDEX], Tuple [v1=51, v2=INDEX], Tuple [v1=144, v2=DELETE], Tuple [v1=18, v2=INDEX], Tuple [v1=111, v2=DELETE], Tuple [v1=76, v2=INDEX], Tuple [v1=43, v2=INDEX], Tuple [v1=101, v2=NO_OP], Tuple [v1=152, v2=DELETE], Tuple [v1=10, v2=INDEX], Tuple [v1=119, v2=DELETE], Tuple [v1=67, v2=INDEX], Tuple [v1=34, v2=INDEX], Tuple [v1=127, v2=DELETE], Tuple [v1=1, v2=INDEX], Tuple [v1=92, v2=INDEX], Tuple [v1=59, v2=INDEX], Tuple [v1=26, v2=INDEX], Tuple [v1=135, v2=DELETE], Tuple [v1=52, v2=INDEX], Tuple [v1=19, v2=INDEX], Tuple [v1=85, v2=INDEX], Tuple [v1=143, v2=DELETE], Tuple [v1=110, v2=DELETE], Tuple [v1=44, v2=INDEX], Tuple [v1=11, v2=INDEX], Tuple [v1=100, v2=NO_OP], Tuple [v1=77, v2=INDEX], Tuple [v1=151, v2=DELETE], Tuple [v1=118, v2=DELETE], Tuple [v1=68, v2=INDEX], Tuple [v1=35, v2=INDEX], Tuple [v1=2, v2=INDEX], Tuple [v1=126, v2=DELETE], Tuple [v1=60, v2=INDEX], Tuple [v1=27, v2=INDEX], Tuple [v1=93, v2=INDEX], Tuple [v1=134, v2=DELETE], Tuple [v1=109, v2=DELETE], Tuple [v1=16, v2=INDEX], Tuple [v1=82, v2=INDEX], Tuple [v1=49, v2=INDEX], Tuple [v1=142, v2=DELETE], Tuple [v1=99, v2=NO_OP], Tuple [v1=117, v2=DELETE], Tuple [v1=8, v2=INDEX], Tuple [v1=74, v2=INDEX], Tuple [v1=41, v2=INDEX], Tuple [v1=150, v2=DELETE], Tuple [v1=125, v2=DELETE], Tuple [v1=32, v2=INDEX], Tuple [v1=98, v2=INDEX], Tuple [v1=65, v2=INDEX], Tuple [v1=133, v2=DELETE], Tuple [v1=24, v2=INDEX], Tuple [v1=90, v2=INDEX], Tuple [v1=57, v2=INDEX], Tuple [v1=141, v2=DELETE], Tuple [v1=83, v2=INDEX], Tuple [v1=108, v2=DELETE], Tuple [v1=50, v2=INDEX], Tuple [v1=17, v2=INDEX], Tuple [v1=149, v2=DELETE], Tuple [v1=75, v2=INDEX], Tuple [v1=116, v2=DELETE], Tuple [v1=9, v2=INDEX], Tuple [v1=106, v2=NO_OP], Tuple [v1=124, v2=DELETE], Tuple [v1=0, v2=INDEX], Tuple [v1=66, v2=INDEX], Tuple [v1=33, v2=INDEX], Tuple [v1=91, v2=INDEX], Tuple [v1=132, v2=DELETE], Tuple [v1=58, v2=INDEX], Tuple [v1=25, v2=INDEX]]>

Reproduces with:

./gradlew :x-pack:plugin:ccr:unitTest -Dtests.seed=3143D9701DD072C7 -Dtests.class=org.elasticsearch.xpack.ccr.action.ShardFollowTaskReplicationTests -Dtests.method="testRetryBulkShardOperations" -Dtests.security.manager=true -Dtests.locale=sk -Dtests.timezone=Pacific/Pohnpei -Dcompiler.java=11 -Druntime.java=8

Build url: https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+7.x+matrix-java-periodic/ES_BUILD_JAVA=java11,ES_RUNTIME_JAVA=java8,nodes=immutable&&linux&&docker/26/console

Metadata

Metadata

Assignees

Labels

:Distributed/CCRIssues around the Cross Cluster State Replication features>test-failureTriaged test failures from CI

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions