Report shard counts for ongoing snapshots#77622
Conversation
09bed75 to
c4d62e4
Compare
|
@elasticmachine update branch |
cfe0c36 to
5027c16
Compare
|
@elasticmachine run elasticsearch-ci/part-1 |
ad2d7f7 to
0c6faaf
Compare
Ref elastic#76704 Add simple unit tests for shard counts
0c6faaf to
e476e42
Compare
server/src/internalClusterTest/java/org/elasticsearch/snapshots/GetSnapshotsIT.java
Show resolved
Hide resolved
|
Pinging @elastic/es-distributed (Team:Distributed) |
Otherwise, the stable sorting isn't working because we get a different amount of the successfully migrated shards.
|
Hi @tlrx and @original-brownbear, could you guys please take a look at the PR, thanks! |
|
|
||
| createIndexWithContent("test-idx-1"); | ||
| String indexName = "test-idx-1"; | ||
| createIndexWithContent(indexName, indexSettingsNoReplicas(4).build()); |
There was a problem hiding this comment.
Instead of forcing no replicas maybe we let the index settings be randomized as usual, and use getNumShards() to retrieve the expected number of shards to compare with later in the test?
|
@elasticmachine update branch |
original-brownbear
left a comment
There was a problem hiding this comment.
Looks good, just a few minor points tests (lets not use actionGet) and the looping.
server/src/internalClusterTest/java/org/elasticsearch/snapshots/SnapshotStatusApisIT.java
Show resolved
Hide resolved
| 0, | ||
| Collections.emptyList(), | ||
| entry.shards().size(), | ||
| Math.toIntExact(entry.shards().stream().filter(c -> c.getValue().state() == SnapshotsInProgress.ShardState.SUCCESS).count()), |
There was a problem hiding this comment.
Might be nice to calculate both this count and the failures list in a single loop by using a static method + Tuple or so? Afaik this may run on a transport thread so it's worth saving the cycles, especially as we scale to larger and larger clusters.
There was a problem hiding this comment.
Yeah, I believe I can do that with a teeing collector.
There was a problem hiding this comment.
Actually, we can't, because we were calling another constructor and couldn't declare variables. I went ahead with removing of the constructor call and using a simple loop for calculating both fields.
...src/internalClusterTest/java/org/elasticsearch/snapshots/SharedClusterSnapshotRestoreIT.java
Show resolved
Hide resolved
|
@elasticmachine run elasticsearch-ci/part-1 |
|
Thanks Tanguy and Armin! |
💔 Backport failed
You can use sqren/backport to manually backport by running |
For in-progress snapshots we can try to calculate the amount of successful shards based on the shard state. We can use a similar approach for failures. Closes elastic#76704
For in-progress snapshots we can try to calculate the amount of successful shards based on the shard state. We can use a similar approach for failures.
Reference #76704