Skip to content

[8.x] Add a cluster listener to fix missing system index mappings after upgrade#115771

Merged
elasticsearchmachine merged 17 commits intoelastic:8.xfrom
ldematte:8-x-compatibility-versions-fixup-listener
Nov 11, 2024
Merged

[8.x] Add a cluster listener to fix missing system index mappings after upgrade#115771
elasticsearchmachine merged 17 commits intoelastic:8.xfrom
ldematte:8-x-compatibility-versions-fixup-listener

Conversation

@ldematte
Copy link
Copy Markdown
Contributor

@ldematte ldematte commented Oct 28, 2024

This PR modifies TransportVersionsFixupListener to include all of compatibility versions (not only TransportVersion) in the fixup.

TransportVersionsFixupListener spots the instances when the master has been upgraded to the most recent code version, along with non-master nodes, but some nodes are missing a "proper" (non-inferred) Transport version.
This PR adds another check to also ensure that we have real (non-empty) system index mapping versions.

To do so, it modifies NodeInfo so it carries all of CompatibilityVersions (TransportVersion + SystemIndexDescriptor.MappingVersions).

This was initially done via a separate fixup listener + ad-hoc transport action, but the 2 listeners "raced" to update ClusterState on the same CompatibilityVersions structure; it just made sense to do it at the same time.

The fixup is very similar to #110710, which does the same for cluster features; plus, it adds a CI test to cover the bug raised in #112694

Closes #112694

@ldematte ldematte added :Core/Infra/Core Core issues without another label >bug auto-backport Automatically create backport pull requests when merged labels Oct 28, 2024
@ldematte ldematte added test-full-bwc Trigger full BWC version matrix tests and removed auto-backport Automatically create backport pull requests when merged v8.16.1 labels Nov 7, 2024
@ldematte
Copy link
Copy Markdown
Contributor Author

ldematte commented Nov 7, 2024

Removing auto-backport as this includes a tv change.
Also, not sure if it's worth it - should we backport it to 8.16? 8.15?

@ldematte ldematte marked this pull request as ready for review November 7, 2024 13:55
@ldematte ldematte requested a review from a team November 7, 2024 13:55
@thecoop
Copy link
Copy Markdown
Member

thecoop commented Nov 7, 2024

I think we should backport it to 8.16.1, given it fixes some erroneous behaviour users have seen. We can use a patch TV to do this change on 8.16.1.

@ldematte
Copy link
Copy Markdown
Contributor Author

ldematte commented Nov 7, 2024

We can use a patch TV to do this change on 8.16.1.

Will do that, thanks!

}
}

private static List<String> getUpgradedNodesAddresses() throws IOException {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There must be a helper method somewhere that does this already?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't find one, but I'll look again

public void clusterChanged(ClusterChangedEvent event) {
if (event.localNodeMaster() == false) return; // only if we're master

List<Stream<String>> queries = new ArrayList<>();
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit - Maybe just have a Stream<String>, you can then add another stream using queries = Stream.concat(queries, ...)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or why not just have a Set<String> and you can do a forEach(addAll)?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A set will do :)

Copy link
Copy Markdown
Member

@thecoop thecoop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Principal looks good, just a few things to tweak

@thecoop
Copy link
Copy Markdown
Member

thecoop commented Nov 8, 2024

If this is being backported to 8.16, the backport PR should be in place too, and the backport transport version referenced in this PR

@thecoop thecoop self-requested a review November 8, 2024 13:15
@ldematte
Copy link
Copy Markdown
Contributor Author

ldematte commented Nov 8, 2024

If this is being backported to 8.16, the backport PR should be in place too, and the backport transport version referenced in this PR

Yes, definitely, I wanted to have this in the final form before backporting.

@ldematte ldematte added auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) auto-backport Automatically create backport pull requests when merged v8.16.1 labels Nov 11, 2024
@elasticsearchmachine elasticsearchmachine merged commit d698e72 into elastic:8.x Nov 11, 2024
@ldematte ldematte deleted the 8-x-compatibility-versions-fixup-listener branch November 11, 2024 18:45
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

💔 Backport failed

The backport operation could not be completed due to the following error:

An unexpected error occurred when attempting to backport this PR.

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 115771

brianseeders added a commit that referenced this pull request Nov 11, 2024
ldematte added a commit to ldematte/elasticsearch that referenced this pull request Nov 12, 2024
jozala pushed a commit that referenced this pull request Nov 13, 2024
Extracting the Transport protocol related changes from
#115771 to make backport
easier.
afoucret pushed a commit to afoucret/elasticsearch that referenced this pull request Nov 14, 2024
Extracting the Transport protocol related changes from
elastic#115771 to make backport
easier.
elasticsearchmachine pushed a commit that referenced this pull request Nov 21, 2024
…ter upgrade (#115771) (#116646)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
alexey-ivanov-es pushed a commit to alexey-ivanov-es/elasticsearch that referenced this pull request Nov 28, 2024
Extracting the Transport protocol related changes from
elastic#115771 to make backport
easier.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Automatically create backport pull requests when merged auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) backport pending >bug :Core/Infra/Core Core issues without another label test-full-bwc Trigger full BWC version matrix tests v8.16.1 v8.17.0 WIP

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants