Implement GET API for System Feature Upgrades by williamrandolph · Pull Request #78642 · elastic/elasticsearch

williamrandolph · 2021-10-04T19:45:19Z

Here is an implementation for the GET API for system feature upgrades. It is pretty simple. When called, we iterate over all features, resolve all indices, and fetch the Version for each index from cluster state index metadata. We then determine the earliest index creation version for each feature to determine whether the features need to be upgraded or not.

I've also added a simple BWC test for rolling upgrades to make sure that we are in fact returning the correct index creation versions.

elasticmachine · 2021-10-04T19:45:23Z

Pinging @elastic/es-core-infra (Team:Core/Infra)

AthenaEryma

Thanks, this is looking good, especially the production code! Other than a super minor nit, my only comments were on the tests.

AthenaEryma · 2021-10-04T22:26:53Z

...org/elasticsearch/action/admin/cluster/migration/TransportGetFeatureUpgradeStatusAction.java

+                .orElse(Version.CURRENT)
+                .before(Version.V_7_0_0);
+
+        listener.onResponse(new GetFeatureUpgradeStatusResponse(features, isUpgradeNeeded ? "UPGRADE_NEEDED" : "NO_UPGRADE_NEEDED"));


Nit: Can you move "UPGRADE_NEEDED" and "NO_UPGRADE_NEEDED" into a constant?

Good call. I should have done that in the API stubs PR.

I used an enum, but let me know if you think a pair of string constants would be better.

...stem-indices/src/javaRestTest/java/org/elasticsearch/system/indices/FeatureUpgradeApiIT.java

server/src/internalClusterTest/java/org/elasticsearch/migration/FeatureUpgradeIT.java

...org/elasticsearch/action/admin/cluster/migration/TransportGetFeatureUpgradeStatusAction.java

qa/rolling-upgrade/src/test/java/org/elasticsearch/upgrades/FeatureUpgradeIT.java

sabarasaba · 2021-10-06T06:51:31Z

Great work @williamrandolph! I have a few questions regarding upgrade_status that I couldn't figure out from looking at the code:

If upgrading and index fails, will the upgrade_status of the feature change to something like ERROR? If so, is there going to be a way to have more details about why it failed?
What are all the possible values upgrade_status can have? So far I've noted 'UPGRADE_NEEDED' | 'NO_UPGRADE_NEEDED' | 'IN_PROGRESS' | 'ERROR' but not sure about the last two

williamrandolph · 2021-10-06T14:05:52Z

@sabarasaba All of the failure handling is being implemented for the POST API, so if we add an ERROR response it will come in a future PR. I will need to talk to @gwbrown about what sort of error reporting we can do in this API. The same goes for IN PROGRESS I will tag you on PRs that affect this behavior.

AthenaEryma

LGTM!

* Implement and test get feature upgrade status API * Add integration test for feature upgrade endpoint * Use constant enum for statuses * Add unit tests for transport class methods

* master: Fix DataTierTests package and add a validation test (elastic#78880) Fix split package org.elasticsearch.common.xcontent (elastic#78831) Store DataTier Preference directly on IndexMetadata (elastic#78668) [DOCS] Fixes typo in calendar API example (elastic#78867) Improve Node Shutdown Observability (elastic#78727) Convert encrypted snapshot license object to LicensedFeature (elastic#78731) Revert "Make nodePaths() singular (elastic#72514)" (elastic#78801) Fix incorrect generic type in PolicyStepsRegistry (elastic#78628) [DOCS] Fixes ML get calendars API (elastic#78808) Implement GET API for System Feature Upgrades (elastic#78642) [TEST] More MetadataStateFormat tests (elastic#78577) Add support for rest compatibility headers to the HLRC (elastic#78490) Un-ignoring tests after backporting fix (elastic#78830) Add node REPLACE shutdown implementation (elastic#76247) Wrap VersionPropertiesLoader in a BuildService to decouple build logic projects (elastic#78704) Adjust /_cat/templates not to request all metadata (elastic#78829) [DOCS] Fixes ML get scheduled events API (elastic#78809) Enable exit on out of memory error (elastic#71542) # Conflicts: # server/src/main/java/org/elasticsearch/cluster/metadata/DataStream.java

@OverRide

* upstream/master: (250 commits) [Transform] HLRC cleanups (elastic#78909) [ML] Make ML indices hidden when the node becomes master (elastic#77416) Introduce a Few Settings Singleton Instances (elastic#78897) Simplify TestCluster extraJar configuration (elastic#78837) Add @OverRide annotations to methods in EnrichPlugin class (elastic#76873) Add v7 restCompat for invalidating API key with the id field (elastic#78664) EQL: Refine repeatable queries (elastic#78895) Fix DataTierTests package and add a validation test (elastic#78880) Fix split package org.elasticsearch.common.xcontent (elastic#78831) Store DataTier Preference directly on IndexMetadata (elastic#78668) [DOCS] Fixes typo in calendar API example (elastic#78867) Improve Node Shutdown Observability (elastic#78727) Convert encrypted snapshot license object to LicensedFeature (elastic#78731) Revert "Make nodePaths() singular (elastic#72514)" (elastic#78801) Fix incorrect generic type in PolicyStepsRegistry (elastic#78628) [DOCS] Fixes ML get calendars API (elastic#78808) Implement GET API for System Feature Upgrades (elastic#78642) [TEST] More MetadataStateFormat tests (elastic#78577) Add support for rest compatibility headers to the HLRC (elastic#78490) Un-ignoring tests after backporting fix (elastic#78830) ... # Conflicts: # server/src/main/java/org/elasticsearch/ingest/IngestService.java # server/src/test/java/org/elasticsearch/ingest/IngestServiceTests.java

* Implement GET API for System Feature Upgrades (#78642) * Implement and test get feature upgrade status API * Add integration test for feature upgrade endpoint * Use constant enum for statuses * Add unit tests for transport class methods * Fix bwc tests for 7.x

williamrandolph added 6 commits September 29, 2021 16:28

Implement and test get feature upgrade status API

ae26206

Merge branch 'master' into get-upgrade-status-api

a5c302e

Use Version object instead of string versions

f5d1414

Refactor for streams

d609f7e

Merge branch 'master' into get-upgrade-status-api

a90f875

Add integration test for feature upgrade endpoint

4db3266

williamrandolph added >non-issue :Core/Infra/Core Core issues without another label v8.0.0 v7.16.0 labels Oct 4, 2021

williamrandolph requested review from AthenaEryma and sabarasaba October 4, 2021 19:45

elasticmachine added the Team:Core/Infra Meta label for core/infra team label Oct 4, 2021

Use tasks instead of watcher to avoid license stuff in testing

6bfe376

AthenaEryma reviewed Oct 4, 2021

View reviewed changes

williamrandolph added 6 commits October 5, 2021 14:10

Add sort order on response and fix docs test

8fe49bf

Use different index to avoid conflict with other system test

555abaa

Use constant enum for statuses

4b9e57c

PR feedback

0a91f1d

Merge branch 'master' into get-upgrade-status-api

9ad60ce

Add unit tests for transport class methods

e73f65f

williamrandolph requested a review from AthenaEryma October 5, 2021 21:45

Add IN_PROGRESS upgrade status

a8f4d09

AthenaEryma approved these changes Oct 7, 2021

View reviewed changes

Merge branch 'master' into get-upgrade-status-api

74e7482

williamrandolph merged commit 0a28c7c into elastic:master Oct 7, 2021

williamrandolph mentioned this pull request Oct 7, 2021

Implement GET API for System Feature Upgrades (#78642) #78860

Merged

williamrandolph mentioned this pull request Oct 15, 2021

REST API for GET system feature upgrade method #77524

Open

jakelandis added v8.0.0-beta1 and removed v8.0.0 labels Oct 27, 2021

williamrandolph deleted the get-upgrade-status-api branch May 23, 2022 17:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement GET API for System Feature Upgrades#78642

Implement GET API for System Feature Upgrades#78642
williamrandolph merged 15 commits intoelastic:masterfrom
williamrandolph:get-upgrade-status-api

williamrandolph commented Oct 4, 2021

Uh oh!

elasticmachine commented Oct 4, 2021

Uh oh!

AthenaEryma left a comment

Uh oh!

AthenaEryma Oct 4, 2021

Uh oh!

williamrandolph Oct 5, 2021

Uh oh!

williamrandolph Oct 5, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sabarasaba commented Oct 6, 2021

Uh oh!

williamrandolph commented Oct 6, 2021

Uh oh!

AthenaEryma left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

williamrandolph commented Oct 4, 2021

Uh oh!

elasticmachine commented Oct 4, 2021

Uh oh!

AthenaEryma left a comment

Choose a reason for hiding this comment

Uh oh!

AthenaEryma Oct 4, 2021

Choose a reason for hiding this comment

Uh oh!

williamrandolph Oct 5, 2021

Choose a reason for hiding this comment

Uh oh!

williamrandolph Oct 5, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sabarasaba commented Oct 6, 2021

Uh oh!

williamrandolph commented Oct 6, 2021

Uh oh!

AthenaEryma left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants