Skip to content

Use latch to speedup multi feature migration test#84007

Merged
elasticsearchmachine merged 2 commits intoelastic:masterfrom
rjernst:feature_migration_it
Feb 16, 2022
Merged

Use latch to speedup multi feature migration test#84007
elasticsearchmachine merged 2 commits intoelastic:masterfrom
rjernst:feature_migration_it

Conversation

@rjernst
Copy link
Copy Markdown
Member

@rjernst rjernst commented Feb 16, 2022

The multi feature migration test works by having two system index
features that are to be upgraded. The reindexing of the system indices
of those features is done in sequence by the system index migrator, and
the test has an assertBusy that waits for all the migrations to
complete. Unfortunately assertBusy backs off exponentially, so it isn't great for
quickly resuming the test once the underlying assertion becomes true,
since the condition might change shortly after an iteration, and then
have to wait for a long backoff.

This commit adds a latch which will count down all the executions of the
test plugins, so that the assert busy will not be run until after all
the indexes have migrated.

closes #83953

The multi feature migration test works by having two system index
features that are to be upgraded. The reindexing of the system indices
of those features is done in sequence by the system index migrator, and
the test has an assertBusy that waits for all the migrations to
complete. Unfortunately assertBusy backs off exponentially, so it isn't great for
quickly resuming the test once the underlying assertion becomes true,
since the condition might change shortly after an iteration, and then
have to wait for a long backoff.

This commit adds a latch which will count down all the executions of the
test plugins, so that the assert busy will not be run until after all
the indexes have migrated.

closes elastic#83953
@rjernst rjernst added >test Issues or PRs that are addressing/adding tests :Core/Infra/Core Core issues without another label labels Feb 16, 2022
@rjernst rjernst requested a review from AthenaEryma February 16, 2022 00:56
@elasticmachine elasticmachine added the Team:Core/Infra Meta label for core/infra team label Feb 16, 2022
@elasticmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@rjernst rjernst added the auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) label Feb 16, 2022
@elasticsearchmachine elasticsearchmachine merged commit a7e57df into elastic:master Feb 16, 2022
@rjernst rjernst deleted the feature_migration_it branch February 16, 2022 02:13
weizijun added a commit to weizijun/elasticsearch that referenced this pull request Feb 16, 2022
…ijun/elasticsearch into fix-none-tsdb-index-dimension-tests

* 'fix-none-tsdb-index-dimension-tests' of github.com:weizijun/elasticsearch: (37 commits)
  [docs] Mention JDK 17 in the Contributing docs (elastic#84018)
  Fix GeoIpDownloader startup during rolling upgrade (elastic#84000)
  Script: Fields API for Dense Vector (elastic#83550)
  Move InferenceConfigUpdate under VersionedNamedWriteable (elastic#84022)
  [ML] Fix license feature test cleanup (elastic#84020)
  Replace deprecated api in artifact transforms (elastic#84015)
  QL: Add leniency option to SQL CLI (elastic#83795)
  [Stack Monitoring] add kibana_stats version alias to -mb template (elastic#83930)
  Optimize spliterator for ImmutableOpenMap (elastic#83899)
  Feature usage actions for archive (elastic#83931)
  Use latch to speedup multi feature migration test (elastic#84007)
  Make action names available in NodeClient (elastic#83919)
  [DOCS] Re-add HTTP proxy setings from elastic#82737 (elastic#84001)
  Add CI matrix configuration for snapshot BWC versions (elastic#83990)
  Update YAML Rest tests to check for product header on all responses (elastic#83290)
  TSDB: Add time series aggs cancellation (elastic#83492)
  [DOCS] Fix percolate query headings (elastic#83988)
  [DOCS] Move tip for percolate query example (elastic#83972)
  Simplify LocalExporter cleaner function to fix failing tests (elastic#83812)
  [GCE Discovery] Correcly handle large zones with 500 or more instances (elastic#83785)
  ...
Copy link
Copy Markdown
Contributor

@AthenaEryma AthenaEryma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Retroactive LGTM! Thanks for fixing this.

@AthenaEryma
Copy link
Copy Markdown
Contributor

One note though, we probably want to backport this to 7.17/8.1, as this test exists in basically the same form on those branches.

rjernst added a commit to rjernst/elasticsearch that referenced this pull request Feb 16, 2022
The multi feature migration test works by having two system index
features that are to be upgraded. The reindexing of the system indices
of those features is done in sequence by the system index migrator, and
the test has an assertBusy that waits for all the migrations to
complete. Unfortunately assertBusy backs off exponentially, so it isn't
great for quickly resuming the test once the underlying assertion
becomes true, since the condition might change shortly after an
iteration, and then have to wait for a long backoff.

This commit adds a latch which will count down all the executions of the
test plugins, so that the assert busy will not be run until after all
the indexes have migrated.

closes elastic#83953
rjernst added a commit to rjernst/elasticsearch that referenced this pull request Feb 16, 2022
The multi feature migration test works by having two system index
features that are to be upgraded. The reindexing of the system indices
of those features is done in sequence by the system index migrator, and
the test has an assertBusy that waits for all the migrations to
complete. Unfortunately assertBusy backs off exponentially, so it isn't
great for quickly resuming the test once the underlying assertion
becomes true, since the condition might change shortly after an
iteration, and then have to wait for a long backoff.

This commit adds a latch which will count down all the executions of the
test plugins, so that the assert busy will not be run until after all
the indexes have migrated.

closes elastic#83953
elasticsearchmachine pushed a commit that referenced this pull request Feb 17, 2022
The multi feature migration test works by having two system index
features that are to be upgraded. The reindexing of the system indices
of those features is done in sequence by the system index migrator, and
the test has an assertBusy that waits for all the migrations to
complete. Unfortunately assertBusy backs off exponentially, so it isn't
great for quickly resuming the test once the underlying assertion
becomes true, since the condition might change shortly after an
iteration, and then have to wait for a long backoff.

This commit adds a latch which will count down all the executions of the
test plugins, so that the assert busy will not be run until after all
the indexes have migrated.

closes #83953
elasticsearchmachine pushed a commit that referenced this pull request Feb 17, 2022
The multi feature migration test works by having two system index
features that are to be upgraded. The reindexing of the system indices
of those features is done in sequence by the system index migrator, and
the test has an assertBusy that waits for all the migrations to
complete. Unfortunately assertBusy backs off exponentially, so it isn't
great for quickly resuming the test once the underlying assertion
becomes true, since the condition might change shortly after an
iteration, and then have to wait for a long backoff.

This commit adds a latch which will count down all the executions of the
test plugins, so that the assert busy will not be run until after all
the indexes have migrated.

closes #83953
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) :Core/Infra/Core Core issues without another label Team:Core/Infra Meta label for core/infra team >test Issues or PRs that are addressing/adding tests v8.2.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI] MultiFeatureMigrationIT testMultipleFeatureMigration failing

4 participants