[Migrations] Provide testing archives + tooling for migrations integration tests#193328

Merged

gsoldevila merged 20 commits intoelastic:mainfrom

gsoldevila:kbn-192719-upgrade-migration-tests-1

Sep 23, 2024

Member

gsoldevila commented Sep 18, 2024 •

edited by kibanamachine

Loading

Summary

Part of https://github.com/elastic/kibana-team/issues/1113

Creates zip packages that can be used by multiple tests.
Provides tooling to easily create new packages with any desired data and ES version.
Fixes some of the failing tests with the newly created archives.

gsoldevila added Team:Core release_note:skip backport:skip Feature:Migrations labels

rudolf approved these changes

View reviewed changes

gsoldevila changed the title ~~[Migrations Integration Tests] Remove dot_kibana_split and improve existing tests~~ [Migrations] Provide testing archives + tooling for migrations integration tests

gsoldevila force-pushed the kbn-192719-upgrade-migration-tests-1 branch from d9223e5 to 187608c Compare

September 20, 2024 14:06

gsoldevila added 20 commits

September 23, 2024 09:38


          Remove tests related to ID rewriting

50b8810


          Remove failed_action_tasks tests

4f59514


          Remove migration_from_older_v1 and dot_kibana_split tests

8dc0c50


          Define reindexing_migration test (covers failed_action_tasks' exclu…

97bfdf9

…deOnUpgrade)


          Add migration_from_older_v1 expect statements to this test

fa33a75


          Create baseline migrations archives + tooling. Adapt tests

be3898b


          Disable create_test_archives pseudo-test

02482e1


          Add new 'task' type + index to baseline archives, adapt tests

1e4344c


          Update single_migration_failures tests to use baseline archive

fb693e2


          Revert "Remove tests related to ID rewriting"

17df08a

This reverts commit 604f8c4ae9aef478141bb4a4d4f549523853aaf7.


          Revert "Remove failed_action_tasks tests"

241ad6d

This reverts commit 157b0a2dcd3e4e1392c565bf762787db916dbd6f.


          Revert "Remove migration_from_older_v1 and dot_kibana_split tests"

da7bfa4

This reverts commit 19500a4ccc757e830437f3c30face4acb7bc577c.


          Remove unused archive

f040d46


          Fix failing tests

c26979d


          Fix TS and linting issues, outdated tests

a1a4e10


          Remove xpack_100k.test, covered in v2_migration.test

9d0cb42


          Remove tests made obsolete by Incremental bulk execution

204d942


          Remove unneccesary test

76dc8b2


          Disable create_test_archives test (dev only)

592762f


          Update test, as any of the two migrators can fail

700e66f

gsoldevila force-pushed the kbn-192719-upgrade-migration-tests-1 branch from 187608c to 700e66f Compare

September 23, 2024 07:38

gsoldevila marked this pull request as ready for review

September 23, 2024 07:38

gsoldevila requested a review from a team as a code owner

September 23, 2024 07:38

afharo mentioned this pull request

[Migrations] Fix integration tests that use ZIP archives that are not compatible with 9.0.0 #193856

Merged

This was referenced Sep 26, 2024

Failing test: Jest Integration Tests.src/core/server/integration_tests/saved_objects/migrations/group5 - multiple migrator instances running in parallel when upgrading to a new stack version with matching mappings will actively delete and successfully complete migration #192623

Closed

Failing test: Jest Integration Tests.src/core/server/integration_tests/saved_objects/migrations/group6 - split .kibana index into multiple system indices failure cases when the .kibana migrator fails on the REINDEX_SOURCE_TO_TEMP_INDEX_BULK state is capable of successfully performing the split migration in subsequent restart #192719

Closed

Failing test: Jest Integration Tests.src/core/server/integration_tests/saved_objects/migrations/group6 - split .kibana index into multiple system indices failure cases when the .kibana_analytics migrator fails on the UPDATE_TARGET_MAPPINGS_PROPERTIES state is capable of successfully performing the split migration in subsequent restart #174311

Closed

Failing test: Jest Integration Tests.src/core/server/integration_tests/saved_objects/migrations/group6 - split .kibana index into multiple system indices failure cases when the .kibana_analytics migrator fails on the CLONE_TEMP_TO_TARGET state is capable of successfully performing the split migration in subsequent restart #174310

Closed

Failing test: Jest Integration Tests.src/core/server/integration_tests/saved_objects/migrations/group6 - split .kibana index into multiple system indices failure cases when the .kibana migrator fails on the CLONE_TEMP_TO_TARGET state is capable of successfully performing the split migration in subsequent restart #174308

Closed

Failing test: Jest Integration Tests.src/core/server/integration_tests/saved_objects/migrations/group2 - migration v2 migrates saved objects normally when multiple Kibana instances are started with an average interval #156117

Closed

Failing test: Jest Integration Tests.src/core/server/integration_tests/saved_objects/migrations/group5 - split .kibana index into multiple system indices when migrating from a legacy version performs v1 migration and then relocates saved objects into different indices, depending on their types #163249

Closed

Failing test: Jest Integration Tests.src/core/server/integration_tests/saved_objects/migrations/group5 - split .kibana index into multiple system indices when multiple Kibana migrators run in parallel correctly migrates 7.7.2_xpack_100k_obj.zip archive #163250

Closed

Failing test: Jest Integration Tests.src/core/server/integration_tests/saved_objects/migrations/group6 - split .kibana index into multiple system indices failure cases when the .kibana migrator fails on the UPDATE_TARGET_MAPPINGS_PROPERTIES state is capable of successfully performing the split migration in subsequent restart #174309

Closed

Failing test: Jest Integration Tests.src/core/server/integration_tests/saved_objects/migrations/group6 - split .kibana index into multiple system indices failure cases when the .kibana_task_manager migrator fails on the TRANSFORMED_DOCUMENTS_BULK_INDEX state, after the other ones have finished is capable of completing the .kibana_task_manager migration in subsequent restart #176432

Closed

Failing test: Jest Integration Tests.src/core/server/integration_tests/saved_objects/migrations/group3/actions - migration actions bulkOverwriteTransformedDocuments resolves left request_entity_too_large_exception when the payload is too large #193592

Closed

gsoldevila added a commit that referenced this pull request


          [8.x] [Migrations] Backport ALL updates to integration tests (#194013)

c7b29d1

# Backport

This will backport the following PRs from `main` to `8.x`:
 - #193328
 - #193856
 - #193696
 - #194151

kibanamachine added the backport missing label

Contributor

kibanamachine commented Sep 30, 2024

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add the label auto-backport or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 193328 locally

1 similar comment

Contributor

kibanamachine commented Sep 30, 2024

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add the label auto-backport or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 193328 locally

Contributor

kibanamachine commented Oct 1, 2024

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add the label auto-backport or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 193328 locally

1 similar comment

Contributor

kibanamachine commented Oct 2, 2024

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add the label auto-backport or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 193328 locally

Contributor

kibanamachine commented Oct 3, 2024

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 193328 locally

4 similar comments

Contributor

kibanamachine commented Oct 4, 2024

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 193328 locally

Contributor

kibanamachine commented Oct 7, 2024

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 193328 locally

Contributor

kibanamachine commented Oct 8, 2024

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 193328 locally

Contributor

kibanamachine commented Oct 9, 2024

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 193328 locally

afharo added v8.16.0 backport:skip and removed backport:prev-minor labels

kibanamachine removed the backport missing label

Member

afharo commented Oct 10, 2024

Already backported as per this comment: #193328 (comment)

kibanamachine mentioned this pull request

Update mappings if/when new SO types are introduced #197061

Merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip Feature:Migrations release_note:skip Team:Core v8.16.0 v9.0.0