Skip to content

Remove test usages of DataStream#getDefaultBackingIndexName in ILM integration tests#124319

Merged
gmarouli merged 6 commits intoelastic:mainfrom
gmarouli:test-fix/replace-default-backing-index-name-ilm
Mar 10, 2025
Merged

Remove test usages of DataStream#getDefaultBackingIndexName in ILM integration tests#124319
gmarouli merged 6 commits intoelastic:mainfrom
gmarouli:test-fix/replace-default-backing-index-name-ilm

Conversation

@gmarouli
Copy link
Copy Markdown
Contributor

@gmarouli gmarouli commented Mar 7, 2025

Relates to #123376.

In this PR we change the way we retrieve the name of the backing index of data streams. We replace usages of time sensitive DataStream#getDefaultBackingIndexName with the retrieval of the name via an API call. The problem with using the time sensitive method is that we can have test failures around midnight.

Because this is part of a larger effort #123376 we think it's a good idea to add a helper method to facilitate the retrieval of the backing index names to the base test classes.

Fixes: #117403
Fixes: #123203
Fixes: #123202
Fixes: #124439

gmarouli added 2 commits March 7, 2025 15:06
…csearch.x-pack.plugin.ilm.qa.multi-node.internalClusterTest`
…csearch.x-pack.plugin.ilm.qa.multi-node.javaRestTest`
@gmarouli gmarouli added >test Issues or PRs that are addressing/adding tests :Data Management/ILM+SLM DO NOT USE. Use ":StorageEngine/ILM" or ":Distributed Coordination/SLM" instead. labels Mar 7, 2025
@elasticsearchmachine elasticsearchmachine added the Team:Data Management (obsolete) DO NOT USE. This team no longer exists. label Mar 7, 2025
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

Copy link
Copy Markdown
Contributor

@nielsbauman nielsbauman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some minor comments, but other than that LGTM. Thanks, Mary! One big step closer to never having these annoying test failures again :)

I think we should backport this because the failures occurred on 8.16 too.

Also, I see #124439 got opened after you opened this PR, so you can close that one with this PR too. Four closed issues with one PR is not too bad at all 😄

Comment on lines +110 to +111
String backingIndexName = getDataStreamBackingIndexNames(dataStream).getFirst();
assertThat(backingIndexName, endsWith("1"));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you feel about asserting the size instead of the last character?

I'm thinking far in the future here, but what if we come to a point where we lazily initialize data streams and we first ingest to the failure store, then the failure store will end with a 1 instead of the backing index. That's super hypothetical, so I'm also fine to ignore it.

I do feel like asserting there is only one backing index is slightly more relevant than asserting the last character, but it's not a blocker for me. What do you think?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do feel like asserting there is only one backing index is slightly more relevant than asserting the last character, but it's not a blocker for me. What do you think?

I only added this assertion after a rollover was performed, so we would have to assert that size is 2. I can do that. No strong feelings about it.

@gmarouli
Copy link
Copy Markdown
Contributor Author

Thank you for the review @nielsbauman , comments addressed :)

@gmarouli gmarouli added auto-backport Automatically create backport pull requests when merged v8.19.0 v9.0.1 v8.18.1 auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) labels Mar 10, 2025
@nielsbauman
Copy link
Copy Markdown
Contributor

LGTM! Thanks!

@gmarouli gmarouli merged commit 44dd44b into elastic:main Mar 10, 2025
3 of 5 checks passed
@gmarouli gmarouli deleted the test-fix/replace-default-backing-index-name-ilm branch March 10, 2025 10:38
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

elasticsearchmachine commented Mar 10, 2025

💚 Backports

Status Branch Result
8.16
8.17
8.18
8.x
9.0

gmarouli added a commit to gmarouli/elasticsearch that referenced this pull request Mar 10, 2025
gmarouli added a commit to gmarouli/elasticsearch that referenced this pull request Mar 10, 2025
…integration tests (elastic#124319)

* Incorporate review comments

(cherry picked from commit 44dd44b)

# Conflicts:
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/TimeSeriesDataStreamsIT.java
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/TimeSeriesLifecycleActionsIT.java
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/actions/SearchableSnapshotActionIT.java
elasticsearchmachine pushed a commit that referenced this pull request Mar 10, 2025
…integration tests (#124319) (#124465)

* Incorporate review comments
elasticsearchmachine pushed a commit that referenced this pull request Mar 10, 2025
…integration tests (#124319) (#124467)

* Incorporate review comments

(cherry picked from commit 44dd44b)

# Conflicts:
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/TimeSeriesDataStreamsIT.java
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/TimeSeriesLifecycleActionsIT.java
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/actions/SearchableSnapshotActionIT.java
gmarouli added a commit to gmarouli/elasticsearch that referenced this pull request Mar 10, 2025
…integration tests (elastic#124319) (elastic#124467)

* Incorporate review comments

(cherry picked from commit 44dd44b)

# Conflicts:
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/TimeSeriesDataStreamsIT.java
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/TimeSeriesLifecycleActionsIT.java
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/actions/SearchableSnapshotActionIT.java
gmarouli added a commit to gmarouli/elasticsearch that referenced this pull request Mar 10, 2025
…integration tests (elastic#124319) (elastic#124467)

* Incorporate review comments

(cherry picked from commit 44dd44b)

# Conflicts:
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/TimeSeriesDataStreamsIT.java
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/TimeSeriesLifecycleActionsIT.java
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/actions/SearchableSnapshotActionIT.java
(cherry picked from commit 474d223)

# Conflicts:
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/actions/SearchableSnapshotActionIT.java
gmarouli added a commit to gmarouli/elasticsearch that referenced this pull request Mar 10, 2025
…integration tests (elastic#124319) (elastic#124467)

* Incorporate review comments

(cherry picked from commit 44dd44b)

# Conflicts:
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/TimeSeriesDataStreamsIT.java
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/TimeSeriesLifecycleActionsIT.java
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/actions/SearchableSnapshotActionIT.java
(cherry picked from commit 474d223)

# Conflicts:
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/actions/SearchableSnapshotActionIT.java
elasticsearchmachine pushed a commit that referenced this pull request Mar 10, 2025
…integration tests (#124319) (#124467) (#124473)

* Incorporate review comments

(cherry picked from commit 44dd44b)

# Conflicts:
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/TimeSeriesDataStreamsIT.java
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/TimeSeriesLifecycleActionsIT.java
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/actions/SearchableSnapshotActionIT.java
elasticsearchmachine pushed a commit that referenced this pull request Mar 10, 2025
…integration tests (#124319) (#124467) (#124476)

* Incorporate review comments

(cherry picked from commit 44dd44b)

# Conflicts:
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/TimeSeriesDataStreamsIT.java
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/TimeSeriesLifecycleActionsIT.java
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/actions/SearchableSnapshotActionIT.java
(cherry picked from commit 474d223)

# Conflicts:
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/actions/SearchableSnapshotActionIT.java
georgewallace pushed a commit to georgewallace/elasticsearch that referenced this pull request Mar 11, 2025
elasticsearchmachine pushed a commit that referenced this pull request Mar 12, 2025
…integration tests (#124319) (#124467) (#124482)

* Incorporate review comments

(cherry picked from commit 44dd44b)

# Conflicts:
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/TimeSeriesDataStreamsIT.java
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/TimeSeriesLifecycleActionsIT.java
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/actions/SearchableSnapshotActionIT.java
(cherry picked from commit 474d223)

# Conflicts:
#	x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/actions/SearchableSnapshotActionIT.java
jfreden pushed a commit to jfreden/elasticsearch that referenced this pull request Mar 13, 2025
nielsbauman added a commit to nielsbauman/elasticsearch that referenced this pull request Mar 27, 2025
This test failure was already fixed by elastic#124319 but wasn't unmuted yet.
@nielsbauman nielsbauman mentioned this pull request Mar 27, 2025
nielsbauman added a commit to nielsbauman/elasticsearch that referenced this pull request Mar 27, 2025
This test failure was already fixed by elastic#124319 but wasn't unmuted yet.
elasticsearchmachine pushed a commit that referenced this pull request Mar 28, 2025
This test failure was already fixed by #124319 but wasn't unmuted yet.
omricohenn pushed a commit to omricohenn/elasticsearch that referenced this pull request Mar 28, 2025
This test failure was already fixed by elastic#124319 but wasn't unmuted yet.
nielsbauman added a commit to nielsbauman/elasticsearch that referenced this pull request Jun 26, 2025
This test was already fixed in elastic#124319.
@nielsbauman nielsbauman mentioned this pull request Jun 26, 2025
nielsbauman added a commit that referenced this pull request Jun 26, 2025
This test was already fixed in #124319.
mridula-s109 pushed a commit to mridula-s109/elasticsearch that referenced this pull request Jul 3, 2025
This test was already fixed in elastic#124319.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Automatically create backport pull requests when merged auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) :Data Management/ILM+SLM DO NOT USE. Use ":StorageEngine/ILM" or ":Distributed Coordination/SLM" instead. Team:Data Management (obsolete) DO NOT USE. This team no longer exists. >test Issues or PRs that are addressing/adding tests v8.16.6 v8.17.4 v8.18.1 v8.19.0 v9.0.1 v9.1.0

Projects

None yet

3 participants