All system indices are hidden indices#79512
Merged
williamrandolph merged 64 commits intoelastic:masterfrom Dec 6, 2021
Merged
Conversation
williamrandolph
commented
Oct 21, 2021
qa/smoke-test-http/src/test/java/org/elasticsearch/http/SystemIndexRestIT.java
Outdated
Show resolved
Hide resolved
williamrandolph
commented
Oct 21, 2021
.../java/org/elasticsearch/action/admin/indices/settings/put/TransportUpdateSettingsAction.java
Outdated
Show resolved
Hide resolved
williamrandolph
commented
Oct 21, 2021
server/src/main/java/org/elasticsearch/cluster/metadata/IndexMetadata.java
Outdated
Show resolved
Hide resolved
williamrandolph
commented
Oct 21, 2021
server/src/main/java/org/elasticsearch/cluster/metadata/IndexMetadata.java
Outdated
Show resolved
Hide resolved
williamrandolph
commented
Oct 21, 2021
server/src/main/java/org/elasticsearch/cluster/metadata/MetadataCreateIndexService.java
Outdated
Show resolved
Hide resolved
williamrandolph
commented
Oct 21, 2021
server/src/main/java/org/elasticsearch/cluster/metadata/MetadataUpdateSettingsService.java
Outdated
Show resolved
Hide resolved
williamrandolph
commented
Oct 21, 2021
server/src/test/java/org/elasticsearch/indices/SystemIndexManagerTests.java
Outdated
Show resolved
Hide resolved
AthenaEryma
approved these changes
Dec 3, 2021
Contributor
AthenaEryma
left a comment
There was a problem hiding this comment.
LGTM, thanks for your hard work and iteration on this one! Chasing down everywhere that needs to be changed for something like this isn't easy.
Collaborator
|
Pinging @elastic/es-core-infra (Team:Core/Infra) |
Contributor
Author
|
@elasticmachine test this please |
Collaborator
|
Hi @williamrandolph, I've created a changelog YAML for you. |
williamrandolph
added a commit
to williamrandolph/elasticsearch
that referenced
this pull request
Dec 6, 2021
System indices should be hidden from users. Since they are already restricted indices, a users that can't view restricted indices already can't see or access them, but they should also be hidden for superusers or users that are otherwise granted advanced privileges. To the greatest degree possible, we apply hidden settings in the transport layer, so that the system can create an index or alias that is set to visible, for example, when operating in a mixed cluster mode. However, in the case of aliases created by templates, we hide the alias in the service layer. This change has broken a number of tests that were relaying unnecessarily on wildcard searches. In general, the fix for these issues was to apply expand_wildcards=open,hidden to the request. * Force system indices to be hidden in IndexMetadata * Hide system data streams * Update feature migration tests * ML datafeed config defaults to searching hidden indices * Prevent unmanaged system indices from becoming visible * Change validation in TransportUpdateSettingsAction * Validate index creation settings in transport action * Make sure system data stream backing indices are hidden * Make sure transport request adds hidden index setting if missing * Validate and set default for autocreated system indices * Add some code to hide system aliases * Hide system aliases in create index service * Hide system aliases when adding them via alias endpoints * Check system indices when simulating and validating templates * Add known issue for reenabling tests * Update docs/changelog/79512.yaml
Collaborator
💚 Backport successful
|
williamrandolph
added a commit
that referenced
this pull request
Dec 6, 2021
System indices should be hidden from users. Since they are already restricted indices, a users that can't view restricted indices already can't see or access them, but they should also be hidden for superusers or users that are otherwise granted advanced privileges. To the greatest degree possible, we apply hidden settings in the transport layer, so that the system can create an index or alias that is set to visible, for example, when operating in a mixed cluster mode. However, in the case of aliases created by templates, we hide the alias in the service layer. This change has broken a number of tests that were relaying unnecessarily on wildcard searches. In general, the fix for these issues was to apply expand_wildcards=open,hidden to the request. * Force system indices to be hidden in IndexMetadata * Hide system data streams * Update feature migration tests * ML datafeed config defaults to searching hidden indices * Prevent unmanaged system indices from becoming visible * Change validation in TransportUpdateSettingsAction * Validate index creation settings in transport action * Make sure system data stream backing indices are hidden * Make sure transport request adds hidden index setting if missing * Validate and set default for autocreated system indices * Add some code to hide system aliases * Hide system aliases in create index service * Hide system aliases when adding them via alias endpoints * Check system indices when simulating and validating templates * Add known issue for reenabling tests * Update docs/changelog/79512.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
System indices should be hidden from users. Since they are already restricted indices, a users that can't view restricted indices already can't see or access them, but they should also be hidden for superusers or users that are otherwise granted advanced privileges.
To the greatest degree possible, we apply hidden settings in the transport layer, so that the system can create an index or alias that is set to visible, for example, when operating in a mixed cluster mode. However, in the case of aliases created by templates, we hide the alias in the service layer.
This change has broken a number of tests that were relaying unnecessarily on wildcard searches. In general, the fix for these issues was to apply
expand_wildcards=open,hiddento the request.