The superuser built-in role doesn't work with an unavailable .security index #85312
Description
Elasticsearch Version
8.*, 7.17.*
Problem Description
We used to think and advise that a file-based superuser can rescue any cluster with a botched .security-* index.
Alas this is not true, in general.
If the .security-7 index exists, but it is closed or hasn't got the primary assigned, the superuser role assigned to a file-based user will not work because it includes (all) application privileges, which themselves are stored in the .security-* index and resolving them would fail the authentication.
Steps to Reproduce
Disable role-related caching in elasticsearch.yml:
xpack.security.authz.store.privileges.cache.ttl: 0
xpack.security.authz.store.roles.cache.max_size: 0
xpack.security.authz.store.roles.negative_lookup_cache.max_size: 0
Add a new super role to the config/roles.yml that can close the .security-7 index:
supersuperuser:
run_as: [ '*' ]
cluster: [ 'all' ]
indices:
- names: [ '*' ]
privileges: [ 'all' ]
allow_restricted_indices: true
Create a new file-based user:
./bin/elasticsearch-users useradd test -p password -r superuser,supersuperuser
Close the .security-7 (in order to make it unavailable):
curl -k -u test:password -X POST "https://localhost:9200/.security-7/_close"
In this state, all file-based users with the superuser role AND some other role (could be an empty role) will fail to authenticate:
curl -k -u test:password -X GET "https://localhost:9200/_security/_authenticate?pretty"
{
"error" : {
"root_cause" : [
{
"type" : "index_closed_exception",
"reason" : "closed",
"index_uuid" : "_na_",
"index" : ".security-7"
}
],
"type" : "index_closed_exception",
"reason" : "closed",
"index_uuid" : "_na_",
"index" : ".security-7"
},
"status" : 400
}
with a sample stack trace extract:
"stack_trace" : "[.security-7] org.elasticsearch.indices.IndexClosedException: closed
at org.elasticsearch.xpack.security.support.SecurityIndexManager.getUnavailableReason(SecurityIndexManager.java:137)
at org.elasticsearch.xpack.security.authz.store.NativePrivilegeStore.innerGetPrivileges(NativePrivilegeStore.java:182)
at org.elasticsearch.xpack.security.authz.store.NativePrivilegeStore.getPrivileges(NativePrivilegeStore.java:163)
at org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.buildRoleFromDescriptors(CompositeRolesStore.java:464)
at org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.buildThenMaybeCacheRole(CompositeRolesStore.java:320)
at org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.lambda$buildRoleFromRoleReference$2(CompositeRolesStore.java:249)
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:136)
at org.elasticsearch.xpack.security.authz.store.RoleDescriptorStore.lambda$resolveRoleNames$3(RoleDescriptorStore.java:168)
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:136)
at org.elasticsearch.xpack.security.authz.store.RoleDescriptorStore.lambda$loadRoleDescriptorsAsync$8(RoleDescriptorStore.java:207)
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:136)
at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:31)
at org.elasticsearch.xpack.core.common.IteratingActionListener.onResponse(IteratingActionListener.java:141)
Interestingly it works if the user ONLY has the superuser role, because of an optimisation in the authz code.