Field-level security with inner_hits can cause index out-of-bounds exception #30624
Description
Elasticsearch version (bin/elasticsearch --version):
Version: 6.2.4, Build: ccec39f/2018-04-12T20:37:28.497551Z
Plugins installed:
x-pack
x-pack-core
x-pack-deprecation
x-pack-graph
x-pack-logstash
x-pack-ml
x-pack-monitoring
x-pack-security
x-pack-upgrade
x-pack-watcher
JVM version (java -version):
java 9
Java(TM) SE Runtime Environment (build 9+181)
Java HotSpot(TM) 64-Bit Server VM (build 9+181, mixed mode)
OS version (uname -a if on a Unix-like system):
Windows 10
Description of the problem including expected versus actual behavior:
Running a query with field-level security on a subfield of a nested array field and asking for inner_hits can cause an index out of bounds exception.
Expected behavior: no error. (If this is a known limitation of either inner_hits or field-level security then I couldn't see it in the documentation.)
Steps to reproduce:
# create a new index
curl -XPUT localhost:9200/test
# define a mapping
curl -XPUT localhost:9200/test/_mapping/document -d '{
"properties": {
"document_name": {
"type": "text"
},
"metadata": {
"type": "nested",
"properties": {
"name": {"type": "text"},
"sv": {"type": "text"},
"nv": {"type": "double"}
}
}
}
}'
# index a document:
# it has one 'metadata' value where nv is populated and another where sv is populated
curl -XPOST localhost:9200/test/document -d '{
"document_name": "testing",
"metadata": [
{
"name": "numeric",
"nv": 0.2
},
{
"name": "string",
"sv": "problem"
}
]
}'
# run this query; it should retrieve the document okay including the inner_hits
# metadata[1] is the inner hit which matches; metadata[0] doesn't match
curl -XPOST localhost:9200/test/_search -d '{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "metadata",
"inner_hits":{},
"score_mode": "avg",
"query": {
"bool": {
"must": [
{
"match": {
"metadata.sv": "problem"
}
}
]
}
}
}
}
]
}
}
}'
# create a role which can only see metadata.sv
curl -XPUT localhost:9200/_xpack/security/role/reproduce -d '{
"indices": [
{
"names": [ "test" ],
"privileges": [ "read" ],
"field_security" : {
"grant": [ "metadata.sv" ]
}
}
]
}'
# create a user in that role
curl -XPUT localhost:9200/_xpack/security/user/repro-user -d '{
"roles": ["reproduce"],
"password": "whatever",
"full_name": "Repro User",
"email": "blah@blah.blah"
}'
Now, if you rerun the previous query but authenticating as this new user, instead of getting results, it fails:
"failures": [
{
...
"reason": {
"type": "index_out_of_bounds_exception",
"reason": "Index 1 out-of-bounds for length 1"
}
}
]
Removing "inner_hits":{} from the query will make it work.
My guess is that it figures out that metadata[1] is the inner hit which matches, then strips out the fields you aren't allowed to see, which removes metadata[0] since it doesn't have a metadata.sv field, which leaves the metadata array one element long instead of two, so when it then looks up metadata[1] to put it in the inner hits node, it goes bang?