Describe the bug
This might be intended behavior based on the existence of Nested Aggregations, but I'm not quite sure. If it is, let my bug report and my folly be a guiding light for others hitting this issue.
When trying to do a composite bucket aggregation on a "type": "nested" field, it always reports the values as missing. If I do the same thing but without "type": "nested" in the mapping, the aggregation works as expected.
Related component
Search
To Reproduce
- Mapping: Any nested field
% cat test_map.json | jq
{
"mappings": {
"properties": {
"rec": {
"type": "nested",
"properties": {
"inner": {
"type": "integer"
}
}
}
}
}
}
% cat test_map.json | xh put localhost:9200/test
HTTP/1.1 200 OK
Content-Encoding: gzip
Content-Length: 71
Content-Type: application/json; charset=UTF-8
X-Opensearch-Version: OpenSearch/3.4.0-SNAPSHOT (opensearch)
{
"acknowledged": true,
"shards_acknowledged": true,
"index": "test"
}
- Any data
% cat test_data.json
{"create": {"_index": "test"}}
{"rec":{"inner": 100}}
{"create": {"_index": "test"}}
{"rec":{"inner": 101}}
{"create": {"_index": "test"}}
{"rec":{"inner": 101}}
% cat test_data.json | xh post localhost:9200/_bulk
HTTP/1.1 200 OK
Content-Encoding: gzip
Content-Length: 213
Content-Type: application/json; charset=UTF-8
X-Opensearch-Version: OpenSearch/3.4.0-SNAPSHOT (opensearch)
{
"took": 12,
"errors": false,
"items": [
{
"create": {
"_index": "test",
"_id": "1Wq-D5sB_TsnXFFLJyHD",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 0,
"_primary_term": 1,
"status": 201
}
},
{
"create": {
"_index": "test",
"_id": "1mq-D5sB_TsnXFFLJyHD",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 1,
"_primary_term": 1,
"status": 201
}
},
{
"create": {
"_index": "test",
"_id": "12q-D5sB_TsnXFFLJyHD",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 2,
"_primary_term": 1,
"status": 201
}
}
]
}
- Bucket it
% cat test_search.json | jq
{
"from": 0,
"size": 0,
"timeout": "1m",
"aggregations": {
"composite_buckets": {
"composite": {
"size": 1000,
"sources": [
{
"rec.inner": {
"terms": {
"field": "rec.inner",
"missing_bucket": true,
"missing_order": "first",
"order": "asc"
}
}
}
]
}
}
}
}
% cat test_search.json | xh post localhost:9200/test/_search
HTTP/1.1 200 OK
Content-Encoding: gzip
Content-Length: 198
Content-Type: application/json; charset=UTF-8
X-Opensearch-Version: OpenSearch/3.4.0-SNAPSHOT (opensearch)
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 3,
"relation": "eq"
},
"max_score": null,
"hits": []
},
"aggregations": {
"composite_buckets": {
"after_key": {
"rec.inner": null
},
"buckets": [
{
"key": {
"rec.inner": null
},
"doc_count": 3
}
]
}
}
}Expected behavior
When the default nested field behavior is used (i.e. delete "type": "nested" from the mapping), the correct results come back:
% cat test_search.json | xh post localhost:9200/test/_search
HTTP/1.1 200 OK
Content-Encoding: gzip
Content-Length: 228
Content-Type: application/json; charset=UTF-8
X-Opensearch-Version: OpenSearch/3.4.0-SNAPSHOT (opensearch)
{
"took": 6,
"timed_out": false,
"terminated_early": true,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 3,
"relation": "eq"
},
"max_score": null,
"hits": []
},
"aggregations": {
"composite_buckets": {
"after_key": {
"rec.inner": 101
},
"buckets": [
{
"key": {
"rec.inner": 100
},
"doc_count": 1
},
{
"key": {
"rec.inner": 101
},
"doc_count": 2
}
]
}
}
}Additional Details
Plugins
N/A
Screenshots
N/A
Host/Environment (please complete the following information):
- OS: Linux 5.10
- Version: 3.3.2, mainline (JDK Temurin 21)
Additional context
Root cause for a PPL bug: several commands rely on composite buckets. SQL-side issue: opensearch-project/sql#4949
Describe the bug
This might be intended behavior based on the existence of Nested Aggregations, but I'm not quite sure. If it is, let my bug report and my folly be a guiding light for others hitting this issue.
When trying to do a composite bucket aggregation on a
"type": "nested"field, it always reports the values as missing. If I do the same thing but without"type": "nested"in the mapping, the aggregation works as expected.Related component
Search
To Reproduce
Expected behavior
When the default nested field behavior is used (i.e. delete
"type": "nested"from the mapping), the correct results come back:Additional Details
Plugins
N/A
Screenshots
N/A
Host/Environment (please complete the following information):
Additional context
Root cause for a PPL bug: several commands rely on composite buckets. SQL-side issue: opensearch-project/sql#4949