Skip to content

Fix Aggregation schema structure and ValueType enum to match OpenSearch implementation#1060

Merged
karenyrx merged 3 commits intoopensearch-project:mainfrom
karenyrx:aggsfixes
Mar 3, 2026
Merged

Fix Aggregation schema structure and ValueType enum to match OpenSearch implementation#1060
karenyrx merged 3 commits intoopensearch-project:mainfrom
karenyrx:aggsfixes

Conversation

@karenyrx
Copy link
Copy Markdown
Collaborator

@karenyrx karenyrx commented Feb 27, 2026

Description

1. Aggregation Name Field Error

Problem: The schema incorrectly modeled aggregation names as object properties with a name field.

Before:

Aggregation:
  properties:
    name:
      type: string
    # ... other properties

Reality: Aggregation names are JSON keys, not object properties:

{
  "my_agg_name": {     // ← Name is the key
    "terms": {...}
  }
}

Fix: Removed the unnamed name field from the Aggregation schema. The name is represented by the key in the parent object's additionalProperties or propertyNames.

Reference: Java implementation uses builder.startObject(name) in AbstractAggregationBuilder, not builder.field("name", name).

2. ValueType Enum Mismatch

Problem: The ValueType enum included geo_point and range which cannot be parsed from REST API requests.

Evidence: In ValueType.java, the lenientParse() method only accepts:

  • "string", "long", "integer", "short", "double", "float", "date", "ip", "boolean", "unsigned_long", "number", "numeric"
  • Does NOT include "geo_point" or "range"

These types are internal-only (used in Java code) but cannot be specified by API users.

Fix: Removed geo_point and range from the ValueType enum.

3. BucketAggregationBase oneOf Validation Bug

Problem: Aggregations without sub-aggregations failed validation with error: "MUST contain the missing properties: adjacency_matrix, auto_date_histogram, avg..."

Root Cause: The oneOf constraint had only 2 branches without proper required fields:

oneOf:
  - properties: {aggregations: ...}  # No required
  - properties: {aggs: ...}          # No required

4. Filter Aggregation Sub-aggregation Support

Problem: The filter aggregation was defined as a simple property without extending BucketAggregationBase, preventing it from having sub-aggregations.

Before:

- properties:
    filter:
      $ref: '_common.query_dsl.yaml#/components/schemas/QueryContainer'
  required: [filter]
  unevaluatedProperties: true

This only allowed the filter property. Using sub-aggregations like this would fail:

{
  "moviesWithHiddenCost": {
    "filter": {"term": {"cost_hidden": true}},
    "aggs": {                          // <-- Not allowed
      "numOfDirectors": {
        "cardinality": {"field": "director_id"}
      }
    }
  }
}

Fix: Changed to allOf that extends BucketAggregationBase:

- allOf:
    - $ref: '#/components/schemas/BucketAggregationBase'  # ← Provides sub-agg support
    - type: object
      properties:
        filter:
          $ref: '_common.query_dsl.yaml#/components/schemas/QueryContainer'
      required: [filter]
  x-protobuf-excluded: true

5. Request/Response Alias Handling

Context: Both aggs and aggregations are accepted in requests, but only aggregations appears in responses.

Fix: Added x-protobuf-excluded: true to the aggs property to exclude it from protobuf generation, since it's just an alias that only makes sense on the REST but not protobuf side.

Issues Resolved

List any issues this PR will resolve, e.g. Closes [...].

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions
Copy link
Copy Markdown
Contributor

Proto Compatibility Report

Merge Report

No changes detected.


Generated by Proto Compatibility Check

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Feb 27, 2026

Changes Analysis

Commit SHA: 6fc9c48
Comparing To SHA: 5f6683c

API Changes

Summary

├─┬Paths
│ ├─┬/{index}/_search
│ │ ├─┬GET
│ │ │ └─┬Request Body
│ │ │   └─┬Content
│ │ │     └─┬application/json
│ │ │       └─┬Schema
│ │ │         └─┬Properties
│ │ │           └─┬aggs
│ │ │             └─┬Extensions
│ │ │               └──[➕] x-protobuf-excluded/true (32732:38)
│ │ └─┬POST
│ │   └─┬Request Body
│ │     └─┬Content
│ │       └─┬application/json
│ │         └─┬Schema
│ │           └─┬Properties
│ │             └─┬aggs
│ │               └─┬Extensions
│ │                 └──[➕] x-protobuf-excluded/true (32732:38)
│ └─┬/_search
│   ├─┬GET
│   │ └─┬Request Body
│   │   └─┬Content
│   │     └─┬application/json
│   │       └─┬Schema
│   │         └─┬Properties
│   │           └─┬aggs
│   │             └─┬Extensions
│   │               └──[➕] x-protobuf-excluded/true (32732:38)
│   └─┬POST
│     └─┬Request Body
│       └─┬Content
│         └─┬application/json
│           └─┬Schema
│             └─┬Properties
│               └─┬aggs
│                 └─┬Extensions
│                   └──[➕] x-protobuf-excluded/true (32732:38)
└─┬Components
  └─┬Schemas
    ├─┬_common.aggregations___BucketAggregationBase
    │ └─┬ALLOF
    │   ├──[➖] properties/aggregations (42175:13)❌ 
    │   ├──[➖] properties/aggs (42180:13)❌ 
    │   ├──[➕] oneOf (42174:15)
    │   ├──[➕] oneOf (42187:15)
    │   └──[➕] oneOf (42201:15)
    ├─┬_common.aggregations___Aggregation
    │ └──[➖] properties/name (41626:9)❌ 
    ├─┬_common.aggregations___AggregationContainer
    │ └─┬ALLOF
    │   └─┬ONEOF
    │     ├──[➖] required (41754:19)❌ 
    │     ├──[➖] unevaluatedProperties/true (41755:38)❌ 
    │     ├──[➖] properties/filter (41751:17)❌ 
    │     ├──[➕] allOf (42170:7)
    │     └──[➕] allOf (41749:19)
    └─┬_common.aggregations___ValueType
      ├──[➖] enum (44810:11)❌ 
      └──[➖] enum (44816:11)❌ 

Document Element Total Changes Breaking Changes
paths 4 0
components 13 13
  • BREAKING Changes: 13 out of 17
  • Removals: 8
  • Additions: 9
  • Breaking Removals: 8

Report

The full API changes report is available at: https://github.com/opensearch-project/opensearch-api-specification/actions/runs/22483516336/artifacts/5688839474

API Coverage

Before After Δ
Covered (%) 666 (65.23 %) 666 (65.23 %) 0 (0 %)
Uncovered (%) 355 (34.77 %) 355 (34.77 %) 0 (0 %)
Unknown 149 149 0

Comment thread CHANGELOG.md

- Fixed aggregation schema strict mode compliance: restructured `MetricAggregationBase` and `TermsAggregation` by moving `field` and `script` into optional `anyOf` branches; added `value_type` to `MinAggregation` and `MaxAggregation`; updated `ValueType` enum (added `byte`, `float`, `integer`, `range`, `short`, `unsigned_long`; removed `date_nanos`); changed `min_doc_count` and `shard_min_doc_count` in `TermsAggregation` from int32 to int64 ([#1056](https://github.com/opensearch-project/opensearch-api-specification/pull/1056))
- Fixed `TermsAggregation` value_type type from `string` to `ValueType`, enforce enum ([#1057](https://github.com/opensearch-project/opensearch-api-specification/pull/1057))
- Exclude FilterContainer and order in TermsAggregation in spec ([#1059](https://github.com/opensearch-project/opensearch-api-specification/pull/1059))
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR missed to add a changelog entry

Comment thread CHANGELOG.md
- Fixed `TermsAggregation` value_type type from `string` to `ValueType`, enforce enum

- Fixed aggregation schema strict mode compliance: restructured `MetricAggregationBase` and `TermsAggregation` by moving `field` and `script` into optional `anyOf` branches; added `value_type` to `MinAggregation` and `MaxAggregation`; updated `ValueType` enum (added `byte`, `float`, `integer`, `range`, `short`, `unsigned_long`; removed `date_nanos`); changed `min_doc_count` and `shard_min_doc_count` in `TermsAggregation` from int32 to int64 ([#1056](https://github.com/opensearch-project/opensearch-api-specification/pull/1056))
- Fixed `TermsAggregation` value_type type from `string` to `ValueType`, enforce enum ([#1057](https://github.com/opensearch-project/opensearch-api-specification/pull/1057))
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR did not have PR # in entry, so backfilling now

@karenyrx karenyrx changed the title Aggsfixes Aggregation fixes: Fix AggregationContainer to inherit Aggregation type, and remove unnamed field name from Aggregation. Remove geo_point and range from ValueType. Exclude aggs alias from protobufs Feb 27, 2026
…unnamed field `name` from `Aggregation`. Remove `geo_point` and `range` from `ValueType`. Exclude `aggs` alias from protobufs.

Signed-off-by: Karen X <karenxyr@gmail.com>
@karenyrx karenyrx changed the title Aggregation fixes: Fix AggregationContainer to inherit Aggregation type, and remove unnamed field name from Aggregation. Remove geo_point and range from ValueType. Exclude aggs alias from protobufs [Aggregation fixes] Fix AggregationContainer to inherit Aggregation type, and remove unnamed field name from Aggregation. Remove geo_point and range from ValueType. Exclude aggs alias from protobufs Feb 27, 2026
Signed-off-by: Karen X <karenxyr@gmail.com>
@karenyrx karenyrx force-pushed the aggsfixes branch 4 times, most recently from e58b5b0 to 3a784d2 Compare February 27, 2026 10:57
Signed-off-by: Karen X <karenxyr@gmail.com>
required: [filter]
unevaluatedProperties: true
- allOf:
- $ref: '#/components/schemas/BucketAggregationBase'
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So now a filter aggregation can have:

  • filter property (the query)
  • aggs or aggregations property (sub-aggregations)
  • meta property (metadata)

title: aggregation_container
$ref: '#/components/schemas/AggregationContainer'
required: [aggs]
- not:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need not/anyOf here, isn't oneOf good enough

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the third branch (the not/anyOf) is to account for the case where neither aggregations nor aggs is present (i.e. no sub-aggregations)

@karenyrx karenyrx changed the title [Aggregation fixes] Fix AggregationContainer to inherit Aggregation type, and remove unnamed field name from Aggregation. Remove geo_point and range from ValueType. Exclude aggs alias from protobufs Fix Aggregation schema structure and ValueType enum to match OpenSearch implementation Mar 3, 2026
@karenyrx karenyrx marked this pull request as ready for review March 3, 2026 07:58
@karenyrx karenyrx merged commit bc18459 into opensearch-project:main Mar 3, 2026
68 of 70 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants