Bug Description
Commit b0c07008f8 ("Support aggregation push down with scripts (#3916)") introduced a change that causes queries with complex expressions in aggregation context to fail due to OpenSearch's 65KB inline script size limit. The aggregation push-down feature now serializes large script payloads, often exceeding 1.2MB, resulting in runtime failures.
Related: PR #3878 triggers this issue (check CI for details)
Error Message
java.lang.IllegalArgumentException: exceeded max allowed inline script size in bytes [65535] with size [1224050] for script
Root Cause Analysis
The issue originates from AggregateBuilderHelper.build() in AggregateAnalyzer.java:
<T> T build(RexNode node, Function<String, T> fieldBuilder, Function<Script, T> scriptBuilder) {
if (node instanceof RexCall || node instanceof RexLiteral) {
// Problem: complex expressions generate huge serialized script objects
return scriptBuilder.apply(
new PredicateAnalyzer.ScriptQueryExpression(node, rowType, fieldTypes, cluster).getScript());
}
}
Technical Context
The original bin command implementation for bins, minspan and start/end used window functions (MIN() OVER() and MAX() OVER()) to calculate data range. This produced a massive RexCall expression tree with deeply nested operations.
When aggregation push-down was introduced, any RexCall in the grouping expression got serialized into an inline script for OpenSearch.
The serialized script ballooned to over 1.2MB, far exceeding OpenSearch's 65KB inline script limit.
CC: @qianheng-aws
Bug Description
Commit
b0c07008f8("Support aggregation push down with scripts (#3916)") introduced a change that causes queries with complex expressions in aggregation context to fail due to OpenSearch's 65KB inline script size limit. The aggregation push-down feature now serializes large script payloads, often exceeding 1.2MB, resulting in runtime failures.Related: PR #3878 triggers this issue (check CI for details)
Error Message
Root Cause Analysis
The issue originates from
AggregateBuilderHelper.build()inAggregateAnalyzer.java:Technical Context
The original
bincommand implementation forbins,minspanandstart/endused window functions (MIN() OVER()andMAX() OVER()) to calculate data range. This produced a massive RexCall expression tree with deeply nested operations.When aggregation push-down was introduced, any RexCall in the grouping expression got serialized into an inline script for OpenSearch.
The serialized script ballooned to over 1.2MB, far exceeding OpenSearch's 65KB inline script limit.
CC: @qianheng-aws