Support `split` eval function by ahkcs · Pull Request #4814 · opensearch-project/sql

ahkcs · 2025-11-14T22:21:48Z

Summary

This PR implements the split eval function for PPL, enabling users to split strings into multivalue arrays based on a delimiter.

Examples

Basic split with semicolon

source=people | eval result = split('a;b;c', ';')

Result: ['a', 'b', 'c']

Split into individual characters (empty delimiter)

source=people | eval result = split('abcd', '')

Result: ['a', 'b', 'c', 'd']

Multi-character delimiter

source=people | eval result = split('name::value', '::')

Result: ['name', 'value']

Split field value

source=people | eval words = split(employer, ' ')

Splits the employer field on spaces.

Check List

New functionality includes testing.
New functionality has been documented.
New functionality has javadoc added.
New functionality has a user manual doc added.
New PPL command checklist all confirmed.
API changes companion pull request created.
Commits are signed per the DCO using --signoff or -s.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

dai-chen

Can this be reused? https://issues.apache.org/jira/browse/CALCITE-6951

ahkcs · 2025-11-18T22:06:41Z

https://issues.apache.org/jira/browse/CALCITE-6951

Hi @dai-chen , currently we are using SPLIT, which is also supported by Calcite. And we did some special handling for empty delimiter on top of that.

Here's the documentation for Calcite SPLIT function:


SPLIT(string [, delimiter ])

Returns the string array of string split at delimiter (if omitted, default is comma). If the string is empty it returns an empty array, otherwise, if the delimiter is empty, it returns an array containing the original string.

dai-chen · 2025-11-19T23:05:56Z

https://issues.apache.org/jira/browse/CALCITE-6951

Hi @dai-chen , currently we are using SPLIT, which is also supported by Calcite. And we did some special handling for empty delimiter on top of that.

Here's the documentation for Calcite SPLIT function:
SPLIT(string [, delimiter ]) 
Returns the string array of string split at delimiter (if omitted, default is comma). If the string is empty it returns an empty array, otherwise, if the delimiter is empty, it returns an array containing the original string.

I see. So the only reason of SplitFunctionImp is special handling for delimiter="", right?
Can the extract function below help?

Case 1: Delimiter is not empty string, translate split to

SELECT SPLIT('a;b;c;d', ';');
+--------------+
|    EXPR$0    |
+--------------+
| [a, b, c, d] |
+--------------+

Case 2: Delimiter is empty string, translate split to:

SELECT REGEXP_EXTRACT_ALL('abcd', '.');
+--------------+
|    EXPR$0    |
+--------------+
| [a, b, c, d] |
+--------------+

ahkcs · 2025-11-19T23:28:52Z

@dai-chen Thanks for the suggestion! I think it makes sense. I have updated the PR to move the implementation to PPLFuncImpTable

docs/user/ppl/functions/collection.rst

dai-chen

Thanks for the changes!

Signed-off-by: Kai Huang <ahkcs@amazon.com> # Conflicts: # core/src/main/java/org/opensearch/sql/expression/function/BuiltinFunctionName.java # integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalciteArrayFunctionIT.java # ppl/src/main/antlr/OpenSearchPPLLexer.g4 # ppl/src/main/antlr/OpenSearchPPLParser.g4 # ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLArrayFunctionTest.java # ppl/src/test/java/org/opensearch/sql/ppl/utils/PPLQueryDataAnonymizerTest.java

Signed-off-by: Kai Huang <ahkcs@amazon.com>

ykmr1224

LGTM

* Support split eval function Signed-off-by: Kai Huang <ahkcs@amazon.com> # Conflicts: # core/src/main/java/org/opensearch/sql/expression/function/BuiltinFunctionName.java # integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalciteArrayFunctionIT.java # ppl/src/main/antlr/OpenSearchPPLLexer.g4 # ppl/src/main/antlr/OpenSearchPPLParser.g4 # ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLArrayFunctionTest.java # ppl/src/test/java/org/opensearch/sql/ppl/utils/PPLQueryDataAnonymizerTest.java * doctest Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update test cases Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update to not use UDF Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> (cherry picked from commit 5dca84f) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Support split eval function Signed-off-by: Kai Huang <ahkcs@amazon.com> # Conflicts: # core/src/main/java/org/opensearch/sql/expression/function/BuiltinFunctionName.java # integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalciteArrayFunctionIT.java # ppl/src/main/antlr/OpenSearchPPLLexer.g4 # ppl/src/main/antlr/OpenSearchPPLParser.g4 # ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLArrayFunctionTest.java # ppl/src/test/java/org/opensearch/sql/ppl/utils/PPLQueryDataAnonymizerTest.java * doctest Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update test cases Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update to not use UDF Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

* Support split eval function Signed-off-by: Kai Huang <ahkcs@amazon.com> * doctest Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update test cases Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update to not use UDF Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

* Support split eval function Signed-off-by: Kai Huang <ahkcs@amazon.com> # Conflicts: # core/src/main/java/org/opensearch/sql/expression/function/BuiltinFunctionName.java # integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalciteArrayFunctionIT.java # ppl/src/main/antlr/OpenSearchPPLLexer.g4 # ppl/src/main/antlr/OpenSearchPPLParser.g4 # ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLArrayFunctionTest.java # ppl/src/test/java/org/opensearch/sql/ppl/utils/PPLQueryDataAnonymizerTest.java * doctest Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update test cases Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update to not use UDF Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com>

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

ahkcs marked this pull request as ready for review November 18, 2025 17:48

dai-chen reviewed Nov 18, 2025

View reviewed changes

dai-chen added enhancement New feature or request PPL Piped processing language labels Nov 18, 2025

dai-chen reviewed Nov 19, 2025

View reviewed changes

docs/user/ppl/functions/collection.rst Show resolved Hide resolved

dai-chen previously approved these changes Nov 20, 2025

View reviewed changes

ahkcs added 4 commits November 24, 2025 11:18

doctest

8b62cf8

Signed-off-by: Kai Huang <ahkcs@amazon.com>

Update test cases

0f070e2

Signed-off-by: Kai Huang <ahkcs@amazon.com>

Update to not use UDF

444cb29

Signed-off-by: Kai Huang <ahkcs@amazon.com>

ahkcs dismissed dai-chen’s stale review via 444cb29 November 24, 2025 19:20

ahkcs force-pushed the feat/eval_split branch from aa898e5 to 444cb29 Compare November 24, 2025 19:20

dai-chen approved these changes Nov 25, 2025

View reviewed changes

dai-chen added the backport 2.19-dev label Nov 25, 2025

ykmr1224 approved these changes Dec 8, 2025

View reviewed changes

dai-chen merged commit 5dca84f into opensearch-project:main Dec 8, 2025
37 checks passed

opensearch-trigger-bot bot mentioned this pull request Dec 8, 2025

[Backport 2.19-dev] Support split eval function #4918

Merged

Swiddis pushed a commit that referenced this pull request Dec 10, 2025

Support split eval function (#4814) (#4918)

effe78d

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

mengweieric mentioned this pull request Mar 10, 2026

Add ahkcs as maintainer #5223

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support `split` eval function#4814

Support `split` eval function#4814
dai-chen merged 4 commits intoopensearch-project:mainfrom
ahkcs:feat/eval_split

ahkcs commented Nov 14, 2025 •

edited

Loading

Uh oh!

dai-chen left a comment

Uh oh!

ahkcs commented Nov 18, 2025 •

edited

Loading

Uh oh!

dai-chen commented Nov 19, 2025

Uh oh!

ahkcs commented Nov 19, 2025 •

edited

Loading

Uh oh!

Uh oh!

dai-chen left a comment

Uh oh!

ykmr1224 left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ahkcs commented Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Examples

Basic split with semicolon

Split into individual characters (empty delimiter)

Multi-character delimiter

Split field value

Check List

Uh oh!

dai-chen left a comment

Choose a reason for hiding this comment

Uh oh!

ahkcs commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dai-chen commented Nov 19, 2025

Uh oh!

ahkcs commented Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

dai-chen left a comment

Choose a reason for hiding this comment

Uh oh!

ykmr1224 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ahkcs commented Nov 14, 2025 •

edited

Loading

ahkcs commented Nov 18, 2025 •

edited

Loading

ahkcs commented Nov 19, 2025 •

edited

Loading