Feature tonumber : issue #4514 tonumber function as part of roadmap #4287 by asifabashar · Pull Request #3 · asifabashar/sql

asifabashar · 2025-12-10T18:40:45Z

Description

[Describe what this change achieves]

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

New functionality includes testing.
New functionality has been documented.
New functionality has javadoc added.
New functionality has a user manual doc added.
New PPL command checklist all confirmed.
API changes companion pull request created.
Commits are signed per the DCO using --signoff or -s.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

… default duration is in seconds which will be used for duration format , added cast call for tostring with 1 argument Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

…ensearch-project#4287 Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com> Signed-off-by: Asif Bashar <abashar@apple.com> Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

* fix anoymizer for search command Signed-off-by: xinyual <xinyual@amazon.com> * pushdown match when only one equal in search command Signed-off-by: xinyual <xinyual@amazon.com> * fix regex case Signed-off-by: xinyual <xinyual@amazon.com> * fix UT Signed-off-by: xinyual <xinyual@amazon.com> * fix UT Signed-off-by: xinyual <xinyual@amazon.com> * revert match change Signed-off-by: xinyual <xinyual@amazon.com> * fix UT by ignore the expression Signed-off-by: xinyual <xinyual@amazon.com> * remove useless change and resolve comment Signed-off-by: xinyual <xinyual@amazon.com> * remove useless change and resolve comment Signed-off-by: xinyual <xinyual@amazon.com> * add test cases for metadata and timestamp identifier Signed-off-by: xinyual <xinyual@amazon.com> * change name Signed-off-by: xinyual <xinyual@amazon.com> --------- Signed-off-by: xinyual <xinyual@amazon.com>

Signed-off-by: Lantao Jin <ltjin@amazon.com>

…pensearch-project#4843) * Remove redundant push-down-filters derived for bucket-non-null aggregation Signed-off-by: Heng Qian <qianheng@amazon.com> * Address comment Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix name typo Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com>

… are different (opensearch-project#4850) Signed-off-by: Lantao Jin <ltjin@amazon.com>

* Support eval function Signed-off-by: Kai Huang <ahkcs@amazon.com> * Updates Signed-off-by: Kai Huang <ahkcs@amazon.com> * update javadoc Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update to use ARRAY_DISTINCT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com>

…oject#4837) * Change like function to case-sensitive and add ilike for case-insensitive Signed-off-by: Lantao Jin <ltjin@amazon.com> * change plan in clickbench Signed-off-by: Lantao Jin <ltjin@amazon.com> * fix ut Signed-off-by: Lantao Jin <ltjin@amazon.com> * fix UT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Update syntax: like(string, PATTERN[, case_sensitive]) Signed-off-by: Lantao Jin <ltjin@amazon.com> * fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * revert import merging Signed-off-by: Lantao Jin <ltjin@amazon.com> * support like(string, PATTERN, bool) in v2 Signed-off-by: Lantao Jin <ltjin@amazon.com> * dedup ut Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com>

…project#4831) * add bucket_nullable for streamstats Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * little change Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix IT, UT and doc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change bucketNullable getOrDefault to get Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com>

…arch-project#4784) * Support param timefield to specify span field in timechart Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update doc to introduce timefield parameter Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update ASTBuilderTest for chart: default args are handled in rel node visitor Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix ast expression builder test Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix anomanyzer test Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Support using specified timefield in per functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Omit by-timestamp clause in timechart command Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Mask timefield argument in anonymizer Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Anonymize argument span Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

…4844) * Enable dedup pushdown Signed-off-by: Lantao Jin <ltjin@amazon.com> * fix doctest Signed-off-by: Lantao Jin <ltjin@amazon.com> * refactor Signed-off-by: Lantao Jin <ltjin@amazon.com> * Disable dedup expr Signed-off-by: Lantao Jin <ltjin@amazon.com> * fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * fix yaml test Signed-off-by: Lantao Jin <ltjin@amazon.com> * add more comments in code Signed-off-by: Lantao Jin <ltjin@amazon.com> * fix conflicts Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com>

* change q43 Signed-off-by: xinyual <xinyual@amazon.com> * change plan after merging latest code Signed-off-by: xinyual <xinyual@amazon.com> --------- Signed-off-by: xinyual <xinyual@amazon.com>

…oject#4867) Signed-off-by: Lantao Jin <ltjin@amazon.com>

…opensearch-project#4868) Signed-off-by: Songkan Tang <songkant@amazon.com>

…roject#4877) * Remove unnecessary filter for DataHistogram aggregation Signed-off-by: Lantao Jin <ltjin@amazon.com> * revert changes of interval selection in DataHistogram Signed-off-by: Lantao Jin <ltjin@amazon.com> * typo Signed-off-by: Lantao Jin <ltjin@amazon.com> * revert one it Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com>

…earch-project#4881) Signed-off-by: Heng Qian <qianheng@amazon.com>

…ct#4345)

…ype (opensearch-project#4885)

* Add review instructions for CodeRabbit Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Disable auto review Signed-off-by: Tomoyuki Morita <moritato@amazon.com> --------- Signed-off-by: Tomoyuki Morita <moritato@amazon.com>

…PL (opensearch-project#4841) * [BugFix] Fix Regex OOM when there are 10+ regex clauses Signed-off-by: Jialiang Liang <jiallian@amazon.com> * fix unit tests Signed-off-by: Jialiang Liang <jiallian@amazon.com> * fix tests Signed-off-by: Jialiang Liang <jiallian@amazon.com> * fix explain tests and corresponding commands Signed-off-by: Jialiang Liang <jiallian@amazon.com> * fix explain tests for testFilterPushDownExplain Signed-off-by: Jialiang Liang <jiallian@amazon.com> * peng - isolate the fix logic to its own visitor class Signed-off-by: Jialiang Liang <jiallian@amazon.com> * Directly apply Calcite CoreRules.FILTER_MERGE before VolcanoPlanner plan Co-authored-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Jialiang Liang <jiallian@amazon.com> * fix the UTs Signed-off-by: Jialiang Liang <jiallian@amazon.com> * fix the ITs after rebase Signed-off-by: Jialiang Liang <jiallian@amazon.com> * fix clickbench IT and more ITs Signed-off-by: Jialiang Liang <jiallian@amazon.com> * address comments from peng Signed-off-by: Jialiang Liang <jiallian@amazon.com> * add yaml test Signed-off-by: Jialiang Liang <jiallian@amazon.com> --------- Signed-off-by: Jialiang Liang <jiallian@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com>

Signed-off-by: Tomoyuki Morita <moritato@amazon.com>

) Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

* Support composite aggregation paginating in HAVING clause Signed-off-by: Lantao Jin <ltjin@amazon.com> * typo Signed-off-by: Lantao Jin <ltjin@amazon.com> * refactor Signed-off-by: Lantao Jin <ltjin@amazon.com> * refactor Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix doctest and IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * secruity it Signed-off-by: Lantao Jin <ltjin@amazon.com> * revert changes in OpenSearchIndexScan Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix compile error Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix v2 paginationIT Signed-off-by: Lantao Jin <ltjin@amazon.com> * optimize request total size in compoisite agg Signed-off-by: Lantao Jin <ltjin@amazon.com> * fix it Signed-off-by: Lantao Jin <ltjin@amazon.com> * Refactor Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com>

* Add basic transpiler impl Signed-off-by: Chen Dai <daichen@amazon.com> * Add builder Signed-off-by: Chen Dai <daichen@amazon.com> * Use lombok builder Signed-off-by: Chen Dai <daichen@amazon.com> * Modify unified query planner UT to extend new test base class Signed-off-by: Chen Dai <daichen@amazon.com> * Update doc with API design caveat Signed-off-by: Chen Dai <daichen@amazon.com> * Move opensearch spark sql dialect out of test folder Signed-off-by: Chen Dai <daichen@amazon.com> * Update doc and test assertion message Signed-off-by: Chen Dai <daichen@amazon.com> * Fix line separator and license header Signed-off-by: Chen Dai <daichen@amazon.com> --------- Signed-off-by: Chen Dai <daichen@amazon.com>

* Support split eval function Signed-off-by: Kai Huang <ahkcs@amazon.com> # Conflicts: # core/src/main/java/org/opensearch/sql/expression/function/BuiltinFunctionName.java # integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalciteArrayFunctionIT.java # ppl/src/main/antlr/OpenSearchPPLLexer.g4 # ppl/src/main/antlr/OpenSearchPPLParser.g4 # ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLArrayFunctionTest.java # ppl/src/test/java/org/opensearch/sql/ppl/utils/PPLQueryDataAnonymizerTest.java * doctest Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update test cases Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update to not use UDF Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com>

* Add workflow for SQL CLI integration tests Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Switch SQL checkout path for CLI Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Tweak paths and mkdir for remote Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Fix filename Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Remove redundant verification Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Don't pull after clone Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

asifabashar and others added 30 commits October 19, 2025 06:14

PPL tostring() implementation issue opensearch-project#4492

0ed23de

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

removed sql changes

5a5b778

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

doc changes

9d71a95

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

docs changes

be2c2e2

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

reverted string doc changes

fc763a4

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

removed extra word

a04eb14

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

added any type

590a8e6

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

doc formatting fixes

6938e2c

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

description for boolean example

314fccd

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

added format_time call from calcite , added duration_millis as splunk…

0ee17b9

… default duration is in seconds which will be used for duration format , added cast call for tostring with 1 argument Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

added doc update to specifically set 2nd argument as optional

6e24aa3

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

mentioned as value instead of number specifically

454cfc8

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

fixed wrong bullet point

b221e8c

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

issue opensearch-project#4514 tonumber function as part of roadmap op…

a0a89c4

…ensearch-project#4287 Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

added more unit tests

ea9ba0f

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

Update docs/user/ppl/functions/conversion.rst

370b9bc

Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com> Signed-off-by: Asif Bashar <abashar@apple.com> Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

fix per recommendation

f070684

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

updated recommended changes

a2accf5

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

updated recommended changes

3adda5d

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

updated recommended changes

17b1d86

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

updated recommended changes

eca7d13

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

updated recommended changes

e780c15

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

updated recommended changes

e3884b0

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

spotless apply

042d3e8

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

merge conflict fix

38ce420

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

merge conflict fix

58d8e1b

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

added recommended changes

34f8635

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

fixed doctest for conversion.rst and applied spotless

8279bf3

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

removed decimal point from hex

b5346a6

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

added doctest fixes

7cf867b

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

xinyual and others added 29 commits December 9, 2025 21:34

Fix the flaky CalcitePPLTcphIT (opensearch-project#4846)

dc98a90

Signed-off-by: Lantao Jin <ltjin@amazon.com>

Grouping key field type can only be overwritten when the ExprCoreType…

18d506a

… are different (opensearch-project#4850) Signed-off-by: Lantao Jin <ltjin@amazon.com>

fix clickbench query 43 (opensearch-project#4861)

f348e99

* change q43 Signed-off-by: xinyual <xinyual@amazon.com> * change plan after merging latest code Signed-off-by: xinyual <xinyual@amazon.com> --------- Signed-off-by: xinyual <xinyual@amazon.com>

Remove count aggregation for sort on aggregate measure (opensearch-pr…

16b0e9b

…oject#4867) Signed-off-by: Lantao Jin <ltjin@amazon.com>

Fix wrong parameter and return result logic for LogPatternAggFunction (…

b366994

…opensearch-project#4868) Signed-off-by: Songkan Tang <songkant@amazon.com>

Refactor alias type field by adding another project with alias (opens…

aa4bfa8

…earch-project#4881) Signed-off-by: Heng Qian <qianheng@amazon.com>

Implement one-batch lookahead for index enumerators (opensearch-proje…

8e24881

…ct#4345)

Add hashCode() and equals() to the implmentation classes of ExprJavaT…

b69d8de

…ype (opensearch-project#4885)

Support timeouts for Calcite queries (opensearch-project#4857)

a85aead

Adjust CodeRabbit review config (opensearch-project#4901)

7ace206

Signed-off-by: Tomoyuki Morita <moritato@amazon.com>

Remove access controller step in Calcite script (opensearch-project#4900

58025be

) Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

Error handling for dot-containing field names (opensearch-project#4907)

e9700d5

Migrate PPL Documentation from RST to Markdown (opensearch-project#4912)

3ffd8eb

merged main branch

b34e374

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

Merge branch 'main' into feature_tonumber

e5edbad

Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

asifabashar merged commit ecb63ea into feature_tonumber_2 Dec 10, 2025
31 of 32 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature tonumber : issue #4514 tonumber function as part of roadmap #4287 #3

Feature tonumber : issue #4514 tonumber function as part of roadmap #4287 #3
asifabashar merged 103 commits intofeature_tonumber_2from
feature_tonumber

asifabashar commented Dec 10, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

15 participants

Conversation

asifabashar commented Dec 10, 2025

Description

Related Issues

Check List

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

15 participants