Fix the bug of explicit makeNullLiteral for UDT fields by songkant-aws · Pull Request #4475 · opensearch-project/sql

songkant-aws · 2025-10-09T09:01:36Z

Description

Fix the bug that Calcite will error out when CalciteRelBuilder explicitly calling makeNullLiteral for UDT type fields.

UDT type cast is handled by UDF. It's totally fine for CAST function, regardless whether the field value is NULL or non NULL. But for some cases in building RelNode, it has the problem of explicitly casting NULL to UDT type by RexBuilder.makeNullLiteral method. UDT cast is a RexCall of calling UDF, but makeNullLiteral internally expects the returned node to be RexLiteral of NULL. Thus, Calcite throws a hard error due to Java class casting failure.

This PR fixes this issue by adding a simple check and calling super method in case the RexNode to be cast is literally NULL.

Related Issues

Resolves #4383

Check List

New functionality includes testing.
New functionality has been documented.
New functionality has javadoc added.
New functionality has a user manual doc added.
New PPL command checklist all confirmed.
API changes companion pull request created.
Commits are signed per the DCO using --signoff or -s.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Songkan Tang <songkant@amazon.com>

songkant-aws · 2025-10-09T09:02:39Z

cc @ahkcs @yuancu Please let me know if this simple fix can cover the issue.

ahkcs · 2025-10-09T17:32:09Z

Thanks for the fix! I think for #4383, it's more like an issue for long-term goals for multisearch command(can be applied to append command as well), so this PR might only partially resolve the issue

LantaoJin · 2025-10-10T02:02:20Z

integ-test/src/yamlRestTest/resources/rest-api-spec/test/issues/4383.yml

+        Content-Type: 'application/json'
+      ppl:
+        body:
+          query: source=log-test1 | append [ source=log-test2 ]


will source=log-test1 | append [ source=log-test2 ] | fields timestamp return - match: { datarows: [["2025-09-04 16:15:00"], [null]] }?

Synced offline. It will append unique name in case of type conflicts and select fields are also successful

LantaoJin · 2025-10-10T02:03:35Z

Thanks for the fix! I think for #4383, it's more like an issue for long-term goals for multisearch command(can be applied to append command as well), so this PR might only partially resolve the issue

Is multisearch available? can you test that after this merged.

yuancu

LGTM

songkant-aws · 2025-10-10T03:21:59Z

@ahkcs Makes sense. It's a short-term bug fix to not fail the query. 4383 wants to resolve another issue. I can create another report bug issue to avoid confusion

songkant-aws · 2025-10-10T05:03:39Z

@LantaoJin Tried the multisearch command. It also fixes the same issue in multisearch command.

Signed-off-by: Songkan Tang <songkant@amazon.com> (cherry picked from commit 7d6357d) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…oject#4475) Signed-off-by: Songkan Tang <songkant@amazon.com>

(cherry picked from commit 7d6357d) Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…oject#4475) Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Simeon Widdis <sawiddis@amazon.com> Co-authored-by: Manasvini B S <manasvis@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Louis Chu <clingzhi@amazon.com> Co-authored-by: Chen Dai <daichen@amazon.com> Co-authored-by: Mebsina <cnoramut@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: Alexey Temnikov <alexey.temnikov@improving.com> Co-authored-by: Riley Jerger <214163063+RileyJergerAmazon@users.noreply.github.com> Co-authored-by: Tomoyuki MORITA <moritato@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Xinyuan Lu <xinyual@amazon.com> Co-authored-by: Jialiang Liang <jiallian@amazon.com> Co-authored-by: Peter Zhu <zhujiaxi@amazon.com> Co-authored-by: Vinay Krishna Pudyodu <vinkrish.neo@gmail.com> Co-authored-by: expani <anijainc@amazon.com> Co-authored-by: expani1729 <110471048+expani@users.noreply.github.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: Marc Handalian <marc.handalian@gmail.com> Co-authored-by: Marc Handalian <handalm@amazon.com> Fix join type ambiguous issue when specify the join type with sql-like join criteria (opensearch-project#4474) Fix issue 4441 (opensearch-project#4449) Fix missing keywordsCanBeId (opensearch-project#4491) Fix the bug of explicit makeNullLiteral for UDT fields (opensearch-project#4475) Fix mapping after aggregation push down (opensearch-project#4500) Fix percentile bug (opensearch-project#4539) Fix JsonExtractAllFunctionIT failure (opensearch-project#4556) Fix sort push down into agg after project already pushed (opensearch-project#4546) Fix push down failure for min/max on derived field (opensearch-project#4572) Fix compile issue in main (opensearch-project#4608) Fix filter parsing failure on date fields with non-default format (opensearch-project#4616) Fix bin nested fields issue (opensearch-project#4606) Fix: Support Alias Fields in MIN, MAX, FIRST, LAST, and TAKE Aggregations (opensearch-project#4621) fix rename issue (opensearch-project#4670) Fixes for `Multisearch` and `Append` command (opensearch-project#4512) Fix asc/desc keyword behavior for sort command (opensearch-project#4651) Fix] Fix unexpected shift of extraction for `rex` with nested capture groups in named groups (opensearch-project#4641) Fix CVE-2025-48924 (opensearch-project#4665) Fix sub-fields accessing of generated structs (opensearch-project#4683) Fix] Incorrect Field Index Mapping in AVG to SUM/COUNT Conversion (opensearch-project#15)

Fix the bug of explicit makeNullLiteral for UDT fields

41ee012

Signed-off-by: Songkan Tang <songkant@amazon.com>

songkant-aws marked this pull request as ready for review October 9, 2025 09:01

ahkcs approved these changes Oct 9, 2025

View reviewed changes

LantaoJin approved these changes Oct 10, 2025

View reviewed changes

LantaoJin added the bug Something isn't working label Oct 10, 2025

yuancu approved these changes Oct 10, 2025

View reviewed changes

yuancu merged commit 7d6357d into opensearch-project:main Oct 10, 2025
34 of 38 checks passed

songkant-aws mentioned this pull request Oct 10, 2025

[BUG] Explicit makeNullLiteral fails the schema merge for append and multisearch commands #4493

Closed

LantaoJin added the backport 2.19-dev label Oct 10, 2025

opensearch-trigger-bot bot mentioned this pull request Oct 10, 2025

[Backport 2.19-dev] Fix the bug of explicit makeNullLiteral for UDT fields #4494

Merged

songkant-aws added a commit to songkant-aws/sql that referenced this pull request Oct 10, 2025

Fix the bug of explicit makeNullLiteral for UDT fields (opensearch-pr…

17b5514

…oject#4475) Signed-off-by: Songkan Tang <songkant@amazon.com>

songkant-aws deleted the udt-null-literal-fix branch October 10, 2025 05:48

asifabashar pushed a commit to asifabashar/sql that referenced this pull request Oct 16, 2025

Fix the bug of explicit makeNullLiteral for UDT fields (opensearch-pr…

3a46064

…oject#4475) Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Asif Bashar <asif.bashar@gmail.com>

qianheng-aws mentioned this pull request Mar 18, 2026

Add songkant-aws as maintainer #5244

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix the bug of explicit makeNullLiteral for UDT fields#4475

Fix the bug of explicit makeNullLiteral for UDT fields#4475
yuancu merged 1 commit intoopensearch-project:mainfrom
songkant-aws:udt-null-literal-fix

songkant-aws commented Oct 9, 2025 •

edited

Loading

Uh oh!

songkant-aws commented Oct 9, 2025

Uh oh!

ahkcs commented Oct 9, 2025

Uh oh!

LantaoJin Oct 10, 2025

Uh oh!

songkant-aws Oct 10, 2025

Uh oh!

LantaoJin commented Oct 10, 2025

Uh oh!

yuancu left a comment

Uh oh!

songkant-aws commented Oct 10, 2025 •

edited

Loading

Uh oh!

Uh oh!

songkant-aws commented Oct 10, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

songkant-aws commented Oct 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

songkant-aws commented Oct 9, 2025

Uh oh!

ahkcs commented Oct 9, 2025

Uh oh!

LantaoJin Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

songkant-aws Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

LantaoJin commented Oct 10, 2025

Uh oh!

yuancu left a comment

Choose a reason for hiding this comment

Uh oh!

songkant-aws commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

songkant-aws commented Oct 10, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

songkant-aws commented Oct 9, 2025 •

edited

Loading

songkant-aws commented Oct 10, 2025 •

edited

Loading