Skip to content

[GLUTEN-11088][VL] Adding SQLQueryTestSuite for Spark-4.0#11136

Merged
rui-mo merged 5 commits intoapache:mainfrom
zhouyuan:wip_spark40_sqlquerytest
Dec 22, 2025
Merged

[GLUTEN-11088][VL] Adding SQLQueryTestSuite for Spark-4.0#11136
rui-mo merged 5 commits intoapache:mainfrom
zhouyuan:wip_spark40_sqlquerytest

Conversation

@zhouyuan
Copy link
Copy Markdown
Member

@zhouyuan zhouyuan commented Nov 20, 2025 •
edited
Loading

What changes are proposed in this pull request?

There's a refactor on SQLQueryTestSuite in Spark-4.0, which pulls out some traits and added more sql tests.
This patch ports the SQLQueryTestSuite for Gluten.

  // There is a limitation in Spark-4.0 on ANSI gap and exception message gap.
  // we disabled these sql tests temporarily.
  // limit.sql, postgreSQL/limit.sql, subquery/in-subquery/in-limit.sql,
  // subquery/in-subquery/in-null-semantics.sql
  // try_arithmetic.sql, try_element_at.sql, typeCoercion/native/stringCastAndExpressions.sql,
  // window.sql

How was this patch tested?

Pass GHA

Related issue: #11088

@github-actions github-actions bot added the CORE works for Gluten Core label Nov 20, 2025
@github-actions
Copy link
Copy Markdown

github-actions bot commented Nov 20, 2025

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions
Copy link
Copy Markdown

github-actions bot commented Nov 21, 2025

Run Gluten Clickhouse CI on x86

@baibaichen baibaichen mentioned this pull request Nov 24, 2025
Open
@zhouyuan zhouyuan force-pushed the wip_spark40_sqlquerytest branch from 9769473 to 028e82f Compare November 25, 2025 17:35
@github-actions
Copy link
Copy Markdown

github-actions bot commented Nov 25, 2025

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions
Copy link
Copy Markdown

github-actions bot commented Nov 25, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan force-pushed the wip_spark40_sqlquerytest branch from 3b52acd to 59a48ea Compare November 26, 2025 17:31
@github-actions
Copy link
Copy Markdown

github-actions bot commented Nov 26, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan force-pushed the wip_spark40_sqlquerytest branch 4 times, most recently from b0b21c7 to 6946e30 Compare November 28, 2025 18:34
@github-actions
Copy link
Copy Markdown

github-actions bot commented Nov 28, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan force-pushed the wip_spark40_sqlquerytest branch from 6946e30 to 7c41db0 Compare November 28, 2025 22:22
@github-actions
Copy link
Copy Markdown

github-actions bot commented Nov 28, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan force-pushed the wip_spark40_sqlquerytest branch from 7c41db0 to f1e15f1 Compare November 28, 2025 23:03
@github-actions
Copy link
Copy Markdown

github-actions bot commented Nov 28, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan force-pushed the wip_spark40_sqlquerytest branch from f1e15f1 to 6639d05 Compare December 2, 2025 08:53
@github-actions
Copy link
Copy Markdown

github-actions bot commented Dec 2, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan force-pushed the wip_spark40_sqlquerytest branch from 6639d05 to 7932550 Compare December 2, 2025 09:21
@github-actions
Copy link
Copy Markdown

github-actions bot commented Dec 2, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan requested a review from marin-ma December 3, 2025 02:56
@zhouyuan zhouyuan force-pushed the wip_spark40_sqlquerytest branch from 7932550 to 3a3b26c Compare December 3, 2025 13:30
@github-actions
Copy link
Copy Markdown

github-actions bot commented Dec 3, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan force-pushed the wip_spark40_sqlquerytest branch from 3a3b26c to ffd73bf Compare December 3, 2025 14:56
@github-actions
Copy link
Copy Markdown

github-actions bot commented Dec 3, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan force-pushed the wip_spark40_sqlquerytest branch from ffd73bf to fa9cd57 Compare December 3, 2025 16:36
@github-actions
Copy link
Copy Markdown

github-actions bot commented Dec 3, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan force-pushed the wip_spark40_sqlquerytest branch from fa9cd57 to 24faa38 Compare December 4, 2025 06:32
@github-actions
Copy link
Copy Markdown

github-actions bot commented Dec 4, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan force-pushed the wip_spark40_sqlquerytest branch from 24faa38 to 290b4e9 Compare December 4, 2025 15:32
@github-actions
Copy link
Copy Markdown

github-actions bot commented Dec 4, 2025

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions
Copy link
Copy Markdown

github-actions bot commented Dec 5, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan force-pushed the wip_spark40_sqlquerytest branch from 737fb26 to e3677a1 Compare December 8, 2025 08:34
@github-actions
Copy link
Copy Markdown

github-actions bot commented Dec 8, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan force-pushed the wip_spark40_sqlquerytest branch from e3677a1 to 2c6b085 Compare December 10, 2025 05:10
@github-actions
Copy link
Copy Markdown

github-actions bot commented Dec 10, 2025

Run Gluten Clickhouse CI on x86

@zhouyuan
[VL] Adding SQLQueryTestSuite for Spark-4.0
87d3a0f 
Signed-off-by: Yuan <yuanzhou@apache.org>
@zhouyuan
fix sql file path
5136a2f 
Signed-off-by: Yuan <yuanzhou@apache.org>
@zhouyuan
update sql input/results
6dc4054 
Signed-off-by: Yuan <yuanzhou@apache.org>
@zhouyuan
apply fix
739a943 
Signed-off-by: Yuan <yuanzhou@apache.org>
@zhouyuan
ignore failed sql due to ansi and exception gap, add more sql tests
e669473 
Signed-off-by: Yuan <yuanzhou@apache.org>
@zhouyuan zhouyuan force-pushed the wip_spark40_sqlquerytest branch from 2c6b085 to e669473 Compare December 11, 2025 11:46
@github-actions
Copy link
Copy Markdown

github-actions bot commented Dec 11, 2025

Run Gluten Clickhouse CI on x86

jinchengchenghh
jinchengchenghh reviewed Dec 22, 2025
View reviewed changes
gluten-ut/spark40/src/test/resources/backends-velox/sql-tests/inputs/binary.sql
@@ -0,0 +1,8 @@
--SET spark.sql.binaryOutputStyle=UTF-8

SELECT X'';
Copy link
Copy Markdown
Contributor

@jinchengchenghh jinchengchenghh Dec 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we not copy so many files from Spark, can we use spark.test.home?

@zhouyuan
Copy link
Copy Markdown
Member Author

zhouyuan commented Dec 22, 2025 via email

jinchengchenghh
jinchengchenghh approved these changes Dec 22, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@jinchengchenghh jinchengchenghh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@rui-mo rui-mo merged commit 232b3ac into apache:main Dec 22, 2025
106 of 107 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rui-mo rui-mo rui-mo approved these changes

@jinchengchenghh jinchengchenghh jinchengchenghh approved these changes

@marin-ma marin-ma Awaiting requested review from marin-ma

Assignees

No one assigned

Labels

CORE works for Gluten Core

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@zhouyuan @rui-mo @jinchengchenghh