Skip to content

[GLUTEN-7631][VL] Fall back lead/lag if input is foldable#8038

Merged
philo-he merged 1 commit intoapache:mainfrom
zml1206:GLUTEN-7631
Nov 27, 2024
Merged

[GLUTEN-7631][VL] Fall back lead/lag if input is foldable#8038
philo-he merged 1 commit intoapache:mainfrom
zml1206:GLUTEN-7631

Conversation

@zml1206
Copy link
Copy Markdown
Contributor

@zml1206 zml1206 commented Nov 25, 2024

What changes were proposed in this pull request?

Velox does not support lead/lag when its input is foldable.

(Fixes: #7631)

How was this patch tested?

UT

@github-actions github-actions bot added CORE works for Gluten Core VELOX labels Nov 25, 2024
@github-actions
Copy link
Copy Markdown

#7631

@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@zml1206 zml1206 requested a review from zhztheplayer November 26, 2024 01:03
@philo-he
Copy link
Copy Markdown
Member

@zml1206, thanks for your pr! What's Velox's current behavior for this case? Will Velox interrupt with some error?

@zml1206
Copy link
Copy Markdown
Contributor Author

zml1206 commented Nov 26, 2024

@zml1206, thanks for your pr! What's Velox's current behavior for this case? Will Velox interrupt with some error?

The error reported by velox is the same as the error reported by gluten in the issue, "bad optional access".
image

image

@zml1206
Copy link
Copy Markdown
Contributor Author

zml1206 commented Nov 26, 2024

Copy link
Copy Markdown
Member

@philo-he philo-he left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Assume the foldable input is rarely used. So it has low priority. It's ok to firstly let Gluten fall back for that.

@philo-he philo-he changed the title [GLUTEN-7631][VL] Fallback lead/lag if input is foldable [GLUTEN-7631][VL] Fall back lead/lag if input is foldable Nov 27, 2024
@philo-he philo-he merged commit bfafa57 into apache:main Nov 27, 2024
@zml1206
Copy link
Copy Markdown
Contributor Author

zml1206 commented Nov 27, 2024

Looks good to me! Assume the foldable input is rarely used. So it has low priority. It's ok to firstly let Gluten fall back for that.

Thank you @philo-he
Yes, although vanilla Spark supports foldable input of lag/lead, there is almost no usage scenario in this case, and the values ​​obtained are all null or default.

@zml1206 zml1206 deleted the GLUTEN-7631 branch December 9, 2025 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CORE works for Gluten Core VELOX

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[VL] Bad optional access with lag function in Window

2 participants