-
Notifications
You must be signed in to change notification settings - Fork 270
Closed
Labels
bugSomething isn't workingSomething isn't working
Milestone
Description
Describe the bug
When we fix #1248 we find a new bug that causes the following Spark SQL test to fail:
reverse preceding/following range between with aggregation
Steps to reproduce
test("reverse preceding/following range between with aggregation") {
val df = Seq(1, 2, 4, 3, 2, 1).toDF("value")
val window = Window.orderBy($"value".desc)
withSQLConf(CometConf.COMET_EXPLAIN_NATIVE_ENABLED.key -> "true") {
val df2 = df.select(
$"value",
sum($"value").over(window.rangeBetween(Window.unboundedPreceding, 1)),
sum($"value").over(window.rangeBetween(1, Window.unboundedFollowing)))
checkSparkAnswer(df2)
}
native plan:
BoundedWindowAggExec: wdw=[sum: Ok(Field { name: "sum", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }), frame: WindowFrame { units: Range, start_bound: Preceding(Int64(NULL)), end_bound: Following(Int64(1)), is_causal: false }, sum: Ok(Field { name: "sum", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }), frame: WindowFrame { units: Range, start_bound: Preceding(Int64(1)), end_bound: Following(Int64(NULL)), is_causal: false }], mode=[Sorted]
SortExec: expr=[col_0@0 DESC NULLS LAST], preserve_partitioning=[false]
CopyExec [UnpackOrDeepCopy]
ScanExec: source=[ShuffleQueryStage (unknown), Statistics(sizeInBytes=96.0 B, rowCount=6)], schema=[col_0: Int32]
Expected behavior
No response
Additional context
No response
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working