Skip to content

Window functions create unwanted projection #11982

@timsaucer

Description

@timsaucer

Describe the bug

When using a window function, you get an additional projection that is not expected.

To Reproduce

    let schema = Schema::new(vec![
        Field::new("a", DataType::Int32, true),
    ]);

    let batch = RecordBatch::try_new(
        Arc::new(schema.clone()),
        vec![
            Arc::new(Int32Array::from(vec![1, 2, 3, 4, 5])),
        ],
    )?;

    let ctx = SessionContext::new();

    let provider = MemTable::try_new(Arc::new(schema), vec![vec![batch]])?;
    ctx.register_table("t", Arc::new(provider))?;

    let df = ctx.table("t").await?;

    let func = Expr::WindowFunction(WindowFunction::new(
        WindowFunctionDefinition::BuiltInWindowFunction(BuiltInWindowFunction::RowNumber),
        vec![],
    )).alias("row_num");

    df.clone().select(vec![col("a"), func.clone()])?.show().await?;

    df.with_column("r", func)?.show().await?;

Expected behavior

When using the select() operation you get the desired result:

+---+---------+
| a | row_num |
+---+---------+
| 1 | 1       |
| 2 | 2       |
| 3 | 3       |
| 4 | 4       |
| 5 | 5       |
+---+---------+

But when using with_column() you get this additional projection:

+---+-----------------------------------------------------------------------+---+
| a | ROW_NUMBER() ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING | r |
+---+-----------------------------------------------------------------------+---+
| 1 | 1                                                                     | 1 |
| 2 | 2                                                                     | 2 |
| 3 | 3                                                                     | 3 |
| 4 | 4                                                                     | 4 |
| 5 | 5                                                                     | 5 |
+---+-----------------------------------------------------------------------+---+

Additional context

No response

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions