Skip to content

Ensure that lazy frames converted via to-lazy are not converted back to eager frames later in the pipeline.#12525

Merged
fdncred merged 2 commits intonushell:mainfrom
ayax79:into_lazy_dont_convert
Apr 15, 2024
Merged

Ensure that lazy frames converted via to-lazy are not converted back to eager frames later in the pipeline.#12525
fdncred merged 2 commits intonushell:mainfrom
ayax79:into_lazy_dont_convert

Conversation

@ayax79
Copy link
Copy Markdown
Contributor

@ayax79 ayax79 commented Apr 15, 2024

Description

@maxim-uvarov discovered the following error:

> [[a b]; [6 2] [1 4] [4 1]] | polars into-lazy | polars sort-by a | polars unique --subset [a]
Error:   × Error using as series
   ╭─[entry #1:1:68]
 1 │ [[a b]; [6 2] [1 4] [4 1]] | polars into-lazy | polars sort-by a | polars unique --subset [a]
   ·                                                                    ──────┬──────
   ·                                                                          ╰── dataframe has more than one column
   ╰────

During investigation, I discovered the root cause was that the lazy frame was incorrectly converted back to a eager dataframe. In order to keep this from happening, I explicitly set that the dataframe did not come from an eager frame. This causes the conversion logic to not attempt to convert the dataframe later in the pipeline.

@ayax79 ayax79 changed the title "Ensure that lazyframe from_eager is false" Ensure that lazy frames converted via to-lazy are not converted back to eager frames later in the pipeline. Apr 15, 2024
@fdncred fdncred added the A:plugin-polars Work related to the polars dataframe implementation label Apr 15, 2024
@fdncred fdncred merged commit 5f818ea into nushell:main Apr 15, 2024
@fdncred
Copy link
Copy Markdown
Contributor

fdncred commented Apr 15, 2024

Thanks

@hustcer hustcer added this to the v0.93.0 milestone Apr 16, 2024
@ayax79 ayax79 deleted the into_lazy_dont_convert branch August 8, 2024 16:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A:plugin-polars Work related to the polars dataframe implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants