Skip to content

tweak polars join for better cross joins#14586

Merged
fdncred merged 3 commits intonushell:mainfrom
fdncred:tweak_polars_cross_join
Dec 15, 2024
Merged

tweak polars join for better cross joins#14586
fdncred merged 3 commits intonushell:mainfrom
fdncred:tweak_polars_cross_join

Conversation

@fdncred
Copy link
Copy Markdown
Contributor

@fdncred fdncred commented Dec 14, 2024

Description

closes #14585

This PR tries to make polars join --cross work better. Example taken from https://docs.pola.rs/user-guide/transformations/joins/#cartesian-product

Before

 let tokens = [[monopoly_token]; [hat] [shoe] [boat]] | polars into-df
 let players = [[name, cash]; [Alice, 78] [Bob, 135]] | polars into-df
 $players | polars into-lazy | polars select (polars col name) | polars join --cross $tokens | polars collect
Error: nu::parser::missing_positional

  × Missing required positional argument.
   ╭─[entry #3:1:92]
 1  $players | polars into-lazy | polars select (polars col name) | polars join --cross $tokens
   ╰────
  help: Usage: polars join {flags} <other> <left_on> <right_on> . Use `--help` for more information.

After

 let players = [[name, cash]; [Alice, 78] [Bob, 135]] | polars into-df
 let tokens = [[monopoly_token]; [hat] [shoe] [boat]] | polars into-df
 $players | polars into-lazy | polars select (polars col name) | polars join --cross $tokens | polars collect
╭─#─┬─name──┬─monopoly_token─╮
 0  Alice  hat            
 1  Alice  shoe           
 2  Alice  boat           
 3  Bob    hat            
 4  Bob    shoe           
 5  Bob    boat           
╰─#─┴─name──┴─monopoly_token─╯

Other examples

 1..3 | polars into-df | polars join --cross (4..6 | polars into-df)
╭─#─┬─0─┬─0_x─╮
 0  1    4 
 1  1    5 
 2  1    6 
 3  2    4 
 4  2    5 
 5  2    6 
 6  3    4 
 7  3    5 
 8  3    6 
╰─#─┴─0─┴─0_x─╯
 1..3 | each {|x| {x: $x}} | polars into-df | polars join --cross (4..6 | each {|y| {y: $y}} | polars into-df) x y
╭─#─┬─x─┬─y─╮
 0  1  4 
 1  1  5 
 2  1  6 
 3  2  4 
 4  2  5 
 5  2  6 
 6  3  4 
 7  3  5 
 8  3  6 
╰─#─┴─x─┴─y─╯

/cc @ayax79

User-Facing Changes

Tests + Formatting

After Submitting

@ayax79
Copy link
Copy Markdown
Contributor

ayax79 commented Dec 14, 2024

@fdncred,

Looks good, could you add another example for it. It also looks like one of the tests is broken.

@fdncred fdncred merged commit baf86df into nushell:main Dec 15, 2024
@fdncred fdncred deleted the tweak_polars_cross_join branch December 15, 2024 03:58
@github-actions github-actions bot added this to the v0.101.0 milestone Dec 15, 2024
@sholderbach sholderbach added the A:plugin-polars Work related to the polars dataframe implementation label Dec 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A:plugin-polars Work related to the polars dataframe implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Polars Cross Join does not work with required args.

3 participants