Skip to content

Change group-by to accept cell paths#9020

Merged
fdncred merged 5 commits intonushell:mainfrom
rgwood:change-group-by-to-accept-cell-paths
May 17, 2023
Merged

Change group-by to accept cell paths#9020
fdncred merged 5 commits intonushell:mainfrom
rgwood:change-group-by-to-accept-cell-paths

Conversation

@rgwood
Copy link
Copy Markdown
Contributor

@rgwood rgwood commented Apr 27, 2023

Closes #9003.

This PR changes group-by so that its optional argument is interpreted as a cell path. In turn, this lets users use ? to ignore rows that are missing the column they wish to group on. For example:

> [{foo: 123}, {foo: 234}, {bar: 345}] | group-by foo
Error: nu::shell::column_not_found

  × Cannot find column
   ╭─[entry #3:1:1]
 1 │ [{foo: 123}, {foo: 234}, {bar: 345}] | group-by foo
   ·                          ─────┬────             ─┬─
   ·                               │                  ╰── cannot find column 'foo'
   ·                               ╰── value originates here
   ╰────

> [{foo: 123}, {foo: 234}, {bar: 345}] | group-by foo?
╭─────┬───────────────╮
│ 123 │ [table 1 row] │
│ 234 │ [table 1 row] │
╰─────┴───────────────╯

This removes the ability to pass group-by a closure or block (I wasn't able to figure out how to make the 2 features coexist), and so it is a breaking change. I think this is OK; I didn't even know group-by could accept a closure or block because there was no example for that functionality.

@rgwood rgwood added the notes:breaking-changes This PR implies a change affecting users and has to be noted in the release notes label Apr 27, 2023
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 27, 2023

Codecov Report

Merging #9020 (5ee938f) into main (39cdf56) will increase coverage by 0.03%.
The diff coverage is 92.63%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #9020      +/-   ##
==========================================
+ Coverage   68.97%   69.00%   +0.03%     
==========================================
  Files         641      641              
  Lines      102440   102497      +57     
==========================================
+ Hits        70656    70727      +71     
+ Misses      31784    31770      -14     
Impacted Files Coverage Δ
crates/nu-command/src/filters/group_by.rs 95.10% <92.19%> (+7.06%) ⬆️
crates/nu-command/src/filters/split_by.rs 91.66% <95.23%> (+0.32%) ⬆️
crates/nu-command/src/example_test.rs 100.00% <100.00%> (ø)

... and 3 files with indirect coverage changes

@fdncred
Copy link
Copy Markdown
Contributor

fdncred commented Apr 27, 2023

I think this is a cool PR, thanks!

I'm wondering if we can follow @Sygmei's suggestion in Discord and create a separate group-by block so we don't lose that powerful type of grouping? Here's an example of how to use it.

ls | group-by { get name | path parse | get extension } 
╭──────┬─────────────────╮
│ md   │ [table 3 rows]  │
│ lock │ [table 1 row]   │
│ toml │ [table 3 rows]  │
│      │ [table 12 rows] │
│ txt  │ [table 1 row]   │
│ sh   │ [table 4 rows]  │
│ cmd  │ [table 1 row]   │
│ nu   │ [table 4 rows]  │
│ rs   │ [table 1 row]   │
│ yml  │ [table 1 row]   │
│ json │ [table 1 row]   │
│ ps1  │ [table 1 row]   │
╰──────┴─────────────────╯

@Sygmei
Copy link
Copy Markdown
Contributor

Sygmei commented Apr 27, 2023

@rgwood I think you can make the two features co-exist by using the following SyntaxShape

            .optional(
                "grouper",
                SyntaxShape::OneOf(vec![
                    SyntaxShape::CellPath,
                    SyntaxShape::Block,
                    SyntaxShape::Closure(None),
                    SyntaxShape::Closure(Some(vec![SyntaxShape::Any])),
                ]),

I started working on this PR as well, I didn't know someone else was willing to take on this issue
What we need to do is probably merge your new code + old code
The old way of doing things (Grouper::ByBlock / Grouper::ByColumn) was kinda clunky imho, I like your way of doing things more, we just need to allow Blocks back and evaluate for each value (we can just copy paste the old way of doing things for blocks and keep your new system for CellPaths)

@rgwood
Copy link
Copy Markdown
Contributor Author

rgwood commented Apr 30, 2023

Good points, thanks. I might not have time/energy to implement that for a couple weeks.

@Sygmei would you like to take this issue+code and run with it? If not no worries, I'll get back to this eventually.

@Sygmei
Copy link
Copy Markdown
Contributor

Sygmei commented Apr 30, 2023

Sure, I'll give it a try !

@sholderbach sholderbach marked this pull request as draft May 5, 2023 09:20
@rgwood
Copy link
Copy Markdown
Contributor Author

rgwood commented May 12, 2023

@Sygmei Any luck? I may be able to pick this up again soon if you don't have time to work on it.

@Sygmei
Copy link
Copy Markdown
Contributor

Sygmei commented May 12, 2023

Go ahead ! In the end I had a pretty busy schedule as well :(

@rgwood rgwood marked this pull request as ready for review May 12, 2023 02:58
@rgwood
Copy link
Copy Markdown
Contributor Author

rgwood commented May 12, 2023

OK, this is ready to go! I've re-added the block/closure functionality and added an example.

@rgwood rgwood changed the title Change group-by to accept cell paths, reject closures Change group-by to accept cell paths May 12, 2023
Copy link
Copy Markdown
Contributor

@fdncred fdncred left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm glad you figured out a way to keep the block type and also have the optional type. It seems more powerful now.

@rgwood
Copy link
Copy Markdown
Contributor Author

rgwood commented May 12, 2023

It's getting a bit close to the next release; I think this one can wait until after the release.

@fdncred
Copy link
Copy Markdown
Contributor

fdncred commented May 17, 2023

let's go!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

notes:breaking-changes This PR implies a change affecting users and has to be noted in the release notes status:wait-until-after-nushell-release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

group-by should accept ? for optional path members

3 participants