Allow spreading arguments to commands#11289
Conversation
|
A few things I'd like feedback on:
|
|
once finished with this pr, it'll be good to get some documentation because i keep confusing myself with what the spread operator is supposed to do, even though i've looked at each pr. maybe equally importantly is to understand what it's not supposed to do. for now, it seems like it's only used for external commands. is that right? regular rest argdef --wrapped mecho [...rest] { echo $rest}
❯ mecho -h a b d
╭───┬────╮
│ 0 │ -h │
│ 1 │ a │
│ 2 │ b │
│ 3 │ d │
╰───┴────╯rest arg with spread operatordef --wrapped mecho [...rest] { echo ...$rest}
❯ mecho -h a b d
╭───┬────╮
│ 0 │ -h │
│ 1 │ a │
│ 2 │ b │
│ 3 │ d │
╰───┴────╯rest arg with rest operator with external command❯ def --wrapped mecho [...rest] { ^echo ...$rest}
❯ mecho -h a b d
-h a b d |
Good idea, I'll get started on a PR for that right after this one (or in parallel).
No, it should also work for internal commands (although I still haven't gotten around to testing it on commands with named parameters). For example: |
|
Converting to draft because I just realized that if you do |
|
Oh, it looks like the reason |
|
I think it's OK to allow spreading for externals because their I wonder if we should unify One last note: I'd change the error message on the screenshot from |
|
OK, error looks better now! I saw the TODO comments when creating Spans in flatten.rs. I think it's OK as the 3 preceding dots should be guaranteed from the parser, and we'd get crashes when running tests thanks to the Looks good to me! |
|
Looks good to me :-) I think we can go and merge it |
# Description in #11289, spreading lists into command invocations was made possible and its implicit version was deprecated, but not everything was updated accordingly. # User-Facing Changes A commented part of the default config no longer throws a deprecation warning when uncommented # After Submitting After #11289, the mention of carapace in the documentation wasn’t updated. See nushell/nushell.github.io#1211
related to - nushell/nushell#11289 ## Description this should fix the deprecation warning that we get these days
related to - nushell/nushell#11289 wait for - nushell/nupm#57
related to - nushell/nushell#11289 wait for - nushell/nupm#57
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Spreading lists automatically when calling externals was deprecated in 0.89 (#11289), and this PR is to remove it in 0.91. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> The new error message looks like this: ``` > ^echo [1 2] Error: nu::shell::cannot_pass_list_to_external × Lists are not automatically spread when calling external commands ╭─[entry #13:1:8] 1 │ ^echo [1 2] · ──┬── · ╰── Spread operator (...) is necessary to spread lists ╰──── help: Either convert the list to a string or use the spread operator, like so: ...[1 2] ``` The old error message didn't say exactly where to put the `...` and seemed to confuse a lot of people, so hopefully this helps. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> There was one test to check that implicit spread was deprecated before, updated that to check that it's disallowed now. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> Finishes implementing nushell#10598, which asks for a spread operator in lists, in records, and when calling commands. # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR will allow spreading arguments to commands (both internal and external). It will also deprecate spreading arguments automatically when passing to external commands. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> - Users will be able to use `...` to spread arguments to custom/builtin commands that have rest parameters or allow unknown arguments, or to any external command - If a custom command doesn't have a rest parameter and it doesn't allow unknown arguments either, the spread operator will not be allowed - Passing lists to external commands without `...` will work for now but will cause a deprecation warning saying that it'll stop working in 0.91 (is 2 versions enough time?) Here's a function to help with demonstrating some behavior: ```nushell > def foo [ a, b, c?, d?, ...rest ] { [$a $b $c $d $rest] | to nuon } ``` You can pass a list of arguments to fill in the `rest` parameter using `...`: ```nushell > foo 1 2 3 4 ...[5 6] [1, 2, 3, 4, [5, 6]] ``` If you don't use `...`, the list `[5 6]` will be treated as a single argument: ```nushell > foo 1 2 3 4 [5 6] # Note the double [[]] [1, 2, 3, 4, [[5, 6]]] ``` You can omit optional parameters before the spread arguments: ```nushell > foo 1 2 3 ...[4 5] # d is omitted here [1, 2, 3, null, [4, 5]] ``` If you have multiple lists, you can spread them all: ```nushell > foo 1 2 3 ...[4 5] 6 7 ...[8] ...[] [1, 2, 3, null, [4, 5, 6, 7, 8]] ``` Here's the kind of error you get when you try to spread arguments to a command with no rest parameter:  And this is the warning you get when you pass a list to an external now (without `...`):  # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> Added tests to cover the following cases: - Spreading arguments to a command that doesn't have a rest parameter (unexpected spread argument error) - Spreading arguments to a command that doesn't have a rest parameter *but* there's also a missing positional argument (missing positional error) - Spreading arguments to a command that doesn't have a rest parameter but does allow unknown arguments, such as `exec` (allowed) - Spreading a list literal containing arguments of the wrong type (parse error) - Spreading a non-list value, both to internal and external commands - Having named arguments in the middle of rest arguments - `explain`ing a command call that spreads its arguments # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> # Examples Suppose you have multiple tables: ```nushell let people = [[id name age]; [0 alice 100] [1 bob 200] [2 eve 300]] let evil_twins = [[id name age]; [0 ecila 100] [-1 bob 200] [-2 eve 300]] ``` Maybe you often find yourself needing to merge multiple tables and want a utility to do that. You could write a function like this: ```nushell def merge_all [ ...tables ] { $tables | reduce { |it, acc| $acc | merge $it } } ``` Then you can use it like this: ```nushell > merge_all ...([$people $evil_twins] | each { |$it| $it | select name age }) ╭───┬───────┬─────╮ │ # │ name │ age │ ├───┼───────┼─────┤ │ 0 │ ecila │ 100 │ │ 1 │ bob │ 200 │ │ 2 │ eve │ 300 │ ╰───┴───────┴─────╯ ``` Except they had duplicate columns, so now you first want to suffix every column with a number to tell you which table the column came from. You can make a command for that: ```nushell def select_and_merge [ --cols: list<string>, ...tables ] { let renamed_tables = $tables | enumerate | each { |it| $it.item | select $cols | rename ...($cols | each { |col| $col + ($it.index | into string) }) }; merge_all ...$renamed_tables } ``` And call it like this: ```nushell > select_and_merge --cols [name age] $people $evil_twins ╭───┬───────┬──────┬───────┬──────╮ │ # │ name0 │ age0 │ name1 │ age1 │ ├───┼───────┼──────┼───────┼──────┤ │ 0 │ alice │ 100 │ ecila │ 100 │ │ 1 │ bob │ 200 │ bob │ 200 │ │ 2 │ eve │ 300 │ eve │ 300 │ ╰───┴───────┴──────┴───────┴──────╯ ``` --- Suppose someone's made a command to search for APT packages: ```nushell # The main command def search-pkgs [ --install # Whether to install any packages it finds log_level: int # Pretend it's a good idea to make this a required positional parameter exclude?: list<string> # Packages to exclude repositories?: list<string> # Which repositories to look in (searches in all if not given) ...pkgs # Package names to search for ] { { install: $install, log_level: $log_level, exclude: ($exclude | to nuon), repositories: ($repositories | to nuon), pkgs: ($pkgs | to nuon) } } ``` It has a lot of parameters to configure it, so you might make your own helper commands to wrap around it for specific cases. Here's one example: ```nushell # Only look for packages locally def search-pkgs-local [ --install # Whether to install any packages it finds log_level: int exclude?: list<string> # Packages to exclude ...pkgs # Package names to search for ] { # All required and optional positional parameters are given search-pkgs --install=$install $log_level [] ["<local URI or something>"] ...$pkgs } ``` And you can run it like this: ```nushell > search-pkgs-local --install=false 5 ...["python2.7" "vim"] ╭──────────────┬──────────────────────────────╮ │ install │ false │ │ log_level │ 5 │ │ exclude │ [] │ │ repositories │ ["<local URI or something>"] │ │ pkgs │ ["python2.7", vim] │ ╰──────────────┴──────────────────────────────╯ ``` One thing I realized when writing this was that if we decide to not allow passing optional arguments using the spread operator, then you can (mis?)use the spread operator to skip optional parameters. Here, I didn't want to give `exclude` explicitly, so I used a spread operator to pass the packages to install. Without it, I would've needed to do `search-pkgs-local --install=false 5 [] "python2.7" "vim"` (explicitly pass `[]` (or `null`, in the general case) to `exclude`). There are probably more idiomatic ways to do this, but I just thought it was something interesting. If you're a virologist of the [xkcd](https://xkcd.com/350/) kind, another helper command you might make is this: ```nushell # Install any packages it finds def live-dangerously [ ...pkgs ] { # One optional argument was given (exclude), while another was not (repositories) search-pkgs 0 [] ...$pkgs --install # Flags can go after spread arguments } ``` Running it: ```nushell > live-dangerously "git" "*vi*" # *vi* because I don't feel like typing out vim and neovim ╭──────────────┬─────────────╮ │ install │ true │ │ log_level │ 0 │ │ exclude │ [] │ │ repositories │ null │ │ pkgs │ [git, *vi*] │ ╰──────────────┴─────────────╯ ``` Here's an example that uses the spread operator more than once within the same command call: ```nushell let extras = [ chrome firefox python java git ] def search-pkgs-curated [ ...pkgs ] { (search-pkgs 1 [emacs] ["example.com", "foo.com"] vim # A must for everyone! ...($pkgs | filter { |p| not ($p | str contains "*") }) # Remove packages with globs python # Good tool to have ...$extras --install=false python3) # I forget, did I already put Python in extras? } ``` Running it: ```nushell > search-pkgs-curated "git" "*vi*" ╭──────────────┬───────────────────────────────────────────────────────────────────╮ │ install │ false │ │ log_level │ 1 │ │ exclude │ [emacs] │ │ repositories │ [example.com, foo.com] │ │ pkgs │ [vim, git, python, chrome, firefox, python, java, git, "python3"] │ ╰──────────────┴───────────────────────────────────────────────────────────────────╯ ```
# Description in nushell#11289, spreading lists into command invocations was made possible and its implicit version was deprecated, but not everything was updated accordingly. # User-Facing Changes A commented part of the default config no longer throws a deprecation warning when uncommented # After Submitting After nushell#11289, the mention of carapace in the documentation wasn’t updated. See nushell/nushell.github.io#1211
…l#11857) <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Spreading lists automatically when calling externals was deprecated in 0.89 (nushell#11289), and this PR is to remove it in 0.91. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> The new error message looks like this: ``` > ^echo [1 2] Error: nu::shell::cannot_pass_list_to_external × Lists are not automatically spread when calling external commands ╭─[entry nushell#13:1:8] 1 │ ^echo [1 2] · ──┬── · ╰── Spread operator (...) is necessary to spread lists ╰──── help: Either convert the list to a string or use the spread operator, like so: ...[1 2] ``` The old error message didn't say exactly where to put the `...` and seemed to confuse a lot of people, so hopefully this helps. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> There was one test to check that implicit spread was deprecated before, updated that to check that it's disallowed now. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
…l#11857) <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Spreading lists automatically when calling externals was deprecated in 0.89 (nushell#11289), and this PR is to remove it in 0.91. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> The new error message looks like this: ``` > ^echo [1 2] Error: nu::shell::cannot_pass_list_to_external × Lists are not automatically spread when calling external commands ╭─[entry nushell#13:1:8] 1 │ ^echo [1 2] · ──┬── · ╰── Spread operator (...) is necessary to spread lists ╰──── help: Either convert the list to a string or use the spread operator, like so: ...[1 2] ``` The old error message didn't say exactly where to put the `...` and seemed to confuse a lot of people, so hopefully this helps. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> There was one test to check that implicit spread was deprecated before, updated that to check that it's disallowed now. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
Finishes implementing #10598, which asks for a spread operator in lists, in records, and when calling commands.
Description
This PR will allow spreading arguments to commands (both internal and external). It will also deprecate spreading arguments automatically when passing to external commands.
User-Facing Changes
...to spread arguments to custom/builtin commands that have rest parameters or allow unknown arguments, or to any external command...will work for now but will cause a deprecation warning saying that it'll stop working in 0.91 (is 2 versions enough time?)Here's a function to help with demonstrating some behavior:
You can pass a list of arguments to fill in the
restparameter using...:If you don't use
..., the list[5 6]will be treated as a single argument:You can omit optional parameters before the spread arguments:
If you have multiple lists, you can spread them all:
Here's the kind of error you get when you try to spread arguments to a command with no rest parameter:

And this is the warning you get when you pass a list to an external now (without
...):Tests + Formatting
Added tests to cover the following cases:
exec(allowed)explaining a command call that spreads its argumentsAfter Submitting
Examples
Suppose you have multiple tables:
Maybe you often find yourself needing to merge multiple tables and want a utility to do that. You could write a function like this:
Then you can use it like this:
Except they had duplicate columns, so now you first want to suffix every column with a number to tell you which table the column came from. You can make a command for that:
And call it like this:
Suppose someone's made a command to search for APT packages:
It has a lot of parameters to configure it, so you might make your own helper commands to wrap around it for specific cases. Here's one example:
And you can run it like this:
One thing I realized when writing this was that if we decide to not allow passing optional arguments using the spread operator, then you can (mis?)use the spread operator to skip optional parameters. Here, I didn't want to give
excludeexplicitly, so I used a spread operator to pass the packages to install. Without it, I would've needed to dosearch-pkgs-local --install=false 5 [] "python2.7" "vim"(explicitly pass[](ornull, in the general case) toexclude). There are probably more idiomatic ways to do this, but I just thought it was something interesting.If you're a virologist of the xkcd kind, another helper command you might make is this:
Running it:
Here's an example that uses the spread operator more than once within the same command call:
Running it: