Dev by hardfau1t · Pull Request #1 · hardfau1t/nushell

hardfau1t · 2023-12-14T16:53:01Z

Description

User-Facing Changes

Tests + Formatting

After Submitting

or any directory that user has access via group

…r_groups` Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>

This fixes an issue brought up by nihilander in [Discord](https://discord.com/channels/601130461678272522/614593951969574961/1201594105986285649). # Description  Nushell panics when the spread operator is used like this (the `...$rest` shouldn't actually be parsed as a spread operator at all): ```nu $ def foo [...rest: string] {...$rest} $ foo bar baz thread 'main' panicked at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/nu-protocol-0.89.0/src/signature.rs:650:9: Internal error: can't run a predeclaration without a body stack backtrace: 0: rust_begin_unwind 1: core::panicking::panic_fmt 2: <nu_protocol::signature::Predeclaration as nu_protocol::engine::command::Command>::run 3: nu_engine::eval::eval_call 4: nu_engine::eval::eval_expression_with_input 5: nu_engine::eval::eval_element_with_input 6: nu_engine::eval::eval_block 7: nu_cli::util::eval_source 8: nu_cli::repl::evaluate_repl 9: nu::run::run_repl 10: nu::main note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. ``` The problem was that whenever the parser saw something like `{...$`, `{...(`, or `{...[`, it would treat that as a record with a spread expression, ignoring the syntax shape of the block it was parsing. This should now be fixed, and the snippet above instead gives the following error: ```nu Error: nu::shell::external_command × External command failed ╭─[entry #1:1:1] 1 │ def foo [...rest] {...$rest} · ────┬─── · ╰── executable was not found ╰──── help: No such file or directory (os error 2) ``` # User-Facing Changes  Stuff like `do { ...$rest }` will now try to run a command `...$rest` rather than complaining that variable `$rest` doesn't exist. # Tests + Formatting  # After Submitting  Sorry about the issue, I am not touching the parser again for a long time :)

# Description  Fixes nushell#11711 Previously, syntax `def a [] (echo 4)` was allowed to parse and then failed with panic duting eval. Current error: ``` Error: nu::parser::parse_mismatch × Parse mismatch during operation. ╭─[entry #1:1:1] 1 │ def a [] (echo 4) · ────┬─── · ╰── expected definition body closure { ... } ╰──── ``` # User-Facing Changes  # Tests + Formatting  # After Submitting

…r ints (nushell#11724) # Description This PR changes `into int` and `into filesize` so that they allow thousands separators. ### Before ```nushell ❯ '1,000' | into filesize Error: nu::shell::cant_convert × Can't convert to int. ╭─[entry #1:1:1] 1 │ '1,000' | into filesize · ───┬─── · ╰── can't convert string to int ╰──── ❯ '1,000' | into int Error: nu:🐚:cant_convert × Can't convert to int. ╭─[entry nushell#2:1:1] 1 │ '1,000' | into int · ────┬─── · ╰── can't convert string to int ╰──── help: string "1,000" does not represent a valid integer ``` ### After ```nushell ❯ '1,000' | into filesize 1.0 KB ❯ '1,000' | into int 1000 ``` This works by getting the system locale and from that, determining what the thousands separator is. So, hopefully, this will work across locales. # User-Facing Changes  # Tests + Formatting  # After Submitting

# Description Close: nushell#9673 Close: nushell#8277 Close: nushell#10944 This pr introduces the following syntax: 1. `e>|`, pipe stderr to next command. Example: `$env.FOO=bar nu --testbin echo_env_stderr FOO e>| str length` 2. `o+e>|` and `e+o>|`, pipe both stdout and stderr to next command, example: `$env.FOO=bar nu --testbin echo_env_mixed out-err FOO FOO e+o>| str length` Note: it only works for external commands. ~There is no different for internal commands, that is, the following three commands do the same things:~ Edit: it raises errors if we want to pipes for internal commands ``` ❯ ls e>| str length Error: × `e>|` only works with external streams ╭─[entry #1:1:1] 1 │ ls e>| str length · ─┬─ · ╰── `e>|` only works on external streams ╰──── ❯ ls e+o>| str length Error: × `o+e>|` only works with external streams ╭─[entry nushell#2:1:1] 1 │ ls e+o>| str length · ──┬── · ╰── `o+e>|` only works on external streams ╰──── ``` This can help us to avoid some strange issues like the following: `$env.FOO=bar (nu --testbin echo_env_stderr FOO) e>| str length` Which is hard to understand and hard to explain to users. # User-Facing Changes Nan # Tests + Formatting To be done # After Submitting Maybe update documentation about these syntax.

# Description Resolves nushell#11274. ``` ~/CodingProjects/nushell> let day = 2; echo 0..<$day ╭───┬───╮ │ 0 │ 0 │ │ 1 │ 1 │ ╰───┴───╯ ~/CodingProjects/nushell> let kb = "jan"; echo 0..$kb Error: nu::shell::type_mismatch × Type mismatch during operation. ╭─[entry #1:1:22] 1 │ let kb = "jan"; echo 0..$kb · ┬─┬─┬─ · │ │ ╰── string · │ ╰── type mismatch for operator · ╰── int ╰──── ``` # Tests + Formatting Relevant test added 🆙 --------- Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>

# Description Resolves nushell#11756. Resolves nushell#12346. As per description, shell no longer hangs: ``` ~/CodingProjects/nushell> [1 2 3] | select (-2) Error: nu::shell::cant_convert × Can't convert to cell path. ╭─[entry #1:1:18] 1 │ [1 2 3] | select (-2) · ──┬─ · ╰── can't convert negative number to cell path ╰──── ```  # User-Facing Changes  # Tests + Formatting  Added relevant test 🚀 # After Submitting  Possibly support `get` `get`ting negative numbers, as per nushell#12346 discussion. Alternatively, we can consider adding a cellpath for negative indexing?

@maxim-uvarov

# Description From @maxim-uvarov's [post](https://discord.com/channels/601130461678272522/1227612017171501136/1228656319704203375). When calling `to-lazy` back to back in a pipeline, an error should not occur: ``` > [[a b]; [6 2] [1 4] [4 1]] | polars into-lazy | polars into-lazy Error: nu::shell::cant_convert × Can't convert to NuDataFrame. ╭─[entry #1:1:30] 1 │ [[a b]; [6 2] [1 4] [4 1]] | polars into-lazy | polars into-lazy · ────────┬─────── · ╰── can't convert NuLazyFrameCustomValue to NuDataFrame ╰──── ``` This pull request ensures that custom value's of NuLazyFrameCustomValue are properly converted when passed in. Co-authored-by: Jack Wright <jack.wright@disqo.com>

# Description Fixes nushell#12520 # User-Facing Changes Breaking change: Any operation parsing input with `PWD` to set the environment will now fail with `ShellError::AutomaticEnvVarSetManually` Furthermore transactions containing the special env-vars will be rejected before executing any modifications. Prevoiusly this was changing valid variables before while leaving valid variables after the violation untouched. ## `PWD` handling. Now failing ``` {PWD: "/trolling"} | load-env ``` already failing ``` load-env {PWD: "/trolling"} ``` ## Error management ``` > load-env {MY_VAR1: foo, PWD: "/trolling", MY_VAR2: bar} Error: nu::shell::automatic_env_var_set_manually × PWD cannot be set manually. ╭─[entry #1:1:2] 1 │ load-env {MY_VAR1: foo, PWD: "/trolling", MY_VAR2: bar} · ────┬─── · ╰── cannot set 'PWD' manually ╰──── help: The environment variable 'PWD' is set automatically by Nushell and cannot be set manually. ``` ### Before: ``` > $env.MY_VAR1 foo > $env.MY_VAR2 Error: nu:🐚:name_not_found .... ``` ### After: ``` > $env.MY_VAR1 Error: nu:🐚:name_not_found .... > $env.MY_VAR2 Error: nu:🐚:name_not_found .... ``` # After Submitting We need to check if any integrations rely on this hack.

@maxim-uvarov

…to eager frames later in the pipeline. (nushell#12525) # Description @maxim-uvarov discovered the following error: ``` > [[a b]; [6 2] [1 4] [4 1]] | polars into-lazy | polars sort-by a | polars unique --subset [a] Error: × Error using as series ╭─[entry #1:1:68] 1 │ [[a b]; [6 2] [1 4] [4 1]] | polars into-lazy | polars sort-by a | polars unique --subset [a] · ──────┬────── · ╰── dataframe has more than one column ╰──── ``` During investigation, I discovered the root cause was that the lazy frame was incorrectly converted back to a eager dataframe. In order to keep this from happening, I explicitly set that the dataframe did not come from an eager frame. This causes the conversion logic to not attempt to convert the dataframe later in the pipeline. --------- Co-authored-by: Jack Wright <jack.wright@disqo.com>

# Description Work for nushell#7149 - **Error `with-env` given uneven count in list form** - **Fix `with-env` `CantConvert` to record** - **Error `with-env` when given protected env vars** - **Deprecate list/table input of vars to `with-env`** - **Remove examples for deprecated input** # User-Facing Changes ## Deprecation of the following forms ``` > with-env [MYENV "my env value"] { $env.MYENV } my env value > with-env [X Y W Z] { $env.X } Y > with-env [[X W]; [Y Z]] { $env.W } Z ``` ## recommended standardized form ``` # Set by key-value record > with-env {X: "Y", W: "Z"} { [$env.X $env.W] } ╭───┬───╮ │ 0 │ Y │ │ 1 │ Z │ ╰───┴───╯ ``` ## (Side effect) Repeated definitions in an env shorthand are now disallowed ``` > FOO=bar FOO=baz $env Error: nu::shell::column_defined_twice × Record field or table column used twice: FOO ╭─[entry #1:1:1] 1 │ FOO=bar FOO=baz $env · ─┬─ ─┬─ · │ ╰── field redefined here · ╰── field first defined here ╰──── ```

@maxim-uvarov

# Description @maxim-uvarov brought up another case where converting back and forth between eager and lazy dataframes was not working correctly: ``` > [[a b]; [6 2] [1 4] [4 1]] | polars into-lazy | polars append -c ([[a b]; [6 2] [1 4] [4 1]] | polars into-df) Error: nu::shell::cant_convert × Can't convert to NuDataFrame. ╭─[entry #1:1:49] 1 │ [[a b]; [6 2] [1 4] [4 1]] | polars into-lazy | polars append -c ([[a b]; [6 2] [1 4] [4 1]] | polars into-df) · ──────┬────── · ╰── can't convert NuLazyFrameCustomValue to NuDataFrame ╰──── ``` This pull request fixes this case and glaringly obvious similar cases I could find. Co-authored-by: Jack Wright <jack.wright@disqo.com>

Should close nushell#10833 — though I'd imagine that should have already been closed. # Description Very minor tweak, but it was quite noticeable when using Zellij which relies on OSC 2 to set pane titles. Before the change: ![image](https://github.com/nushell/nushell/assets/6251883/b944bbce-2040-4886-9955-3c5b57d368e9) Note that the default `Pane #1` is still showing for the untouched shell, but running a command like `htop` or `ls` correctly sets the title during / afterwards. After this PR: ![image](https://github.com/nushell/nushell/assets/6251883/dd513cfe-923c-450f-b0f2-c66938b0d6f0) There are now no-longer any unset titles — even if the shell hasn't been touched. **As an aside:** I feel quite strongly that (at least OSC 2) shell integration should be enabled by default, as it is for every other Linux shell I've used, but I'm not sure which issues that caused that the default config refers to? Which terminals are broken by shell integration, and could some of the shell integrations be turned on by default after splitting things into sub-options as suggested in nushell#11301 ? # User-Facing Changes You'll just have shell integrations working from right after the shell has been launched, instead of needing to run something first. # Tests + Formatting Not quite sure how to test this one? Are there any other tests that currently exist for shell integration? I couldn't quite track them down... # After Submitting Let me know if you think this needs any user-facing docs changes!

# Description Close: nushell#12514 # User-Facing Changes `^ls | skip 1` will raise an error ```nushell ❯ ^ls | skip 1 Error: nu::shell::only_supports_this_input_type × Input type not supported. ╭─[entry #1:1:2] 1 │ ^ls | skip 1 · ─┬ ──┬─ · │ ╰── only list, binary or range input data is supported · ╰── input type: raw data ╰──── ``` # Tests + Formatting Sorry I can't add it because of the issue: nushell#12558 # After Submitting Nan

…stStream` (nushell#12412)  # Description  Prior, it seemed that nested errors would not get detected and shown. This PR fixes that. Resolves nushell#10176: ``` ~/CodingProjects/nushell> [[1,2]] | each {|x| $x | each {|y| error make {msg: "oh noes"} } } 05/04/2024 21:34:08 Error: nu::shell::eval_block_with_input × Eval block failed with pipeline input ╭─[entry #1:1:3] 1 │ [[1,2]] | each {|x| $x | each {|y| error make {msg: "oh noes"} } } · ┬ · ╰── source value ╰──── Error: × oh noes ╭─[entry #1:1:36] 1 │ [[1,2]] | each {|x| $x | each {|y| error make {msg: "oh noes"} } } · ─────┬──── · ╰── originates from here ╰──── ``` Resolves nushell#11224: ``` ~/CodingProjects/nushell> [0] | each { |_| 05/04/2024 21:35:40 ::: [0] | each { |_| ::: non-existent-command ::: } ::: } Error: nu::shell::eval_block_with_input × Eval block failed with pipeline input ╭─[entry #1:2:6] 1 │ [0] | each { |_| 2 │ [0] | each { |_| · ┬ · ╰── source value 3 │ non-existent-command ╰──── Error: nu:🐚:external_command × External command failed ╭─[entry #1:3:9] 2 │ [0] | each { |_| 3 │ non-existent-command · ──────────┬───────── · ╰── executable was not found 4 │ } ╰──── help: No such file or directory (os error 2) ``` # User-Facing Changes  # Tests + Formatting  # After Submitting

@yizhepku

# Description Fix a regression introduced by nushell#12921, where tilde expansion was no longer done on the external command name, breaking things like ```nushell > ~/.cargo/bin/exa ``` This properly handles quoted strings, so they don't expand: ```nushell > ^"~/.cargo/bin/exa" Error: nu::shell::external_command × External command failed ╭─[entry #1:1:2] 1 │ ^"~/.cargo/bin/exa" · ─────────┬──────── · ╰── Command `~/.cargo/bin/exa` not found ╰──── help: `~/.cargo/bin/exa` is neither a Nushell built-in or a known external command ``` This required a change to the parser, so the command name is also parsed in the same way the arguments are - i.e. the quotes on the outside remain in the expression. Hopefully that doesn't break anything else. 🤞 Fixes nushell#13000. Should include in patch release 0.94.1 cc @yizhepku # User-Facing Changes - Tilde expansion now works again for external commands - The `command` of `run-external` will now have its quotes removed like the other arguments if it is a literal string - The parser is changed to include quotes in the command expression of `ExternalCall` if they were present # Tests + Formatting I would like to add a regression test for this, but it's complicated because we need a well-known binary within the home directory, which just isn't a thing. We could drop one there, but that's kind of a bad behavior for a test to do. I also considered changing the home directory for the test, but that's so platform-specific - potentially could get it working on specific platforms though. Changing `HOME` env on Linux definitely works as far as tilde expansion works. - 🟢 `toolkit fmt` - 🟢 `toolkit clippy` - 🟢 `toolkit test` - 🟢 `toolkit test stdlib`

…ushell#13131) # Description Closes: nushell#13010 It adds an additional check inside `parse_string`, and returns `unbalanced quote` if input string is unbalanced # User-Facing Changes After this pr, the following is no longer allowed: ```nushell ❯ "asdfasdf"asdfasdf Error: nu::parser::extra_token_after_closing_delimiter × Invaild characters after closing delimiter ╭─[entry #1:1:11] 1 │ "asdfasdf"asdfasdf · ────┬─── · ╰── invalid characters ╰──── help: Try removing them. ❯ 'asdfasd'adsfadf Error: nu::parser::extra_token_after_closing_delimiter × Invaild characters after closing delimiter ╭─[entry nushell#2:1:10] 1 │ 'asdfasd'adsfadf · ───┬─── · ╰── invalid characters ╰──── help: Try removing them. ``` # Tests + Formatting Added 1 test

# Description Fixes nushell#13280. After apply this patch, we can use non-timezone string + format option `into datetime` cmd # User-Facing Changes AS-IS (before fixing) ``` $ "09.02.2024 11:06:11" | into datetime --format '%m.%d.%Y %T' Error: nu::shell::cant_convert × Can't convert to could not parse as datetime using format '%m.%d.%Y %T'. ╭─[entry #1:1:25] 1 │ "09.02.2024 11:06:11" | into datetime --format '%m.%d.%Y %T' · ──────┬────── · ╰── can't convert input is not enough for unique date and time to could not parse as datetime using format '%m.%d.%Y %T' ╰──── help: you can use `into datetime` without a format string to enable flexible parsing $ "09.02.2024 11:06:11" | into datetime Mon, 2 Sep 2024 11:06:11 +0900 (in 2 months) ``` TO-BE(After fixing) ``` $ "09.02.2024 11:06:11" | into datetime --format '%m.%d.%Y %T' Mon, 2 Sep 2024 20:06:11 +0900 (in 2 months) $ "09.02.2024 11:06:11" | into datetime Mon, 2 Sep 2024 11:06:11 +0900 (in 2 months) ``` # Tests + Formatting If there is agreement on the direction, I will add a test. # After Submitting --------- Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>

@amtoine

# Description From the feedbacks from @amtoine , it's good to make nushell shows error for `o>|` syntax. # User-Facing Changes ## Before ```nushell 'foo' o>| print 07/09/2024 06:44:23 AM Error: nu::parser::parse_mismatch × Parse mismatch during operation. ╭─[entry nushell#6:1:9] 1 │ 'foo' o>| print · ┬ · ╰── expected redirection target ``` ## After ```nushell 'foo' o>| print 07/09/2024 06:47:26 AM Error: nu::parser::parse_mismatch × Parse mismatch during operation. ╭─[entry #1:1:7] 1 │ 'foo' o>| print · ─┬─ · ╰── expected `|`. Redirection stdout to pipe is the same as piping directly. ╰──── ``` # Tests + Formatting Added one test --------- Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>

# Description As part of fixing nushell#13586, this PR checks the types of the operands when creating a range. Stuff like `0..(glob .)` will be rejected at parse time. Additionally, `0..$x` will be treated as a range and rejected if `x` is not defined, rather than being treated as a string. A separate PR will need to be made to do reject streams at runtime, so that stuff like `0..(open /dev/random)` doesn't hang. Internally, this PR adds a `ParseError::UnsupportedOperationTernary` variant, for when you have a range like `1..2..(glob .)`. # User-Facing Changes Users will now receive an error if any of the operands in the ranges they construct have types that aren't compatible with `Type::Number`. Additionally, if a piece of code looks like a range but some parse error is encountered while parsing it, that piece of code will still be treated as a range and the user will be shown the parse error. This means that a piece of code like `0..$x` will be treated as a range no matter what. Previously, if `x` weren't the expression would've been treated as a string `"0..$x"`. I feel like it makes the language less complicated if we make it less context-sensitive. Here's an example of the error you get: ``` > 0..(glob .) Error: nu::parser::unsupported_operation × range is not supported between int and any. ╭─[entry #1:1:1] 1 │ 0..(glob .) · ─────┬─────┬┬ · │ │╰── any · │ ╰── int · ╰── doesn't support these values ╰──── ``` And as an image: ![image](https://github.com/user-attachments/assets/5c76168d-27db-481b-b541-861dac899dbf) Note: I made the operands themselves (above, `(glob .)`) be garbage, rather than the `..` operator itself. This doesn't match the behavior of the math operators (if you do `1 + "foo"`, `+` gets highlighted red). This is because with ranges, the range operators aren't `Expression`s themselves, so they can't be turned into garbage. I felt like here, it makes more sense to highlight the individual operand anyway.

# Description This PR makes it so that non-zero exit codes and termination by signal are treated as a normal `ShellError`. Currently, these are silent errors. That is, if an external command fails, then it's code block is aborted, but the parent block can sometimes continue execution. E.g., see nushell#8569 and this example: ```nushell [1 2] | each { ^false } ``` Before this would give: ``` ╭───┬──╮ │ 0 │ │ │ 1 │ │ ╰───┴──╯ ``` Now, this shows an error: ``` Error: nu::shell::eval_block_with_input × Eval block failed with pipeline input ╭─[entry #1:1:2] 1 │ [1 2] | each { ^false } · ┬ · ╰── source value ╰──── Error: nu:🐚:non_zero_exit_code × External command had a non-zero exit code ╭─[entry #1:1:17] 1 │ [1 2] | each { ^false } · ──┬── · ╰── exited with code 1 ╰──── ``` This PR fixes nushell#12874, fixes nushell#5960, fixes nushell#10856, and fixes nushell#5347. This PR also partially addresses nushell#10633 and nushell#10624 (only the last command of a pipeline is currently checked). It looks like nushell#8569 is already fixed, but this PR will make sure it is definitely fixed (fixes nushell#8569). # User-Facing Changes - Non-zero exit codes and termination by signal now cause an error to be thrown. - The error record value passed to a `catch` block may now have an `exit_code` column containing the integer exit code if the error was due to an external command. - Adds new config values, `display_errors.exit_code` and `display_errors.termination_signal`, which determine whether an error message should be printed in the respective error cases. For non-interactive sessions, these are set to `true`, and for interactive sessions `display_errors.exit_code` is false (via the default config). # Tests Added a few tests. # After Submitting - Update docs and book. - Future work: - Error if other external commands besides the last in a pipeline exit with a non-zero exit code. Then, deprecate `do -c` since this will be the default behavior everywhere. - Add a better mechanism for exit codes and deprecate `$env.LAST_EXIT_CODE` (it's buggy).

# Description Old code was comparing remaining positional arguments with total number of arguments, where it should've compared remaining positional with with remaining arguments of any kind. This means that if a function was given too few arguments, `calculate_end_span` would believe that it actually had too many arguments, since after parsing the first few arguments, the number of remaining arguments needed were fewer than the *total* number of arguments, of which we had used several. Fixes nushell#9072 Fixes: nushell#13930 Fixes: nushell#12069 Fixes: nushell#8385 Extracted from nushell#10381 ## Bonus It also improves the error handling on missing positional arguments before keywords (no longer crashing since nushell#9851). Instead of just giving the keyword to the parser for the missing positional, we give an explicit error about a missing positional argument. I would like better descriptions than "missing var_name" though, but I'm not sure if that's available without Old error ``` Error: nu::parser::parse_mismatch × Parse mismatch during operation. ╭─[entry #1:1:1] 1 │ let = if foo · ┬ · ╰── expected valid variable name ╰──── ``` New error ``` Error: nu::parser::missing_positional × Missing required positional argument. ╭─[entry nushell#18:1:1] 1 │ let = foo · ┬ · ╰── missing var_name ╰──── help: Usage: let <var_name> = <initial_value> ``` # User-Facing Changes The program `alias = = =` is no longer accepted by the parser

) # Description This PR updates `group-by` and `split-by` to allow other nushell Values to be used, namely bools. ### Before ```nushell ❯ [false, false, true, false, true, false] | group-by | table -e Error: nu::shell::cant_convert × Can't convert to string. ╭─[entry #1:1:2] 1 │ [false, false, true, false, true, false] | group-by | table -e · ──┬── · ╰── can't convert bool to string ╰──── ``` ### After ```nushell ❯ [false, false, true, false, true, false] | group-by | table -e ╭───────┬───────────────╮ │ │ ╭───┬───────╮ │ │ false │ │ 0 │ false │ │ │ │ │ 1 │ false │ │ │ │ │ 2 │ false │ │ │ │ │ 3 │ false │ │ │ │ ╰───┴───────╯ │ │ │ ╭───┬──────╮ │ │ true │ │ 0 │ true │ │ │ │ │ 1 │ true │ │ │ │ ╰───┴──────╯ │ ╰───────┴───────────────╯ ``` # User-Facing Changes  # Tests + Formatting  # After Submitting

…ell#14118) Fixes nushell#14023 # Description - Prevents "failed to find added variable" when modules export constants with type signatures: ```nushell > module foo { export const bar: int = 2 } Error: nu::parser::unknown_state × Internal error. ╭─[entry #1:1:21] 1 │ module foo { export const bar: int = 2 } · ─────────┬──────── · ╰── failed to find added variable ``` - Returns `name_is_builtin_var` errors for names with type signatures: ```nushell > let env: string = ""; Error: nu::parser::name_is_builtin_var × `env` used as variable name. ╭─[entry #1:1:5] 1 │ let env: string = ""; · ─┬─ · ╰── already a builtin variable ```

# Description  This PR fixes the quoting and escaping of column names in `to nuon`. Before the PR, column names with quotes inside them would get quoted, but not escaped: ```nushell > { 'a"b': 2 } | to nuon { "a"b": 2 } > { 'a"b': 2 } | to nuon | from nuon Error: × error when loading nuon text ╭─[entry #1:1:27] 1 │ { "a\"b": 2 } | to nuon | from nuon · ────┬──── · ╰── could not load nuon text ╰──── Error: × error when parsing nuon text ╭─[entry #1:1:27] 1 │ { "a\"b": 2 } | to nuon | from nuon · ────┬──── · ╰── could not parse nuon text ╰──── Error: × error when parsing ╭──── 1 │ {"a"b": 2} · ┬ · ╰── Unexpected end of code. ╰──── > [['a"b']; [2] [3]] | to nuon [["a"b"]; [2], [3]] > [['a"b']; [2] [3]] | to nuon | from nuon Error: × error when loading nuon text ╭─[entry #1:1:32] 1 │ [['a"b']; [2] [3]] | to nuon | from nuon · ────┬──── · ╰── could not load nuon text ╰──── Error: × error when parsing nuon text ╭─[entry #1:1:32] 1 │ [['a"b']; [2] [3]] | to nuon | from nuon · ────┬──── · ╰── could not parse nuon text ╰──── Error: × error when parsing ╭──── 1 │ [["a"b"]; [2], [3]] · ┬ · ╰── Unexpected end of code. ╰──── ``` After this PR, the quote is escaped properly: ```nushell > { 'a"b': 2 } | to nuon { "a\"b": 2 } > { 'a"b': 2 } | to nuon | from nuon ╭─────┬───╮ │ a"b │ 2 │ ╰─────┴───╯ > [['a"b']; [2] [3]] | to nuon [["a\"b"]; [2], [3]] > [['a"b']; [2] [3]] | to nuon | from nuon ╭─────╮ │ a"b │ ├─────┤ │ 2 │ │ 3 │ ╰─────╯ ``` The cause of the issue was that `to nuon` simply wrapped column names in `'"'` instead of calling `escape_quote_string`. As part of this change, I also moved the functions related to quoting (`needs_quoting` and `escape_quote_string`) into `nu-utils`, since previously they were defined in very ad-hoc places (and, in the case of `escape_quote_string`, it was defined multiple times with the same body!). # User-Facing Changes  `to nuon` now properly escapes quotes in column names. # Tests + Formatting  All tests pass, including workspace and stdlib tests. # After Submitting

…nushell#14353) Fixes nushell#14252 # User-Facing Changes - Special characters in module names are replaced with underscores when importing constants, preventing "expected valid variable name": ```nushell > module foo-bar { export const baz = 1 } > use foo-bar > $foo_bar.baz ``` - "expected valid variable name" errors now include a suggestion list: ```nushell > module foo-bar { export const baz = 1 } > use foo-bar > $foo-bar Error: nu::parser::parse_mismatch_with_did_you_mean × Parse mismatch during operation. ╭─[entry #1:1:1] 1 │ $foo-bar; · ────┬─── · ╰── expected valid variable name. Did you mean '$foo_bar'? ╰──── ```

…time (nushell#14303) Alternative solution to: - nushell#12195 The other approach: - nushell#14305 # Description Adds ~`path const`~ `path self`, a parse-time only command for getting the absolute path of the source file containing it, or any file relative to the source file. - Useful for any script or module that makes use of non nuscript files. - Removes the need for `$env.CURRENT_FILE` and `$env.FILE_PWD`. - Can be used in modules, sourced files or scripts. # Examples ```nushell # ~/.config/nushell/scripts/foo.nu const paths = { self: (path self), dir: (path self .), sibling: (path self sibling), parent_dir: (path self ..), cousin: (path self ../cousin), } export def main [] { $paths } ``` ```nushell > use foo.nu > foo ╭────────────┬────────────────────────────────────────────╮ │ self │ /home/user/.config/nushell/scripts/foo.nu │ │ dir │ /home/user/.config/nushell/scripts │ │ sibling │ /home/user/.config/nushell/scripts/sibling │ │ parent_dir │ /home/user/.config/nushell │ │ cousin │ /home/user/.config/nushell/cousin │ ╰────────────┴────────────────────────────────────────────╯ ``` Trying to run in a non-const context ```nushell > path self Error: × this command can only run during parse-time ╭─[entry #1:1:1] 1 │ path self · ─────┬──── · ╰── can't run after parse-time ╰──── help: try assigning this command's output to a const variable ``` Trying to run in the REPL i.e. not in a file ```nushell > const foo = path self Error: × Error: nu:🐚:file_not_found │ │ × File not found │ ╭─[entry nushell#3:1:13] │ 1 │ const foo = path self │ · ─────┬──── │ · ╰── Couldn't find current file │ ╰──── │ ╭─[entry nushell#3:1:13] 1 │ const foo = path self · ─────┬──── · ╰── Encountered error during parse-time evaluation ╰──── ``` # Comparison with nushell#14305 ## Pros - Self contained implementation, does not require changes in the parser. - More concise usage, especially with parent directories. --------- Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>

# Description Closes nushell#14521 This PR tweaks the way 64-bit hex numbers are parsed. ### Before ```nushell ❯ 0xffffffffffffffef Error: nu::shell::external_command × External command failed ╭─[entry #1:1:1] 1 │ 0xffffffffffffffef · ─────────┬──────── · ╰── Command `0xffffffffffffffef` not found ╰──── help: `0xffffffffffffffef` is neither a Nushell built-in or a known external command ``` ### After ```nushell ❯ 0xffffffffffffffef -17 ``` # User-Facing Changes  # Tests + Formatting  # After Submitting

# Description  This PR makes two changes related to [run-time pipeline input type checking](nushell#14741): 1. The check which bypasses type checking for commands with only `Type::Nothing` input types has been expanded to work with commands with multiple `Type::Nothing` inputs for different outputs. For example, `ast` has three input/output type pairs, but all of the inputs are `Type::Nothing`: ``` ╭───┬─────────┬────────╮ │ # │ input │ output │ ├───┼─────────┼────────┤ │ 0 │ nothing │ table │ │ 1 │ nothing │ record │ │ 2 │ nothing │ string │ ╰───┴─────────┴────────╯ ``` Before this PR, passing a value (which would otherwise be ignored) to `ast` caused a run-time type error: ``` Error: nu::shell::only_supports_this_input_type × Input type not supported. ╭─[entry #1:1:6] 1 │ echo 123 | ast -j -f "hi" · ─┬─ ─┬─ · │ ╰── only nothing, nothing, and nothing input data is supported · ╰── input type: int ╰──── ``` After this PR, no error is raised. This doesn't really matter for `ast` (the only other built-in command with a similar input/output type signature is `cal`), but it's more logically consistent. 2. Bypasses input type-checking (parse-time ***and*** run-time) for some (not all, see below) commands which have both a `Type::Nothing` input and some other non-nothing `Type` input. This is accomplished by adding a `Type::Any` input with the same output as the corresponding `Type::Nothing` input/output pair.   This is necessary because some commands are intended to operate on an argument with empty pipeline input, or operate on an empty pipeline input with no argument. This causes issues when a value is implicitly passed to one of these commands. I [discovered this issue](https://discord.com/channels/601130461678272522/615962413203718156/1329945784346611712) when working with an example where the `open` command is used in `sort-by` closure: ```nushell ls | sort-by { open -r $in.name | lines | length } ``` Before this PR (but after the run-time input type checking PR), this error is raised: ``` Error: nu::shell::only_supports_this_input_type × Input type not supported. ╭─[entry #1:1:1] 1 │ ls | sort-by { open -r $in.name | lines | length } · ─┬ ──┬─ · │ ╰── only nothing and string input data is supported · ╰── input type: record<name: string, type: string, size: filesize, modified: date> ╰──── ``` While this error is technically correct, we don't actually want to return an error here since `open` ignores its pipeline input when an argument is passed. This would be a parse-time error as well if the parser was able to infer that the closure input type was a record, but our type inference isn't that robust currently, so this technically incorrect form snuck by type checking until nushell#14741. However, there are some commands with the same kind of type signature where this behavior is actually desirable. This means we can't just bypass type-checking for any command with a `Type::Nothing` input. These commands operate on true `null` values, rather than ignoring their input. For example, `length` returns `0` when passed a `null` value. It's correct, and even desirable, to throw a run-time error when `length` is passed an unexpected type. For example, a string, which should instead be measured with `str length`: ```nushell ["hello" "world"] | sort-by { length } # => Error: nu::shell::only_supports_this_input_type # => # => × Input type not supported. # => ╭─[entry nushell#32:1:10] # => 1 │ ["hello" "world"] | sort-by { length } # => · ───┬─── ───┬── # => · │ ╰── only list<any>, binary, and nothing input data is supported # => · ╰── input type: string # => ╰──── ``` We need a more robust way for commands to express how they handle the `Type::Nothing` input case. I think a possible solution here is to allow commands to express that they operate on `PipelineData::Empty`, rather than `Value::Nothing`. Then, a command like `open` could have an empty pipeline input type rather than a `Type::Nothing`, and the parse-time and run-time pipeline input type checks know that `open` will safely ignore an incorrectly typed input. That being said, we have a release coming up and the above solution might take a while to implement, so while unfortunate, bypassing input type-checking for these problematic commands serves as a workaround to avoid breaking changes in the release until a more robust solution is implemented. This PR bypasses input type-checking for the following commands: * `load-env`: can take record of envvars as input or argument * `nu-check`: checks input string or filename argument * `open`: can take filename as input or argument * `polars when`: can be used with input, or can be chained with another `polars when` * `stor insert`: data record can be passed as input or argument * `stor update`: data record can be passed as input or argument * `format date`: `--list` ignores input value * `into datetime`: `--list` ignores input value (also added a `Type::Nothing` input which was missing from this command) These commands have a similar input/output signature to the above commands, but are working as intended: * `cd`: The input/output signature was actually incorrect, `cd` always ignores its input. I fixed this in this PR. * `generate` * `get` * `history import` * `interleave` * `into bool` * `length` # User-Facing Changes  As a temporary workaround, pipeline input type-checking for the following commands has been bypassed to avoid undesirable run-time input type checking errors which were previously not caught at parse-time: * `open` * `load-env` * `format date` * `into datetime` * `nu-check` * `stor insert` * `stor update` * `polars when` # Tests + Formatting  CI became green in the time it took me to type the description 😄 # After Submitting  N/A

Fixes nushell#14960 # User-Facing Changes - The output of non-streaming values can now be interrupted with ctrl-c: ```nushell ~> use std repeat; random chars --length 100kb | repeat 2000 | str join ' ' | collect <data omitted>^C Error: × Operation interrupted ╭─[entry #1:1:61] 1 │ use std repeat; random chars --length 100kb | repeat 2000 | str join ' ' | collect · ────┬─── · ╰── This operation was interrupted ╰──── ``` - When IO errors occur while printing data, nushell no longer panics: ```diff $ nu -c "true | print" | - -Error: - x Main thread panicked. - |-> at crates/nu-protocol/src/errors/shell_error/io.rs:198:13 - `-> for unknown spans with paths, use `new_internal_with_path` +Error: nu:🐚:io::broken_pipe + + x I/O error + `-> x Broken pipe + + ,-[source:1:1] + 1 | true | print + : ^^|^ + : `-| Writing to stdout failed + : | Broken pipe + `---- ```

# Description This PR adds two new `ParseError` and `ShellError` cases for type errors relating to operators. - `OperatorUnsupportedType` is used when a type is not supported by an operator in any way, shape, or form. E.g., `+` does not support `bool`. - `OperatorIncompatibleTypes` is used when a operator is used with types it supports, but the combination of types provided cannot be used together. E.g., `filesize + duration` is not a valid combination. The other preexisting error cases related to operators have been removed and replaced with the new ones above. Namely: - `ShellError::OperatorMismatch` - `ShellError::UnsupportedOperator` - `ParseError::UnsupportedOperationLHS` - `ParseError::UnsupportedOperationRHS` - `ParseError::UnsupportedOperationTernary` # User-Facing Changes - `help operators` now lists the precedence of `not` as 55 instead of 0 (above the other boolean operators). Fixes nushell#13675. - `math median` and `math mode` now ignore NaN values so that `[NaN NaN] | math median` and `[NaN NaN] | math mode` no longer trigger a type error. Instead, it's now an empty input error. Fixing this in earnest can be left for a future PR. - Comparisons with `nan` now return false instead of causing an error. E.g., `1 == nan` is now `false`. - All the operator type errors have been standardized and reworked. In particular, they can now have a help message, which is currently used for types errors relating to `++`. ```nu [1] ++ 2 ``` ``` Error: nu::parser::operator_unsupported_type × The '++' operator does not work on values of type 'int'. ╭─[entry #1:1:5] 1 │ [1] ++ 2 · ─┬ ┬ · │ ╰── int · ╰── does not support 'int' ╰──── help: if you meant to append a value to a list or a record to a table, use the `append` command or wrap the value in a list. For example: `$list ++ $value` should be `$list ++ [$value]` or `$list | append $value`. ```

Closes nushell#15160 # User-Facing Changes Certain "variable not found" errors no longer highlight the surrounding block. Before: ```nushell do { match foo { _ => $in } } Error: nu::shell::variable_not_found × Variable not found ╭─[entry #1:1:1] 1 │ ╭─▶ do { 2 │ │ match foo { 3 │ │ _ => $in 4 │ │ } 5 │ ├─▶ } · ╰──── variable not found ``` After: ```nushell Error: nu:🐚:variable_not_found × Variable not found ╭─[entry #1:3:10] 2 │ match foo { 3 │ _ => $in · ─┬─ · ╰── variable not found ```

# Description When attempting to pass a null byte in a commandline argument, Nu currently fails with: ``` > ^echo (char -i 0) Error: nu::shell::io::invalid_input × I/O error ╰─▶ × Could not spawn foreground child ╭──── 1 │ crates/nu-command/src/system/run_external.rs:284:17 · ─────────────────────────┬───────────────────────── · ╰── Invalid input parameter ╰──── ``` This does not explain which input parameter is invalid, or why. Since Nu does not typically seem to escape null bytes when printing values containing them, this can make it a bit tricky to track down the problem. After this change, it fails with: ``` > ^echo (char -i 0) Error: nu::shell::io::invalid_input × I/O error ╰─▶ × Could not spawn foreground child: nul byte found in provided data ╭──── 1 │ crates/nu-command/src/system/run_external.rs:282:17 · ─────────────────────────┬───────────────────────── · ╰── Invalid input parameter ╰──── ``` which is more useful. This could be improved further but this is niche enough that is probably not necessary. This might make some other errors unnecessarily verbose but seems like the better default. I did check that attempting to execute a non-executable file still has a reasonable error: the error message for that failure is not affected by this change. It is still an "internal" error (referencing the Nu code triggering it, not the user's input) because the `call.head` span available to this code is for the command, not its arguments. Using it would result in ``` × I/O error ╰─▶ × Could not spawn foreground child: nul byte found in provided data ╭─[entry #1:1:2] 1 │ ^echo (char -i 0) · ──┬─ · ╰── Invalid input parameter ╰──── ``` which is actively misleading because "echo" does not contain the nul byte. # User-Facing Changes  # Tests + Formatting  Haven't tried to write a test yet: it's tricky because the better error message comes from the Rust stdlib (so a straightforward integration test checking for the specific message would be brittle)... # After Submitting

# Description  Rel: nushell#14429, nushell#16079 Finishes up a TODO in the assignment type checking. - For regular assignment operations (only applies to `mut`), type checking is now done using `type_compatible` (which is what `let` uses) - This allows some mutable assignments to work which weren't allowed before Before: ```nushell let x: glob = "" # => ok, no error mut x: glob = ""; $x = "" # => Error: nu::parser::operator_incompatible_types # => # => × Types 'glob' and 'string' are not compatible for the '=' operator. # => ╭─[entry nushell#6:1:19] # => 1 │ mut x: glob = ""; $x = "" # => · ─┬ ┬ ─┬ # => · │ │ ╰── string # => · │ ╰── does not operate between 'glob' and 'string' # => · ╰── glob # => ╰──── let x: number = 1 # ok, no error mut x: number = 1; $x = 2 # => Error: nu::parser::operator_incompatible_types # => # => × Types 'number' and 'int' are not compatible for the '=' operator. # => ╭─[source:1:20] # => 1 │ mut x: number = 1; $x = 2 # => · ─┬ ┬ ┬ # => · │ │ ╰── int # => · │ ╰── does not operate between 'number' and 'int' # => · ╰── number # => ╰──── ``` After: ```nushell let x: glob = "" # ok, no error (same as before) mut x: glob = ""; $x = "" # ok, no error let x: number = 1 # ok, no error (same as before) mut x: number = 1; $x = 2 # ok, no error ``` - Properly type check compound operations. First checks if the operation (eg. `+` for `+=`) type checks successfully, and then checks if the assignment type checks successfully (also using `type_compatible`) - This fixes some issues where the "long version" of a compound assignment operator would error, but the compound assignment operator itself would not Before: ```nushell mut x = 1; $x = $x / 2 # => Error: nu::parser::operator_incompatible_types # => # => × Types 'int' and 'float' are not compatible for the '=' operator. # => ╭─[entry nushell#15:1:12] # => 1 │ mut x = 1; $x = $x / 2 # => · ─┬ ┬ ───┬── # => · │ │ ╰── float # => · │ ╰── does not operate between 'int' and 'float' # => · ╰── int # => ╰──── mut x = 1; $x /= 2 # uh oh, no error... mut x = (date now); $x = $x - 2019-05-10 # => Error: nu::parser::operator_incompatible_types # => # => × Types 'datetime' and 'duration' are not compatible for the '=' operator. # => ╭─[entry #1:1:21] # => 1 │ mut x = (date now); $x = $x - 2019-05-10 # => · ─┬ ┬ ───────┬─────── # => · │ │ ╰── duration # => · │ ╰── does not operate between 'datetime' and 'duration' # => · ╰── datetime # => ╰──── mut x = (date now); $x -= 2019-05-10 # uh oh, no error... (the result of this is a duration, not a datetime) ``` After: ```nushell mut x = 1; $x = $x / 2 # => Error: nu::parser::operator_incompatible_types # => # => × Types 'int' and 'float' are not compatible for the '=' operator. # => ╭─[entry nushell#5:1:12] # => 1 │ mut x = 1; $x = $x / 2 # => · ─┬ ┬ ───┬── # => · │ │ ╰── float # => · │ ╰── does not operate between 'int' and 'float' # => · ╰── int # => ╰──── mut x = (date now); $x -= 2019-05-10 # => Error: nu::parser::operator_incompatible_types # => # => × Types 'datetime' and 'datetime' are not compatible for the '-=' operator. # => ╭─[entry nushell#11:1:21] # => 1 │ mut x = (date now); $x -= 2019-05-10 # => · ─┬ ─┬ ─────┬──── # => · │ │ ╰── datetime # => · │ ╰── does not operate between 'datetime' and 'datetime' # => · ╰── datetime # => ╰──── # => help: The result type of this operation is not compatible with the type of the variable. ``` This is technically a breaking change if you relied on the old behavior (for example, there was a test that broke after this change because it relied on `/=` improperly type checking) # User-Facing Changes  * Mutable assignment operations now use the same type checking rules as normal assignments * For example, `$x = 123` now uses the same type checking rules as `let x = 123` or `mut x = 123` * Compound assignment operations now type check using the same rules as the operation they use * Assignment errors will also now highlight the invalid assignment operator in red # Tests + Formatting  Adds some tests for the examples given above # After Submitting  N/A

@cptpiepmatz

# Description Before this PR, errors without error codes are printed somewhat strangely, with the `×` and the description being printed on the same line as the `Error:` text: Error: × Invalid literal ╭─[entry #1:1:2] 1 │ "\z" · ─┬─ · ╰── unrecognized escape after '\' in string ╰──── This PR adds a default error code for the different error types: Error: nu::parser::error × Invalid literal ╭─[entry #1:1:2] 1 │ "\z" · ─┬─ · ╰── unrecognized escape after '\' in string ╰──── While maybe not as informative as a proper error code, it makes `GenericError`s and other things which don't have error codes look a lot nicer. It would be nicer if we could just set `diagnostic(code: "nu:🐚:error")` at the top of `ShellError`, but unfortunately you can't set a "default" at the `enum` level and then override it in the variants. @cptpiepmatz mentioned he might change miette's derive macro to accommodate this, in that case we can switch the approach here. # User-Facing Changes * Errors without error codes now have a default error code corresponding to from which part of Nushell the error occurred (shell, parser, compile, etc) --------- Co-authored-by: Bahex <17417311+Bahex@users.noreply.github.com>

@cptpiepmatz

# Description Before this PR, errors without error codes are printed somewhat strangely, with the `×` and the description being printed on the same line as the `Error:` text: Error: × Invalid literal ╭─[entry #1:1:2] 1 │ "\z" · ─┬─ · ╰── unrecognized escape after '\' in string ╰──── This PR adds a default error code for the different error types: Error: nu::parser::error × Invalid literal ╭─[entry #1:1:2] 1 │ "\z" · ─┬─ · ╰── unrecognized escape after '\' in string ╰──── While maybe not as informative as a proper error code, it makes `GenericError`s and other things which don't have error codes look a lot nicer. It would be nicer if we could just set `diagnostic(code: "nu:🐚:error")` at the top of `ShellError`, but unfortunately you can't set a "default" at the `enum` level and then override it in the variants. @cptpiepmatz mentioned he might change miette's derive macro to accommodate this, in that case we can switch the approach here. # User-Facing Changes * Errors without error codes now have a default error code corresponding to from which part of Nushell the error occurred (shell, parser, compile, etc) --------- Co-authored-by: Bahex <17417311+Bahex@users.noreply.github.com>

hardfau1t and others added 5 commits December 14, 2023 22:09

fix(cd): on android/termux fails to cd into /sdcard

3552abd

or any directory that user has access via group

refmt

833e373

fix lints

24e0eac

Removed slipped patch file

bbe8021

use group count returned from the lateral getgroups in `current_use…

ab208a5

…r_groups` Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>

hardfau1t merged commit e3641f9 into main Dec 14, 2023

hardfau1t deleted the dev branch December 14, 2023 16:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dev#1

Dev#1
hardfau1t merged 5 commits intomainfrom
dev

hardfau1t commented Dec 14, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

hardfau1t commented Dec 14, 2023

Description

User-Facing Changes

Tests + Formatting

After Submitting

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant