standard library: implement `help` commands with `$nu` by amtoine · Pull Request #8505 · nushell/nushell

amtoine · 2023-03-17T17:30:45Z

Related to one task in #8311.

Description

this PR implements the built-in help commands in the standard library using the $nu variable
as the only source to access information about the commands, aliases and modules.
this can be thought as a proof of concept and a replacement proposition in the long-run, as the
help commands are not performance-critical and might not require taking up compilation time
and space.

Note
this PR has in mind the goal of cloning the builtin help commands.
this is what i've tried as much as possible with the information available in $nu 😌

However, some things appear impossible right now, see the Warning below.

progress

help modules
help aliases
help externs ⚠️ might require augmenting $nu with external flags
help operators ✔️ ~~might require augmenting $nu with the table of operators~~ => done in 55350c1
help commands
help ✔️ ~~can not find a way to implement that...~~ => done in a387256b4453cd8c04c28dce002e301da7d06157..62cf425efd8d5907115b5fc7f17ecccfe3c2763e

details

the error system

the *-not-found-error are here as convenience wrappers to throw all required errors, i.e. when
a user parameter does not correspond to any commands, alias or module.

the actual `help` commands

the implemented help commands follow the same structure

use print-help-header to print the section names in the help messages
define a show-* command that takes the command, alias or module information in the form of a
record and prints as much as possible in the same format as the built-ins
define a help * each having the same signature and behaviour
- get the whole collections of commands, aliases or modules, sorted by name
- if --find is used, matches the name against the pattern
  - if the result is alone, expand it
  - if not a single result, show the raw table
- if --find is not used but a name is still provided, show it or throw a *-not-found-error
- otherwise show all the commands, aliases or modules

User-Facing Changes

users can now access the help commands via use std.nu and not rely on the builtins anymore.

Tests + Formatting

i do not really know what to write here...
should i take a few know commands and compare the outputs of the help on them?

After Submitting

i'd say probably part of the standard library announcement 👍

By header i mean the green text that appears as a section name. e.g. the "search terms" in "Search terms: load, read, load_file, read_file" when running ``` help open | lines | get 2 ```

- `module-not-found-error` is the error message wrapper - `show-module` is the function that shows a single module doc - `help modules` is designed as a clone of the built-in `help modules`

I've simply copy the table given by `help operators` and kept the same logic as before.

They do not sarch usages.

@jntrnr

This is almost a perfect copy-paste from the [`print_help.nu` gist from @jntrnr](https://gist.github.com/jntrnr/bb7f8827db14131ab95229e1c1efcc42).

…dard-library This commit solves the merge conflict for nushell#8505. The conflict has been solved such that the new `help ...` commands are appended to `std.nu`.

amtoine · 2023-03-18T17:03:37Z

there it is finally 🎉

i'd love some feedback on that 😋

my main concerns

the ~~two~~ one 🔴 points above
the 🟠 point above
the flags of externals that are not in $nu
the help command => i have a draft for this, but it does not work that well for now: should i keep working on this?

amtoine · 2023-03-29T17:27:21Z

help commands is returning too many columns compared to the built-in

oh yeah good catch 👍

fixed in 6f7c7a1 👍

Warning
$nu.scope.commands.command_type does not exist... so this is the best i can do with $nu only

>_ use crates/nu-utils/standard_library/std.nu
>_ std help commands | columns
╭───┬──────────────╮
│ 0 │ name         │
│ 1 │ category     │
│ 2 │ usage        │
│ 3 │ signatures   │
│ 4 │ search_terms │
╰───┴──────────────╯
>_ help commands | columns
╭───┬──────────────╮
│ 0 │ name         │
│ 1 │ category     │
│ 2 │ command_type │
│ 3 │ usage        │
│ 4 │ signatures   │
│ 5 │ search_terms │
╰───┴──────────────╯

amtoine · 2023-03-29T17:27:49Z

@fdncred i think i've answered to everything 😌

fdncred · 2023-03-29T18:57:27Z

Related to help -f arg and coloring. We could do something like this

$nu.scope.commands | find arg | reject module_name examples is_builtin is_sub is_plugin is_custom is_keyword is_extern creates_scope extra_usage

but i think the signatures would need to be updated. Something weird is going on with signatures here recently.

This

versus this

Note the signature in the later are like <nothing> | bytes build --> <binary>

amtoine · 2023-03-30T16:37:10Z

Related to help -f arg and coloring. We could do something like this
$nu.scope.commands | find arg | reject module_name examples is_builtin is_sub is_plugin is_custom is_keyword is_extern creates_scope extra_usage

with this, we gain the highlight but, as i said, we not only search the name, usage and search terms 🤔
do we want

hightlighting with find
narrower search to avoid hitting garbase matches
?

but i think the signatures would need to be updated. Something weird is going on with signatures here recently.

Note the signature in the later are like <nothing> | bytes build --> <binary>

oh yeah you're right 🤔
that's $nu that needs an update for this, right?

fdncred · 2023-03-30T17:20:25Z

help -f

I definitely want highlighting and maybe a flag to turn it off. If we don't do it with find we'll need to do it manually by finding the index-of and str replace. It will probably be slow, but I'd prefer to have identical output of what we have now.

that's $nu that needs an update for this, right?

I'm not sure that's stored anywhere right now just like this. You'd have to get signatures then get syntax_shape | where parameter_type == input and something similar for output in order to construct something like exists now.

If we're trying to exactly match help commands we'll also have to add command_type.

amtoine · 2023-03-30T17:28:29Z

I definitely want highlighting and maybe a flag to turn it off. If we don't do it with find we'll need to do it manually by finding the index-of and str replace. It will probably be slow, but I'd prefer to have identical output of what we have now.

okey, i add that back 👍

I'm not sure that's stored anywhere right now just like this. You'd have to get signatures then get syntax_shape | where parameter_type == input and something similar for output in order to construct something like exists now.

mm we do not have this kind of output with the new help commands.
$nu.scope.commands has structured signatures, so the output is never like these lines!

If we're trying to exactly match help commands we'll also have to add command_type.

outside of this PR, right?

amtoine · 2023-03-30T17:34:35Z

@fdncred
highlighting added in 3305e8f with find $find 👍

Warning
now, any of the help command searches for the find pattern in ALL the columns of the selected $nu.scope.

fdncred · 2023-03-30T19:08:41Z

I think we need to document the differences between the stdlib help and rust help. I'm pushing to have them be identical in functionality, but others may disagree. In my mind, if they're not identical, then we have a regression in functionality.

Here's what I know is different so far.

Speed - We can land with having slight speed differences because we can always optimize that later.
help -f highlights too many columns in script form.
help commands has different columns (no command_type) column and the signatures column is different.

amtoine · 2023-03-31T16:43:44Z

@fdncred
there are also the points above that are not yet addressed

💡 LEGEND
🔵 an idea
🟡 -> 🟠 -> 🔴 from low to high issue

🔵 i currently print the exported environment blocks, see help help modules
🔵 i thought of adding a --raw option to allow custom data processing by not expanding the help page found, but instead return the raw data
🟡 with the builtin, help externs open returns the help open even though open is not an external => the new std help externs open does not
🔴 in the built-in help externs, we have all the registered flags that show up => as they are not available in $nu, i do not know how to do that

fdncred · 2023-03-31T17:03:02Z

ok, let's make this draft to show it's still a work in progress.

amtoine · 2023-03-31T17:14:08Z

ok, let's make this draft to show it's still a work in progress.

👍

kubouch · 2023-04-05T21:42:49Z

crates/nu-utils/standard_library/std.nu

+        let commands_string = (
+            $module.commands
+            | each {|command|
+                $"($command) (char lparen)(ansi cyan_bold)($module.name) ($command)(ansi reset)(char rparen)"


This one is not correct. I think it's better to drop the cyan version for now because I don't think there is a way to query it correctly from inside Nushell.

The bold cyan shows the actual name of the exported command inside the current scope which depends on how you import it. use std will have std prefix while use std * won't. Later, we might support renaming of modules which would lead to a completely different prefix. We'd probably need to expose the ID of commands inside $nu in order to be able to link the module's command with an existing symbol in the current scope.

should be good in 3c54ca6

kubouch · 2023-04-05T21:45:05Z

crates/nu-utils/standard_library/std.nu

+        print $"This module (ansi cyan)does not export(ansi reset) environment."
+    } else {
+        print $"This module (ansi cyan)exports(ansi reset) environment."
+        view source $module.env_block | nu-highlight


This one currently shows as a huge red blob. Prepending export-env before the block should fix it. Although I'm thinking if the export-env block is too big, it might be quite annoying (see https://github.com/pypa/virtualenv/blob/main/src/virtualenv/activation/nushell/activate.nu, for example), we can just leave it out as well. You can use view source module-name to view the source code if you want.

mm you are rigth

use crates/nu-utils/standard_library/std.nu 'help modules' help modules std

gives a red blob 🤔

i removed it for now in 1e14b46

this was causing "red blob" problems.

kubouch · 2023-04-06T18:51:17Z

Oops, sorry, I misclicked.

Looks good to me. It might need some touches but we can improve it as we go. For example, I personally wouldn't show the export-env block but that's a detail. Otherwise it looks really impressive! The help aliases are a bit different, but it isn't broken when querying a specific alias, that's good.

This adds the `help` commands from nushell#8505 and the `assert skip` from nushell#8748.

the `help` command needs to be there... 👀... thanks to nushell#8505

Related to #8505. # Description as #8505 has been landed, this PR fixes the prelude TODO and uses the `help` commands from the standard library in the prelude. # User-Facing Changes can now access `std help ...` to use the `help` implementations from the standard library # Tests + Formatting ``` $nothing ``` # After Submitting ``` $nothing ```

amtoine added 6 commits March 17, 2023 18:22

add tool functions to throw nice errors

703d81d

add a common tool function to print the "header" of the help

0ff89a4

By header i mean the green text that appears as a section name. e.g. the "search terms" in "Search terms: load, read, load_file, read_file" when running ``` help open | lines | get 2 ```

add the help modules implementation

ad36046

- `module-not-found-error` is the error message wrapper - `show-module` is the function that shows a single module doc - `help modules` is designed as a clone of the built-in `help modules`

implement help aliases and follow the same structure

a6398b9

implement help externs with the same structure

9619274

add doc with examples to help modules and help aliases

97aac7a

amtoine mentioned this pull request Mar 17, 2023

standard library: a few ideas of commands #8311

Open

14 tasks

amtoine added 5 commits March 18, 2023 13:42

implement the help operators command

55350c1

I've simply copy the table given by `help operators` and kept the same logic as before.

add the documentation to help externs

c4098df

implement help commands with empty show-command, same logic

cc31209

fix the --find descriptions for all commands

7bd9c5a

They do not sarch usages.

search in the search_terms for the commands

eee29f5

fdncred added the A:std-library Defining and improving the standard library written in Nu label Mar 18, 2023

amtoine added 15 commits March 18, 2023 16:14

implement show-command with raw "signatures" and "examples"

8d9df34

implement the signature and example sections of show-command

481fed6

This is almost a perfect copy-paste from the [`print_help.nu` gist from @jntrnr](https://gist.github.com/jntrnr/bb7f8827db14131ab95229e1c1efcc42).

print "(optional)" on optional params in the "Parameters" section

d0a7a14

add the flag types in "<>" when available

f81fd6e

simplify the signature loop by computing flags and positionals

2ebf5b2

use print-help-header for all section headers

c1e5be6

use char lparen instead of + "(" +, same with ")"

31b55d1

print the true signatures of the commands

656bfa4

remove the ...rest from the positionals

555070a

print {flags} in the usage if there are options

9076348

print the ...rest in case there is one

ad7f4c4

show the subcommands, if any, before the flags

5d5237f

fix the "Flags" caption, missing "s"

e83d8d3

add an extra newline at the end of the help

6cad0a9

Merge commit 'f9cf1d943' into feature/implement-help-commands-in-stan…

605c386

…dard-library This commit solves the merge conflict for nushell#8505. The conflict has been solved such that the new `help ...` commands are appended to `std.nu`.

amtoine marked this pull request as ready for review March 18, 2023 17:01

amtoine mentioned this pull request Mar 30, 2023

stdlib: add an automatic loading of "prelude" commands #8627

Merged

match any column with find to have highlights on hits

3305e8f

fdncred marked this pull request as draft March 31, 2023 17:02

amtoine marked this pull request as ready for review April 5, 2023 20:47

kubouch reviewed Apr 5, 2023

View reviewed changes

amtoine added 2 commits April 6, 2023 18:43

remove the coloring on used commands

3c54ca6

remove the highlight on env blocks for modules

1e14b46

this was causing "red blob" problems.

kubouch closed this Apr 6, 2023

kubouch reopened this Apr 6, 2023

kubouch merged commit d06ebb1 into nushell:main Apr 6, 2023

amtoine deleted the feature/implement-help-commands-in-standard-library branch April 7, 2023 15:50

amtoine mentioned this pull request Apr 7, 2023

add the standard help to the prelude as std help ... #8794

Merged

amtoine added a commit to amtoine/nushell that referenced this pull request Apr 7, 2023

Merge commit 'd06ebb168' into refactor/stdlib/move-the-crate

71a2aa3

This adds the `help` commands from nushell#8505 and the `assert skip` from nushell#8748.

amtoine added a commit to amtoine/nushell that referenced this pull request Apr 7, 2023

Merge commit 'd06ebb168' into feature/stdlib/help-commands-to-prelude

2582e6a

the `help` command needs to be there... 👀... thanks to nushell#8505

This was referenced Apr 8, 2023

stdlib: the tests do not run using the prelude #8809

Closed

bounty: divergences between built-in help and the implementations in nu-std #8813

Open

Conversation

amtoine commented Mar 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

progress

details

the error system

the actual help commands

User-Facing Changes

Tests + Formatting

After Submitting

Uh oh!

amtoine commented Mar 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

my main concerns

Uh oh!

amtoine commented Mar 29, 2023

Uh oh!

amtoine commented Mar 29, 2023

Uh oh!

fdncred commented Mar 29, 2023

Uh oh!

amtoine commented Mar 30, 2023

Uh oh!

fdncred commented Mar 30, 2023

Uh oh!

amtoine commented Mar 30, 2023

Uh oh!

amtoine commented Mar 30, 2023

Uh oh!

fdncred commented Mar 30, 2023

Uh oh!

amtoine commented Mar 31, 2023

Uh oh!

fdncred commented Mar 31, 2023

Uh oh!

amtoine commented Mar 31, 2023

Uh oh!

kubouch Apr 5, 2023

Choose a reason for hiding this comment

Uh oh!

amtoine Apr 6, 2023

Choose a reason for hiding this comment

Uh oh!

kubouch Apr 5, 2023

Choose a reason for hiding this comment

Uh oh!

amtoine Apr 6, 2023

Choose a reason for hiding this comment

Uh oh!

amtoine Apr 6, 2023

Choose a reason for hiding this comment

Uh oh!

kubouch commented Apr 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

amtoine commented Mar 17, 2023 •

edited

Loading

the actual `help` commands

amtoine commented Mar 18, 2023 •

edited

Loading

kubouch commented Apr 6, 2023 •

edited

Loading