Skip to content

Add pipeline span to metadata#16014

Merged
sholderbach merged 4 commits intonushell:mainfrom
132ikl:metadata-span
Jun 30, 2025
Merged

Add pipeline span to metadata#16014
sholderbach merged 4 commits intonushell:mainfrom
132ikl:metadata-span

Conversation

@132ikl
Copy link
Copy Markdown
Member

@132ikl 132ikl commented Jun 20, 2025

Description

This PR makes the span of a pipeline accessible through metadata, meaning it's possible to get the span of a pipeline without collecting it.

Examples:

ls | metadata
# => ╭────────┬────────────────────╮
# => │        │ ╭───────┬────────╮ │
# => │ span   │ │ start │ 170218 │ │
# => │        │ │ end   │ 170220 │ │
# => │        │ ╰───────┴────────╯ │
# => │ source │ ls                 │
# => ╰────────┴────────────────────╯
ls | metadata access {|meta|
  error make {msg: "error", label: {text: "here", span: $meta.span}}
}
# => Error:   × error
# =>    ╭─[entry #7:1:1]
# =>  1 │ ls | metadata access {|meta|
# =>    · ─┬
# =>    ·  ╰── here
# =>  2 │   error make {msg: "error", label: {text: "here", span: $meta.span}}
# =>    ╰────

Here's an example that wouldn't be possible before, since you would have to use metadata $in to get the span, collecting the (infinite) stream

generate {|x=0| {out: 0, next: 0} } | metadata access {|meta|
  # do whatever with stream
  error make {msg: "error", label: {text: "here", span: $meta.span}}
}
# => Error:   × error
# =>    ╭─[entry #16:1:1]
# =>  1 │ generate {|x=0| {out: 0, next: 0} } | metadata access {|meta|
# =>    · ────┬───
# =>    ·     ╰── here
# =>  2 │   # do whatever with stream
# =>    ╰────

I haven't done the tests or anything yet since I'm not sure how we feel about having this as part of the normal metadata, rather than a new command like metadata span or something. We could also have a metadata access like functionality for that with an optional closure argument potentially.

User-Facing Changes

  • The span of a pipeline is now available through metadata and metadata access without collecting a stream.

Tests + Formatting

TODO

After Submitting

N/A

@sholderbach
Copy link
Copy Markdown
Member

Ngl feels like exposing a dirty implementation detail of the nu engine...
...but we set the bar so high what we expect from an error message, so the people should get nice things

@132ikl
Copy link
Copy Markdown
Member Author

132ikl commented Jun 20, 2025

yea, this was motivated by me struggling to get a good error message in a command I would prefer to add to std than be a built-in

@Bahex
Copy link
Copy Markdown
Member

Bahex commented Jun 22, 2025

metadata access is a very useful command, but since shortly after adding it I've felt its scope was limited.

span makes sense as a part of metadata, but I think we can expand to a pipeline inspect(🚲🛖) command that:

  • provides metadata like metadata access
  • provides type information similar to describe --detailed --no-collect, like whether the input is a stream or a value, if it's a stream is it a byte, string, or a list stream
  • and for list streams a --peek n option that also provides the first n elements without disrupting the stream (this probably can be a separate command)

fdncred pushed a commit that referenced this pull request Jun 23, 2025
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

This PR adds the `only` command to `std-rfc/iter`, which is a command I
wrote a while ago that I've found so useful that I think it could have a
place in the standard library. It acts similarly to `get 0`, but ensures
that the value actually exists, and there aren't additional values. I
find this most useful when chained with `where`, when you want to be
certain that no additional elements are accidentally selected when you
only mean to get a single element.

I'll copy the help page here for additional explanation:

> Get the only element of a list or table, ensuring it exists and there
are no extra elements.
> 
> Similar to `first` with no arguments, but errors if there are no
additional
> items when there should only be one item. This can help avoid issues
when more
> than one row than expected matches some criteria.
> 
> This command is useful when chained with `where` to ensure that only
one row
> meets the given condition.
> 
> If a cell path is provided as an argument, it will be accessed after
the first
> element. For example, `only foo` is roughly equivalent to `get 0.foo`,
with
> the guarantee that there are no additional elements.
> 
> Note that this command currently collects streams.

> Examples:
>  
> Get the only item in a list, ensuring it exists and there's no
additional items
> ```nushell
> [5] | only
> # => 5
> ```
> 
> Get the `name` column of the only row in a table
> ```nushell
> [{name: foo, id: 5}] | only name
> # => foo
> ```
> 
> Get the modification time of the file named foo.txt
> ```nushell
> ls | where name == "foo.txt" | only modified
> ```

Here's some additional examples showing the errors:

![image](https://github.com/user-attachments/assets/d5e6f202-db52-42e4-a2ba-fb7c4f1d530a)


![image](https://github.com/user-attachments/assets/b080da2a-7aff-48a9-a523-55c638fdcce3)

Most of the time I chain this with a simple `where`, but here's a couple
other real world examples of how I've used this:

[With `parse`, which outputs a
table](https://git.ikl.sh/132ikl/dotfiles/src/branch/main/.scripts/manage-nu#L53):
```nushell
let commit = $selection | parse "{start}.g{commit}-{end}" | only commit
```

[Ensuring that only one row in a table has a name that ends with a
certain
suffix](https://git.ikl.sh/132ikl/dotfiles/src/branch/main/.scripts/btconnect):
```nushell
$devices | where ($chosen_name ends-with $it.name) | only
```


Unfortunately to get these nice errors I had to collect the stream (and
I think the errors are more useful for this). This should be to be
mitigated with (something like) #16014.


Putting this in `std/iter` might be pushing it, but it seems *just*
close enough that I can't really justify putting it in a different/new
module.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
* Adds the `only` command to `std-rfc/iter`, which can be used to ensure
that a table or list only has a single element.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

Added a few tests for `only` including error cases

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
N/A

---------

Co-authored-by: Bahex <Bahex@users.noreply.github.com>
@132ikl
Copy link
Copy Markdown
Member Author

132ikl commented Jun 25, 2025

@Bahex what do you think the way forward for this idea should be? do you think we should go forward with this implementation or try to integrate it with a "pipeline inspect" type command?

@Bahex
Copy link
Copy Markdown
Member

Bahex commented Jun 25, 2025

@132ikl It's fine to move forward with this, if/when we add a pipeline inspect metadata access would be deprecated anyway

@132ikl
Copy link
Copy Markdown
Member Author

132ikl commented Jun 25, 2025

okay should be good now

edit: ok apparently not lol

@132ikl
Copy link
Copy Markdown
Member Author

132ikl commented Jun 27, 2025

i don't exactly understand why these tests use eval_pipeline_without_terminal_expression (or really why any tests use that?) but they pass now 🙏

Copy link
Copy Markdown
Member

@sholderbach sholderbach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weird tests indeed

Looks good to me, not sure if it clashes with #16049 if we merge this one first

@sholderbach sholderbach merged commit f4136aa into nushell:main Jun 30, 2025
16 checks passed
@github-actions github-actions bot added this to the v0.106.0 milestone Jun 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants