Skip to content

Disable pipeline echo#8292

Merged
sophiajt merged 10 commits intonushell:mainfrom
NotLebedev:disable-pipeline-echo
Mar 16, 2023
Merged

Disable pipeline echo#8292
sophiajt merged 10 commits intonushell:mainfrom
NotLebedev:disable-pipeline-echo

Conversation

@NotLebedev
Copy link
Copy Markdown
Contributor

@NotLebedev NotLebedev commented Mar 2, 2023

Description

Change behavior of block evaluation to not print result of intermediate commands.
Previously result of every but last pipeline in a block was printed to stdout, and last one was returned
image
With this change results of intermediate pipelines are discarded after they finish and the last one is returned as before:
image
Now one should use print explicitly to print something to stdout
image

Note, that this behavior is not limited to functions! The scope of this change are all blocks. All of the below are executed as blocks and thus exibited this behavior in the same way:
image

With this change outputs for all types of blocks are cleaned:
image

User-Facing Changes

All types of blocks (function bodies, closures, if branches, for and loop bodies e.t.c.) no longer print result of intermediate pipelines.

Tests + Formatting

Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

  • cargo fmt --all -- --check to check standard code formatting (cargo fmt --all applies these changes)
  • cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect to check that you're using the standard code style
  • cargo test --workspace to check that all tests pass

After Submitting

If your PR had any user-facing changes, update the documentation after the PR is merged, if necessary. This will help us keep the docs up to date.

@fdncred
Copy link
Copy Markdown
Contributor

fdncred commented Mar 2, 2023

The first casualty. /cc @jntrnr

At the end of this script it calls several custom commands. I think they should all still print output but only the last one does. This is how the end of the script looks

print_run 0 16 # The first 16 colours are spread over the whole spectrum
print ""             # Single line
print_blocks 16 123 6 6 3 # 6x6x6 colour cube between 16 and 123 inclusive
print_blocks 124 231 6 6 3 # 6x6x6 colour cube between 124 and 231 inclusive
print_blocks 232 255 12 2 1 # Not 50, but 24 Shades of Grey

The new output
image
The old output
image


Fixed by changing the last lines to this:

[
  (print_run 0 16) # The first 16 colours are spread over the whole spectrum
  (print "")             # Single line
  (print_blocks 16 123 6 6 3) # 6x6x6 colour cube between 16 and 123 inclusive
  (print_blocks 124 231 6 6 3) # 6x6x6 colour cube between 124 and 231 inclusive
  (print_blocks 232 255 12 2 1) # Not 50, but 24 Shades of Grey
] | str join (char nl)

@fdncred
Copy link
Copy Markdown
Contributor

fdncred commented Mar 2, 2023

2nd casualty. turtle.nu
This just doesn't look right. Maybe we need to capture the output in variables?

source turtle.nu
Turtle 1 = Turtle 2 = Turtle 3 = Turtle 4 = Turtle 5 = 2936

The script

# The infamous turtle benchmark
def turtle [column: string] {
    wrap $column | table
}

print -n "Turtle 1 = "
"turtle" | turtle 1 | str length
print -n "Turtle 2 = "
"turtle" | turtle 1 | turtle 2 | str length
print -n "Turtle 3 = "
"turtle" | turtle 1 | turtle 2 | turtle 3 | str length
print -n "Turtle 4 = "
"turtle" | turtle 1 | turtle 2 | turtle 3 | turtle 4 | str length
print -n "Turtle 5 = "
"turtle" | turtle 1 | turtle 2 | turtle 3 | turtle 4 | turtle 5 | str length

Fixed by changing the end part to this

[
    (print $"Turtle 1 = ('turtle' | turtle 1 | str length)")
    (print $"Turtle 2 = ('turtle' | turtle 1 | turtle 2 | str length)")
    (print $"Turtle 3 = ('turtle' | turtle 1 | turtle 2 | turtle 3 | str length)")
    (print $"Turtle 4 = ('turtle' | turtle 1 | turtle 2 | turtle 3 | turtle 4 | str length)")
    (print $"Turtle 5 = ('turtle' | turtle 1 | turtle 2 | turtle 3 | turtle 4 | turtle 5 | str length)")
] | str join

@fdncred
Copy link
Copy Markdown
Contributor

fdncred commented Mar 2, 2023

And 3rd - my poor friend gradient.nu. The prime benchmark that started the craze.
There's no output at all. :(

I'm not sure what to do with gradient.nu because it wants to draw each row at a time. I actually would rather continually output on all the scripts versus collecting everything at the end. I don't think that's a good solution. We should be able to output iteratively.

Now that I think about it. I have some progress bar scripts that probably won't work either now.

@sophiajt
Copy link
Copy Markdown
Contributor

sophiajt commented Mar 2, 2023

@fdncred - what is print doing I wonder. Each print should go directly to the screen 🤔

Can you try this too:

print "hello"
print "world"

If you don't see both, I think we don't have it, yet.

@sophiajt
Copy link
Copy Markdown
Contributor

sophiajt commented Mar 2, 2023

2nd casualty. turtle.nu This just doesn't look right. Maybe we need to capture the output in variables?

source turtle.nu
Turtle 1 = Turtle 2 = Turtle 3 = Turtle 4 = Turtle 5 = 2936

The script

# The infamous turtle benchmark
def turtle [column: string] {
    wrap $column | table
}

print -n "Turtle 1 = "
"turtle" | turtle 1 | str length
print -n "Turtle 2 = "
"turtle" | turtle 1 | turtle 2 | str length
print -n "Turtle 3 = "
"turtle" | turtle 1 | turtle 2 | turtle 3 | str length
print -n "Turtle 4 = "
"turtle" | turtle 1 | turtle 2 | turtle 3 | turtle 4 | str length
print -n "Turtle 5 = "
"turtle" | turtle 1 | turtle 2 | turtle 3 | turtle 4 | turtle 5 | str length

Fixed by changing the end part to this

[
    (print $"Turtle 1 = ('turtle' | turtle 1 | str length)")
    (print $"Turtle 2 = ('turtle' | turtle 1 | turtle 2 | str length)")
    (print $"Turtle 3 = ('turtle' | turtle 1 | turtle 2 | turtle 3 | str length)")
    (print $"Turtle 4 = ('turtle' | turtle 1 | turtle 2 | turtle 3 | turtle 4 | str length)")
    (print $"Turtle 5 = ('turtle' | turtle 1 | turtle 2 | turtle 3 | turtle 4 | turtle 5 | str length)")
] | str join

This one is correct now and needs print for the output of each. You could probably make them 'turtle' | turtle 1 | str length | print $in

@fdncred
Copy link
Copy Markdown
Contributor

fdncred commented Mar 2, 2023

@jntrnr I did this on turtle and it works

print $"Turtle 1 = ('turtle' | turtle 1 | str length)"
print $"Turtle 2 = ('turtle' | turtle 1 | turtle 2 | str length)"
print $"Turtle 3 = ('turtle' | turtle 1 | turtle 2 | turtle 3 | str length)"
print $"Turtle 4 = ('turtle' | turtle 1 | turtle 2 | turtle 3 | turtle 4 | str length)"
print $"Turtle 5 = ('turtle' | turtle 1 | turtle 2 | turtle 3 | turtle 4 | turtle 5 | str length)"

and this on 256_color_testpattern and it works too

print (print_run 0 16) # The first 16 colours are spread over the whole spectrum
print ""             # Single line
print (print_blocks 16 123 6 6 3) # 6x6x6 colour cube between 16 and 123 inclusive
print (print_blocks 124 231 6 6 3) # 6x6x6 colour cube between 124 and 231 inclusive
print (print_blocks 232 255 12 2 1) # Not 50, but 24 Shades of Grey

and I changed thie one line in gradient and it seems to have fixed it
old line

    $"($row_data)(char newline)"

new line

    print -n $"($row_data)(char newline)"

sophiajt added a commit that referenced this pull request Mar 3, 2023
# Description

Have `print` print it's input, so it's easier to print a pipeline (esp
after we land #8292 and related)

# User-Facing Changes

`print` will now print its input, if there are no args given

# Tests + Formatting

Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass

# After Submitting

If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 4, 2023

Codecov Report

Merging #8292 (8e1e761) into main (8c487ed) will decrease coverage by 0.10%.
The diff coverage is 60.60%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #8292      +/-   ##
==========================================
- Coverage   68.05%   67.96%   -0.10%     
==========================================
  Files         621      621              
  Lines       99938    99923      -15     
==========================================
- Hits        68015    67911     -104     
- Misses      31923    32012      +89     
Impacted Files Coverage Δ
crates/nu-protocol/src/value/stream.rs 80.00% <35.29%> (-5.14%) ⬇️
crates/nu-protocol/src/pipeline_data.rs 76.92% <85.71%> (+0.19%) ⬆️
crates/nu-engine/src/eval.rs 74.38% <100.00%> (-0.91%) ⬇️

... and 6 files with indirect coverage changes

@NotLebedev
Copy link
Copy Markdown
Contributor Author

Fixed everything except this python-virtualenv tests. Can't quite figure out what the problem is there. Output is not helpfull at all

@sholderbach sholderbach added the notes:breaking-changes This PR implies a change affecting users and has to be noted in the release notes label Mar 5, 2023
@NotLebedev
Copy link
Copy Markdown
Contributor Author

Fixed everything except this python-virtualenv tests. Can't quite figure out what the problem is there. Output is not helpfull at all

Identified problem. To test prompt that shell is behaving correctly virtualenv tests is running a script created like this:

def _get_test_lines(self, activate_script):
        commands = [
            self.print_python_exe(),
            self.print_os_env_var("VIRTUAL_ENV"),
            self.activate_call(activate_script),
            self.print_python_exe(),
            self.print_os_env_var("VIRTUAL_ENV"),
            self.print_prompt(),
            # \\ loads documentation from the virtualenv site packages
            self.pydoc_call,
            self.deactivate,
            self.print_python_exe(),
            self.print_os_env_var("VIRTUAL_ENV"),
            "",  # just finish with an empty new line
        ]
        return commands

Which results in following nushell script:

python.EXE -c 'import sys; print(sys.executable)'
python.EXE -c 'import os; import sys; v = os.environ.get("VIRTUAL_ENV"); print(v)'
overlay use 'C:\Users\Artemiy\AppData\Local\Temp\pytest-of-unknown\pytest-3\activation-tester-env0\e-$ -j\Scripts\activate.nu'
python.EXE -c 'import sys; print(sys.executable)'
python.EXE -c 'import os; import sys; v = os.environ.get("VIRTUAL_ENV"); print(v)'
$env.VIRTUAL_PROMPT
pydoc -w pydoc_test
deactivate
python.EXE -c 'import sys; print(sys.executable)'
python.EXE -c 'import os; import sys; v = os.environ.get("VIRTUAL_ENV"); print(v)'

The problem here is that everything is printing to stdout except $env.VIRTUAL_PROMPT which by old rules is a separate pipeline that is echoed. But by new rules it does nothing. To fix this a small change is needed in test_nushell.py, change print_prompt function to

def print_prompt(self):
      return r"print $env.VIRTUAL_PROMPT"

to print that output.

I think I will need to PR to virtualenv repository to resolve this issue.

@kubouch
Copy link
Copy Markdown
Contributor

kubouch commented Mar 7, 2023

I haven't followed this PR in detail, but you can make PR to virtualenv. Regardless of this PR getting merged or not, adding print there shouldn't hurt anything.

I personally support not auto-printing intermediate pipelines, only the last one.

Btw, what happens if you have nested blocks? Like do { do { 'spam' }; 'bar' }.

@NotLebedev
Copy link
Copy Markdown
Contributor Author

I haven't followed this PR in detail, but you can make PR to virtualenv. Regardless of this PR getting merged or not, adding print there shouldn't hurt anything.

Will do!

I personally support not auto-printing intermediate pipelines, only the last one.

Btw, what happens if you have nested blocks? Like do { do { 'spam' }; 'bar' }.

Inner do will return 'spam'. Outer will execute inner do, get return value and discard it. Then it will return 'bar':
image

@kubouch
Copy link
Copy Markdown
Contributor

kubouch commented Mar 7, 2023

Cool, to me, this behavior seems good 👍

For the virtualenv PR, you'll need to add a changelog entry as well. You can check previous PRs in the virtualenv repo done by me, or one open just now by WindSoilder, for reference.

@sophiajt
Copy link
Copy Markdown
Contributor

I think we're ready to try again with this PR. Can you fix the conflicts and then we'll let CI run again and see if it goes through.

@NotLebedev
Copy link
Copy Markdown
Contributor Author

I think we're ready to try again with this PR. Can you fix the conflicts and then we'll let CI run again and see if it goes through.

Done. I think it is ready for merge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

notes:breaking-changes This PR implies a change affecting users and has to be noted in the release notes status:wait-until-after-nushell-release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants