ipykernel 6.x breaks redirecting stdout

This issue is related to several other issues that have been reported recently, including #789, #786, #735, #737 and https://github.com/psf/black/issues/2516 but they seem to have gone unanswered over the past few months in this issue tracker. They all revolve around the fact that upgrading from ipykernel 5.x to 6.x breaks users' ability to redirect output via stdout.

As a simple reproducible example:

import sys
from contextlib import redirect_stdout
terminal = sys.__stdout__
with redirect_stdout(terminal):
    print("test", file=terminal, flush=True)

With ipykernel 5.5.5 this code redirects print output to the computer’s terminal window from which JupyterLab was launched, rather than outputting it to the notebook in which the code was running. However, in ipykernel 6.x, this no longer works. Now the code above directs the output to both the terminal and the notebook. It seems new versions of ipykernel hijack stdout via the ipykernel.iostream.OutStream object and force output to the notebook, when you used to (and presumably should, since it's very useful) be able to output directly and only to the stdout of your choice, such as the terminal.

If I downgrade ipykernel to 5.5.5 (from say 6.4.1), the old behavior is back and I am able to control where stdout goes. However, staying on v5.x is obviously not a sustainable solution. Is there a different technique in 6.x for users who need to control where stdout goes while working in an ipython Jupyter notebook? Or is this a bug that should be fixed in future ipykernel releases?

See also this topic on Jupyter discourse and the related issues linked above from @uldz @MarcoGorelli @ItamarShDev.

While we work this out, you can disable the behavior in your ipython_config.py to restore the 5.x behavior (aadded in #752):

# ipython_config.py
c.IPKernelApp.capture_fd_output = False

Thanks @minrk. It looks like presently there's no API to configure this programmatically? This creates a challenge for developing a package that's cluttering the notebook with log output which previously was all divertible to the terminal window.

@minrk a little more troubleshooting detail. This issue occurs on all of my Linux machines. But, this issue does not occur on my Windows machine: all output can be diverted to the terminal window. However, if I run a notebook in a Linux Docker container on my windows machine this issue occurs.

So it seems there is cross-platform inconsistency. On Windows I'm still able to control where stdout gets directed, like always, but on Linux that ability has been broken.

There's not an API to turn it on and off, but looking at the code, this snippet will disable it permanently at runtime:

for std in (sys.stdout, sys.stderr):
    if hasattr(std, "_should_watch"):
        std._should_watch = False  # signal that fd-watching should stop via private API

Turning it on and off via a public API might be tricky, but should be doable. wurlitzer can do this.

So it seems there is cross-platform inconsistency.

I believe the pipe-based low-level capture only occurs on posix systems, which is why Windows would not be affected.

Unfortunately using that private _should_watch attribute doesn't seem to get the output to the terminal window either:

import sys
from contextlib import redirect_stdout

terminal = sys.__stdout__
print('test1')  # this prints to the notebook

if hasattr(sys.stdout, "_should_watch"):
    sys.stdout._should_watch = False

with redirect_stdout(terminal):
    print("test2", file=terminal, flush=True)  # this doesn't print anywhere

print('test3')  # this prints to the notebook

My mistake, that only stops reading the pipe, it doesn't restore the original FD. This should do it:

import os
import sys

for std, __std__ in [
    (sys.stdout, sys.__stdout__),
    (sys.stderr, sys.__stderr__),
]:
    if getattr(std, "_original_stdstream_copy", None) is not None:
        # redirect captured pipe back to original FD
        os.dup2(std._original_stdstream_copy, __std__.fileno())
        std._original_stdstream_copy = None

Thanks, that seems to do it.