Jupyter PlotWidget example by pijyoi · Pull Request #2055 · pyqtgraph/pyqtgraph

pijyoi · 2021-11-02T00:57:44Z

This PR is a continuation of #1963 (comment). Github does not allow attaching ipynb, so creating a PR seemed the easier way to continue the discussion.

A standalone Jupyter Notebook is in this PR that implements a Jupyter PlotWidget and example usage.
What works:

resizing
mouse events (panning / zooming)
running under WSL2. Set the environment variable QT_QPA_PLATFORM=offscreen before running jupyter-notebook

What doesn't work well:

mouse wheel zoom
- easy to accidentally trigger it while intending to scroll the notebook page.

What doesn't work:

context menu caused a crash so is disabled
- doesn't fit into the role of a notebook widget anyway

Eventually, the PlotWidget class could reside somewhere else within pyqtgraph codebase. But right now, the easiest way to try it out now is to just download the file and run it with your existing installation of pyqtgraph. (plus jupyter_rfb and pillow)

Some notes:

show()-ing the underlying GraphicsView widget causes a window to pop up (on Windows platform anyway). Fortunately, rendering to a QImage works fine without the show().
That it works without having to run the Qt event loop is nice. Note that the mouse events are sent synchronously using sendEvent

j9ac9k · 2021-11-02T02:43:30Z

I just tried this out, my goodness, this is amazing @pijyoi ... I had completely written off this functionality some time ago...

This works as intended in Chrome, in Safari none of the interactive bits were working, and I got the following error in the console:

RuntimeError: Cannot enter into task <Task pending name='Task-47' coro=<HTTP1ServerConnection._server_request_loop() running at /Users/ogi/.zinit/plugins/pyenv---pyenv/versions/3.9.7/envs/pyqtgraph-pyqt_62-jupyter-py39/lib/python3.9/site-packages/tornado/http1connection.py:823> wait_for=<Future finished result=b'GET /static...flate\r\n\r\n'> cb=[IOLoop.add_future.<locals>.<lambda>() at /Users/ogi/.zinit/plugins/pyenv---pyenv/versions/3.9.7/envs/pyqtgraph-pyqt_62-jupyter-py39/lib/python3.9/site-packages/tornado/ioloop.py:688]> while another task <Task pending name='Task-2' coro=<KernelManager._async_start_kernel() running at /Users/ogi/.zinit/plugins/pyenv---pyenv/versions/3.9.7/envs/pyqtgraph-pyqt_62-jupyter-py39/lib/python3.9/site-packages/jupyter_client/manager.py:337>> is being executed.
ERROR:asyncio:Exception in callback <TaskWakeupMethWrapper object at 0x105801a60>(<Future finis...late\r\n\r\n'>)
handle: <Handle <TaskWakeupMethWrapper object at 0x105801a60>(<Future finis...late\r\n\r\n'>)>
Traceback (most recent call last):
  File "/Users/ogi/.zinit/plugins/pyenv---pyenv/versions/3.9.7/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
RuntimeError: Cannot enter into task <Task pending name='Task-48' coro=<HTTP1ServerConnection._server_request_loop() running at /Users/ogi/.zinit/plugins/pyenv---pyenv/versions/3.9.7/envs/pyqtgraph-pyqt_62-jupyter-py39/lib/python3.9/site-packages/tornado/http1connection.py:823> wait_for=<Future finished result=b'GET /nbexte...flate\r\n\r\n'> cb=[IOLoop.add_future.<locals>.<lambda>() at /Users/ogi/.zinit/plugins/pyenv---pyenv/versions/3.9.7/envs/pyqtgraph-pyqt_62-jupyter-py39/lib/python3.9/site-packages/tornado/ioloop.py:688]> while another task <Task pending name='Task-2' coro=<KernelManager._async_start_kernel() running at /Users/ogi/.zinit/plugins/pyenv---pyenv/versions/3.9.7/envs/pyqtgraph-pyqt_62-jupyter-py39/lib/python3.9/site-packages/jupyter_client/manager.py:337>> is being executed.

More errors, I don't think this is an issue with our end of the code, so I'm mostly putting this for record keeping purposes.

pijyoi · 2021-11-02T02:47:54Z

I found somewhere and copied it down (but now I can't find the webpage again) that the following will workaround the asyncio error. But the error doesn't seem to prevent the notebook from working, so I just launch it the simple way now.

# avoid asyncio error
jupyter notebook --NotebookApp.kernel_manager_class=notebook.services.kernels.kernelmanager.AsyncMappingKernelManager

j9ac9k · 2021-11-02T05:13:53Z

nit-pick part 2, on macOS I'm getting the "bouncing icon" after I run the first cell...

This normally occurs when launching an application; I can try and investigate this one a bit more since I'm on this platform.

EDIT: a few minutes after I started this, it stopped...huh wonder what took so long.

pijyoi · 2021-11-03T07:06:04Z

mouse wheel zoom
easy to accidentally trigger it while intending to scroll the notebook page.

Seems like there is some web standard for mouse-wheel : https://developer.mozilla.org/en-US/docs/Web/API/WheelEvent

wheel alone --> scroll
wheel + CTRL --> zoom

pyqtgraph, as a Desktop application, doesn't consider the CTRL modifier for mouse wheel, which makes sense for it since GraphicsView (is a) Re-implementation of QGraphicsView that removes scrollbars

almarklein · 2021-11-03T09:35:17Z

pyqtgraph, as a Desktop application, doesn't consider the CTRL modifier for mouse wheel

I think most viz applications prefer to keep modifiers to themselves :) there are other ways though.

almarklein · 2021-11-05T00:02:59Z

pyqtgraph/jupyter/GraphicsView.py

+        elif event_type == "wheel":
+            pos = QtCore.QPointF(event["x"], event["y"])
+            pixdel = QtCore.QPoint()
+            scale = -1.0    # map JavaScript wheel to Qt wheel


From what I've seen, Qt uses steps of 120, versus 96 of JS, so using -120/96 == -1.25 here should be more precise.

I tried out with a mouse wheel. On Qt Desktop, each step generated a delta of 120. But on jupyter_rfb (on Windows Chrome), each step generated a delta of 200 (unscaled). But you are saying that you are getting 96 per step?

Yes, I get 96 on Windows Firefox. I assumed that this was more or less standard. Arg. I'll do some more research.

I got 100 on Chrome. I wonder why you get 200. Do you happen to be using a high-rez display?

Oh yes, I was on a laptop with dpr 2.0.

I suppose you "feel" the difference in scrolling too. What about using the touchpad, does it feel more sensitive too?

Oh yes, I was on a laptop with dpr 2.0.

Could you try on Firefox too please? I don't have access to my hidpi screen right now :)

On the same Windows laptop with dpr 2.0:
Chrome: 200
Firefox: 96

I made an issue, with a little test html to get some more details: vispy/jupyter_rfb#50

almarklein · 2021-11-05T00:06:50Z

pyqtgraph/jupyter/GraphicsView.py

+            inverted = False
+            evt = QtGui.QWheelEvent(pos, pos, pixdel, angdel, btns, mods, phase, inverted)
+            QtCore.QCoreApplication.sendEvent(self.gfxView.viewport(), evt)
+            self.request_draw()


Typically the event will be emitted into the application, and if the application reacts to it with changes that need a redraw, it will call request_draw. Then again, this should probably work too.

The implementation done here has jupyter_rfb using pyqtgraph as a library; rather than pyqtgraph using jupyter_rfb as a backend. i.e. pyqtgraph has no idea that it is being used in jupyter_notebook, and would not be able to call request_draw()

I did initially start off with a class inheriting from both pyqtgraph's GraphicsView and RemoteFrameBuffer; but got this error with PySide6: TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

pijyoi · 2021-11-05T10:03:57Z

Based on @almarklein 's comment on not calling request_draw() upon any mouse event, I connected 2 of ViewBox's signals (sigRangeChangedManually and sigStateChanged) to request_draw(). Having a "pointer_move" trigger a request_draw() doesn't sound correct.

There are some other signals (sigYRangeChanged, sigXRangeChanged, sigRangeChanged, sigTransformChanged). Not being familiar with ViewBox, I am not sure if they should be connected.
Should I connect them all just to be safe? Or should I only connect those known to be needed? EDIT: connected them anyway

pijyoi · 2021-11-05T12:27:04Z

Ok, so the downside with not requesting draw on any mouse event is that ViewBox RectMode no longer works since the frame is not redrawn to draw the rubber band.

almarklein · 2021-11-05T16:58:28Z

I don't know the internals of pyqtgraph, but does it no have an update/request_draw method or something that you can tap into? Requesting a new draw on user interaction can be a practical solution, but its not airtight, because there are other events (thinking mostly of timers for animations) that may require a redraw.

pijyoi · 2021-11-06T00:16:26Z

As far as I can tell, pyqtgraph internals is Qt. In Desktop Qt, update() is the equivalent of request_draw(), but is non-virtual, so we can't override it.

pijyoi · 2021-11-07T10:48:49Z

Ok, so the downside with not requesting draw on any mouse event is that ViewBox RectMode no longer works since the frame is not redrawn to draw the rubber band.

So I thought I had a working workaround for the above, but that only handles actions taken by the ViewBox. There are other graphicItems that accept clicks and hover events. (The included ScatterPlot example is one of them) The simplest solution for now is still to request_draw() on each pointer_* event.

pijyoi · 2021-11-08T01:37:56Z

Compared to the Desktop Qt version, the ScatterPlot example doesn't show tooltip on hover.

There are 2 main items that support clicks, PlotCurveItem and ScatterPlotItem. Clicks don't need special handling, the user, if they hook onto the click signal, could just call request_draw() within their callback.

ScatterPlotItem in addition supports hover and the library automatically creates a tooltip on hover, which is already not working in this rfb implementation.

If we reduce the scope of what we want to achieve in this implementation, we could remove the request_draw() on pointer_move.

j9ac9k · 2021-11-13T16:12:46Z

Hi @pijyoi

This looks good, I did run into one weird behavior, if you run the plotting notebook, and manipulate the linear region item in the 8th plot, the end result isn't quite right.

Also super nit-picky, but probably should remove the encoding pragma.

pijyoi · 2021-11-13T22:38:34Z

I added the packages that pyqtgraph adds in its CI, but the kernel still crashes upon instantiation of the QApplication. You can try it out here:
https://mybinder.org/v2/gh/pijyoi/pyqtgraph/jupyter_rfb?labpath=pyqtgraph%2Fexamples%2Fnotebooks

if you run the plotting notebook, and manipulate the linear region item in the 8th plot, the end result isn't quite right.

I don't see a difference with the Desktop version though.

j9ac9k · 2021-11-13T22:46:11Z

if you run the plotting notebook, and manipulate the linear region item in the 8th plot, the end result isn't quite right.

I don't see a difference with the Desktop version though.

I'll capture some video shortly.

pijyoi · 2021-11-13T22:46:18Z

okay, to get it to work on binder, I had to set

import os
os.environ["QT_QPA_PLATFORM"] = "offscreen"

pijyoi · 2021-11-13T23:32:09Z

Okay, the binder link above does work now!

j9ac9k · 2021-11-14T06:50:14Z

The LinearRegionItem behavior I experienced was only occurring in the Safari web browser (in Chrome everything behaves as it should), which makes me think this potentially a jupyter_rfb issue, not a pyqtgraph one. Being able to replicate the issue in binder is fantastic.

j9ac9k · 2021-11-14T06:54:38Z

I think the only thing this PR would need is to add a blurb to the README, and perhaps to the documentation somewhere, but given we don't reference jupyter anywhere in the docs, I'm not sure where the place to put it would be.

EDIT: also should we move this bit somewhere else a bit more obfuscated, such as in pyqtgraph/jupyter/GraphicsView.py ?

if "BINDER_SERVICE_HOST" in os.environ:
    os.environ["QT_QPA_PLATFORM"] = "offscreen"

pijyoi · 2021-11-14T07:01:53Z

Just a reminder, before any merge, I would need to point the binder configuration to fetch from pyqtgraph repo.

pijyoi · 2021-11-14T09:58:29Z

Okay, I can replicate the ROIExamples.ipynb being truncated on binder also when running on WSL2 with QT_QPA_PLATFORM=offscreen. So it probably has something to do with using offscreen.

After bumping up the css sizes, running on WSL2 no longer truncates; but it still gets truncated on binder. Although WSL2 was running PySide6 while binder was running PyQt5.

j9ac9k · 2021-11-14T22:33:59Z

@pijyoi do you want to see if this is a PyQt5 vs. PySide6 difference or you think we should just merge? FWIW this looks good to me.

pijyoi · 2021-11-14T23:25:27Z

Okay, ready to merging.

Moved binder badge location
Point to pyqtgraph repo
Restore CI workflow

j9ac9k · 2021-11-15T00:31:00Z

This LGTM, I would be interested in having a binder config that doesn't depend on conda (so we're not stuck w/ ancient versions of qt bindings); but given this works and a supported environment; let's leave it as is 👍🏻 ...I suppose notebook environments are more likely to use a conda distribution so probably for the best to replicate that environment.

pijyoi · 2021-11-15T14:34:52Z

do you want to see if this is a PyQt5 vs. PySide6 difference

I did some testing locally with WSL2. It's a Qt5 vs Qt6 difference.
As the provided container is Ubuntu 18.04, we cannot use Qt6.

j9ac9k · 2021-11-15T17:06:33Z

As the provided container is Ubuntu 18.04, we cannot use Qt6.

While I believe you, this really surprised me, and I double-checked the documentation, and sure enough, [Ubuntu 20.04 is the supported configuration](As the provided container is Ubuntu 18.04, we cannot use Qt6.)

pijyoi force-pushed the jupyter_rfb branch 3 times, most recently from bbbe891 to 41ae281 Compare November 2, 2021 01:25

almarklein mentioned this pull request Nov 2, 2021

Prevent scroll event while scrolling the page vispy/jupyter_rfb#47

Closed

almarklein reviewed Nov 5, 2021

View reviewed changes

pijyoi force-pushed the jupyter_rfb branch from 78fd380 to 09debd0 Compare November 6, 2021 09:43

pijyoi force-pushed the jupyter_rfb branch from 96841c6 to 1a68294 Compare November 12, 2021 02:02

pijyoi mentioned this pull request Nov 13, 2021

Added multi axis plots #1359

Open

13 tasks

pijyoi force-pushed the jupyter_rfb branch 2 times, most recently from 38a5474 to 67b94e9 Compare November 13, 2021 09:27

pijyoi marked this pull request as ready for review November 13, 2021 09:47

pijyoi force-pushed the jupyter_rfb branch from 1b000e8 to d87f898 Compare November 13, 2021 23:07

pijyoi force-pushed the jupyter_rfb branch from 1b19893 to df98a85 Compare November 14, 2021 09:09

pijyoi force-pushed the jupyter_rfb branch from 0577875 to 0b5506c Compare November 14, 2021 23:23

Jupyter RemoteFrameBuffer support

87e74d9

pijyoi force-pushed the jupyter_rfb branch from 0b5506c to 87e74d9 Compare November 14, 2021 23:26

j9ac9k merged commit 1b0465a into pyqtgraph:master Nov 15, 2021

pijyoi deleted the jupyter_rfb branch November 15, 2021 14:33

pijyoi mentioned this pull request Nov 15, 2021

change binder to requirements.txt #2104

Merged

j9ac9k mentioned this pull request Oct 1, 2022

Jupyterlab support #1963

Closed

Uh oh!

Conversation

pijyoi commented Nov 2, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

j9ac9k commented Nov 2, 2021

Uh oh!

pijyoi commented Nov 2, 2021

Uh oh!

j9ac9k commented Nov 2, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pijyoi commented Nov 3, 2021

Uh oh!

almarklein commented Nov 3, 2021

Uh oh!

almarklein Nov 5, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pijyoi commented Nov 5, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pijyoi commented Nov 5, 2021

Uh oh!

almarklein commented Nov 5, 2021

Uh oh!

pijyoi commented Nov 6, 2021

Uh oh!

pijyoi commented Nov 7, 2021

Uh oh!

pijyoi commented Nov 8, 2021

Uh oh!

j9ac9k commented Nov 13, 2021

Uh oh!

pijyoi commented Nov 13, 2021

Uh oh!

j9ac9k commented Nov 13, 2021

Uh oh!

pijyoi commented Nov 13, 2021

Uh oh!

pijyoi commented Nov 13, 2021

Uh oh!

j9ac9k commented Nov 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

j9ac9k commented Nov 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pijyoi commented Nov 14, 2021

Uh oh!

pijyoi commented Nov 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

j9ac9k commented Nov 14, 2021

Uh oh!

pijyoi commented Nov 14, 2021

Uh oh!

j9ac9k commented Nov 15, 2021

Uh oh!

pijyoi commented Nov 2, 2021 •

edited

Loading

j9ac9k commented Nov 2, 2021 •

edited

Loading

almarklein Nov 5, 2021 •

edited

Loading

pijyoi commented Nov 5, 2021 •

edited

Loading

j9ac9k commented Nov 14, 2021 •

edited

Loading

j9ac9k commented Nov 14, 2021 •

edited

Loading

pijyoi commented Nov 14, 2021 •

edited

Loading