Skip to content

dynamic range limiting in PlotDataItem#1140

Merged
j9ac9k merged 13 commits intopyqtgraph:masterfrom
NilsNemitz:dynamic_range_limit
Oct 19, 2020
Merged

dynamic range limiting in PlotDataItem#1140
j9ac9k merged 13 commits intopyqtgraph:masterfrom
NilsNemitz:dynamic_range_limit

Conversation

@NilsNemitz
Copy link
Copy Markdown
Contributor

Workaround for issue #31
PlotCurveItem disappears with large ViewBox scale factor

(Code editor ate trailing whitespace, sorry.)

Test code:

from PyQt5 import QtWidgets
import sys
import pyqtgraph as pg

class MainWindow(QtWidgets.QMainWindow):
    def __init__(self, *args, **kwargs):
        super(MainWindow, self).__init__(*args, **kwargs)
        self.graphWidget = pg.PlotWidget()
        self.setCentralWidget(self.graphWidget)

        y_data_a = [  10,  10, -1,  1,  0,  10,  10]
        y_data_b = [ 2e9, 2e9,  0,  1, -1, 2e9, 2e9]
        y_data_c = [-2e9,-2e9,  0, -1,  1,-2e9,-2e9]
        x_data = [idx for idx, val in enumerate(y_data_a)]
        self.graphWidget.setYRange( -1.1, 1.1)
        item0 = pg.PlotDataItem( x=x_data, y=y_data_a, pen='g')
        item1 = pg.PlotDataItem( x=x_data, y=y_data_b, pen='r')
        item2 = pg.PlotDataItem( x=x_data, y=y_data_c, pen='m')

        disable = False
        for item in [item0, item1, item2]:
            self.graphWidget.addItem( item )
            if disable: item.setLimitDynamicRange(False)

def main():
    app = QtWidgets.QApplication(sys.argv)
    main = MainWindow()
    main.show()
    sys.exit(app.exec_())

if __name__ == '__main__':
    main()

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Mar 10, 2020

Hi @NilsNemitz thanks for the PR!

A quick question, is self.opts['limitDynamicRange'] intended to just take a boolean? if so does the method setLimitDynamicRange take a boolean as the input?

    def setLimitDynamicRange(self, limit):
        if self.opts['limitDynamicRange'] == limit:
            return
        self.opts['limitDynamicRange'] = limit
        self.xDisp = self.yDisp = None
        self.updateItems()

If so, can you replace the == with the is operator?

Also, can you add a docstring on that method?

Thanks for the PR!

@NilsNemitz
Copy link
Copy Markdown
Contributor Author

Hi @j9ac9k !
I am taking your quick reply as encouragement and have updated pull request #1140 .

  1. Operator is changed
  2. docstring is added to what is now "setDynamicRangeLimit()" and to the _init_() function's argument parsing.
  3. setDynamicRangeLimit(limit) now takes an argument that exposes the actual limit to the user.
  4. added some checks to bypass clipping the data in the 99% of cases that do not need it.
    For the last point, I have added a function dataRect() parallelling the existing viewRect(). This stores the range of the data and is only calculated once per data set (cleared in setData()). The functionality is a little too close to dataBounds(), but that relies on functionality of the scatter and curve children... which see the data only after clipping.
    I hope we can find a solution with minimal computational overhead and enable the workaround to the disappearing plots by default. Those who seek ultimate performance will probably call ScatterPlotItem and PlotCurveItem directly.
    There is still some improvement available by modifying viewRangeChanged() to not invalidate xDisp and yDisp unless necessary, but that is a larger change than I want to propose right now.
    Let me know what else I can do :)

Extended testing code:

from PyQt5 import QtWidgets
import sys
import pyqtgraph as pg

class MainWindow(QtWidgets.QMainWindow):
    def __init__(self, *args, **kwargs):
        super(MainWindow, self).__init__(*args, **kwargs)
        self.graphWidget = pg.PlotWidget()
        self.setCentralWidget(self.graphWidget)

        y_data_a = [  10,  10, -1,  1,  0,  10,  10]
        y_data_b = [ 2e9, 2e9,  0,  1, -1, 2e9, 2e9]
        y_data_c = [-2e9,-2e9,  0, -1,  1,-2e9,-2e9]
        x_data = [idx for idx, val in enumerate(y_data_a)]
        self.graphWidget.setYRange( -1.1, 1.1)
        item0 = pg.PlotDataItem( x=x_data, y=y_data_a, pen='g')
        item1 = pg.PlotDataItem( x=x_data, y=y_data_b, pen='r')
        item2 = pg.PlotDataItem( x=x_data, y=y_data_c, pen='m')
        # white curve demonstrates bug and disappears with viewport change
        # also demonstrates magic argument parsing
        item3 = pg.PlotDataItem( x=x_data, y=y_data_c, pen='w', dynamicRangeLimit=None)

        disable = False
        reconfigure = False
        for item in [item0, item1, item2, item3]:
            self.graphWidget.addItem( item )
            if disable:
                item.setDynamicRangeLimit(None) # curves will disappear with viewport change
            if reconfigure:
                item.setDynamicRangeLimit(2) # does not make any sense, but shows up as visible distortion
                item.setDynamicRangeLimit(2) # check that repeated setting works

def main():
    app = QtWidgets.QApplication(sys.argv)
    main = MainWindow()
    main.show()
    sys.exit(app.exec_())

if __name__ == '__main__':
    main()

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Mar 12, 2020

my activity here is indeed feast or famine with nothing in between, I know that's totally not fair to contributors like yourself, and for that I apologize... just not sure what else I can do on a volunteer effort.

Anyway, back to the PR!

Looks like the CI system doesn't like your changes for the python2 builds:

https://dev.azure.com/pyqtgraph/pyqtgraph/_build/results?buildId=647

SyntaxError: Non-ASCII character '\xc2' in file pyqtgraph\graphicsItems\PlotDataItem.py on line 357, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

@NilsNemitz
Copy link
Copy Markdown
Contributor Author

Your work is very appreciated! I did not expect to get such a quick reply.
Sorry about the errors. I am not familiar with the CI system and it took me too long to get to the actual error. "bash exited with code 1" is not very helpful by itself.
Thank you for the hint. All checks pass now. Python 2 does not share my love for unicode.

at any time.
dynamicRangeLimit (float or None) Limit off-screen positions of data points at large
magnification to avoids display errors. Disabled if None.
=============== =============================================================
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like you need to indent this line over another 4 spaces (as well as the next couple of lines)

limit (float or None) Maximum allowed vertical distance of plotted points in units of viewport height.
'None' disables the check for a minimal increase in performance. Default is 1E+06.
"""
if self.opts['dynamicRangeLimit'] is limit:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be checking if None?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe it should be if self.opts['dynamicRangeLimit'] is None or limit == self.opts['dynamicRangeLimit']?

if data_range is not None:
view_height = view_range.height()
lim = self.opts['dynamicRangeLimit']
# print('data:', data_range.height(), ' view:', view_height, ' limit:', lim )
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove commented out print statements.


def dataRect(self):
"""
Returns a bounding rectangle (as QRectF) for the full set of data
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this function can return None, that should be indicated in the docstring (as well as a description of why that might be the case)

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Mar 13, 2020

Thanks for the update!

Azure Pipelines isn't the most UI friendly interface, I think if you click on the "bash exited with code 1" it takes you to the portion...

I made some inline comments, I could be off-base on some of them, if so, let me know.

@NilsNemitz
Copy link
Copy Markdown
Contributor Author

NilsNemitz commented Mar 13, 2020

Thank you for the helpful comments @j9ac9k !
I have applied your suggestions to PR #1140

One additional change: I have moved the limiting code after downsampling (instead of before).
a) Shifting even far-offscreen points might give visible effects with downsampling in mean mode.
b) limiting is faster after downsampling reduces the number of datapoints.

Two observations:
a) Should self.xClean and self.yClean be removed? I ran a full-text search over the entire pyqtgraph folder, and the only hits were three lines in PlotDataItem.py where they get set to None. I think they are never used, never set to an actual value.
b) Is it intentional that PlotItem.addItem() (in line 507) calls PlotDataItem.setDownsampling() and overwrites any earlier setting? It seems that this makes the documented way of setting downsampling through _init_()\ parameters of PlotDataItem quite useless. In my tests, any downsampling I set before the .additem() call gets reverted to the default of "no downsampling". The same is probably true for ClipToView, FFT, Alpha, and PointMode (but I did not test that.) Is there a use case that does not involve adding to some variant of PlotItem?

return self._dataRect
if self.xData is None or self.yData is None:
return None
ymin = np.nanmin(self.yData)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should catch the runtime warning being emitted here as we account for the nan case just below.

This can be done with the following code

import warnings

...

with warnings.catch_warnings():
    warnings.simplefilter("ignore")
    np.nanmin(self.yData)

@j9ac9k j9ac9k changed the base branch from develop to master October 15, 2020 04:03
@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Oct 15, 2020

Hi @NilsNemitz sorry it's taken way too long for me to follow up. I changed the base of where this PR should be pointed at, resolved a merge conflict, and added one more comment... enough time has passed by I would understand if you don't want to implement that last suggestion, and I don't mind implementing it myself.

Thanks again for this PR!

@NilsNemitz
Copy link
Copy Markdown
Contributor Author

Dear @j9ac9k ,
I wish I was more fluid with the collaborative aspects of git, but I think I have managed to update the pull request to suppress the unneeded "all-NaN" warnings. Sorry for the persistent "trailing whitespace" changes. I fixed my IDE now, but the damage in PlotDataItem.py is already done. :(

I tried my previous test cases; The changes still seem to fix the issue of disappearing plots.

I tested with all-NaN x-data, y-data, and both. No problem was observed. I still got an "all-NaN" warning from the code in PlotCurveItem.py. Since there is only a single line with a np.nanmin / np.nanmax call, I suppressed the warning there, too.

Now the code is silent for an all-NaN (curve) plot. I suspect that scatter plots will still throw a warning, but I did not want to touch more files than necessary for now.

Let me know if you have more suggestions. Any idea why the "pre_build build_docs" test is failing?

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Oct 18, 2020

Hey @NilsNemitz thanks for following up; again apologize for the long time that elapsed.

Regarding the docs the error message is this:

/home/vsts/work/1/s/doc/source/../../pyqtgraph/graphicsItems/PlotDataItem.py:docstring of pyqtgraph.PlotDataItem.setDynamicRangeLimit:6:Unexpected indentation.

I'm not that familiar w/ sphinx, but I can try and debug this for you as I have my local machine setup to build documentation and it's much easier to debug locally than on the CI level. I'll hop onto your branch, and see if I can figure out the necessary modification to the docstring to make sphinx happy.

@NilsNemitz
Copy link
Copy Markdown
Contributor Author

@j9ac9k: Let's see if this change fixes it, otherwise it would be great if you could have a look. Thank you for your time!

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Oct 19, 2020

LGTM, thanks for the PR @NilsNemitz merging!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants