EDIT: on second thought, this NaN check might not be buying us anything... doesn't look like arrayToQPath is called any more often than PlotCurveItem.updateData... so we're just moving the calculation from one part of the library to another...

Do you guys have any benchmarks set up? If not, maybe consider https://github.com/airspeed-velocity/asv.

Im currently working on trying to optimize the connect='pairs' use-case;

this here is almost as fast as the current library implementation

        moveTo = path.moveTo
        lineTo = path.lineTo
        for elementType, x, y in zip(
            itertools.cycle((lineTo, moveTo)),
            arr['x'],
            arr['y']
        ):
            elementType(x, y)
        return path

I suspect there exists a np.nditer or np.apply_along_axis method that can beat this out though.

For the use-case of connect='array', I suspect there is likely some cleverness that can be done with np.split and the QPolygonF construction that connect='all' uses.

@pijyoi I'm curious if you have input here. I was trying to find alternative methods for doing the connect='pairs' use-case.

The fastest I got (which is about 1-2 fps slower than current) is:

        # 26-27 fps
        for elementType, x, y in zip(
            itertools.cycle((path.moveTo, path.lineTo)),
            arr[1:-1]['x'].tolist(),
            arr[1:-1]['y'].tolist()
        ):
            elementType(x, y)
        return path

some other things I tried:

really had high hopes for this one... haha

        # 3 fps
        for segment in np.split(arr[1:-1], np.argwhere(arr[1:-1]['c'] == 0).flatten()):
            polygon = arrayToQPolygonF(
                segment['x'],
                segment['y']
            )
            path.addPolygon(polygon)
        return path

        # 18 fps
        _ = list(
            map(
                lambda x: path.lineTo(x[0], x[1]) if x[2] else path.moveTo(x[0], x[1]),
                zip(arr[1:-1]['x'], arr[1:-1]['y'], arr[1:-1]['c'])
            )
        )
        return path

        # 11 fps
        operation = itertools.cycle((path.moveTo, path.lineTo))
        list(map(lambda x: next(operation)(x[0], x[1]), arr[1:-1][['x', 'y']]))
        return path

I also experimented with np.vectorize() but didn't really get anywhere there either...

The one thing I couldn't get was a go of np.nditer from the arr structured array. If the np.nditer use-case is just a tiny bit faster than the naive implementation above, it could be worthwhile to throw in. Furthermore, if it's faster, and doesn't depend on itertools.cycle, this would be a good general case for connect='array' as well.

EDIT: for context I'm using examples/MultiPlotSpeedTest.py for testing.

I would say that reading arr is not a good idea due to wrong-endian and misalignment of float64. It is a serialization format, not a computation format.

arr = np.empty(n+2, dtype=[('c', '>i4'), ('x', '>f8'), ('y', '>f8')])

Why not just:

for i in range(0, n, 2):
    path.moveTo(x[i], y[i])
    path.lineTo(x[i+1], y[i+1])
return path

oh that's a good tip; still not quite as performant as master but close; one thing that definitely sped things up was to convert to a list:

        x = x.tolist()
        y = y.tolist()
        for i in range(0, n, 2):
            path.moveTo(x[i], y[i])
            path.lineTo(x[i+1], y[i+1])

I have an old patch that uses bytearray as the backing store instead of np.empty
pijyoi@a8733cc
pijyoi@5a6871b

It avoids an extra conversion to bytes for PySide{2,6} because QByteArray accepts bytes or bytearray but not a generic buffer.
If nothing, it avoids having to use the pesky offsets arr[1:-1]

If you find it worthwhile, you could incorporate it.

UPDATE: looking at the patch again, it looks like there is a bug in the elif eq(connect, 'finite'): branch. FIXED

I have an old patch that uses bytearray as the backing store instead of np.empty
pijyoi@a8733cc

It avoids an extra conversion to bytes for PySide{2,6} because QByteArray accepts bytes or bytearray but not a generic buffer.
If nothing, it avoids having to use the pesky offsets arr[1:-1]

If you find it worthwhile, you could incorporate it.

UPDATE: looking at the patch again, it looks like there is a bug in the elif eq(connect, 'finite'): branch.

If that low level code can be simplified without taking a performance penalty or losing compatibility, I'd be all for merging.

I think part of this PR will add a number of tests to this function as well.

I applied the (fixed) patch to your PR in https://github.com/pijyoi/pyqtgraph/commits/arraypath.

I applied the (fixed) patch to your PR in https://github.com/pijyoi/pyqtgraph/commits/arraypath.

Looks like you and I were both working on handling merge conflicts; I ended up cherry picking your commit, and embedded it earlier in the commit history of the branch, if you'd rather I adopt the commit at the head of your branch, let me know, I can do that too.

I really want to get faster methods for connect='finite' and connect='array', but I'm not sure I can guarantee a performance improvement with the use of QPolygonF, if the paths are very frequently disconnected, likely anything I do will be slower. I could try some kind of logic with np.split(np.vstack(x, y), np.argwhere(arr[c] == 0)) and then check the length of the resulting segment, if it's short, use a loop with path.lineTo() calls, if it's longer than some number, use QPolygonF segments...

Maybe this is a good time to do some test cases for the test suite before I try more.

You're right; probably shouldn't put our eggs in the undocumented feature basket. I should probably lobby the pyside2/qt devs to see if

1. they can restore np.nan / (float(nan) as a valid value for QPainterPath creation
2. speed up the QDataStream >> QPainterPath operation

either that, or give us a way to pass numpy arrays in directly (functionality they seem interested in providing, but unfortunately it probably won't be matched by PyQt6 :( ).

Depending on how much you want to split hairs, I see a few % increase by avoiding dot access on some methods, i.e. from numpy import isfinite and using isfinite instead of np.isfinite and so on for other member functions...

You also can compare string versions directly, i.e. qtver > ['5', '12', '3'] which will yield the same result without int conversion and list comp on the version number.

You can also define ndarray_from_qpolygonf_qt6 and qt5, set it during module init, and avoid a branch statement inside the tight loop.

Again, splitting hairs but what's the point of a CS minor if I can't agonize over worthless performance gains?

Also the Qt6-only codepaths are using undocumented behaviour, so who knows,
they may need to get reverted once people really try it out.

Aha, I found that examples/beeswarm.py is horribly broken by the usage of undocumented behavior.
What actually breaks it is the usage of a pen width other than 1.

err = pg.ErrorBarItem(x=np.arange(4), y=data.mean(axis=1), height=data.std(axis=1), beam=0.5, pen={'color':'w', 'width':2})

This is probably related to the issue of plots being slow when using thick pens; some other drawing codepath probably gets activated in Qt.

oooooof and yeah, works fine on master well, this is discouraging

hmm...there may be something more to it than that. On PySide6, I modified examples/ErrorBarItem.py so that the error barcall is

Disregard, I was on the wrong branch

plt = pg.plot()
plt.setWindowTitle('pyqtgraph example: ErrorBarItem')
err = pg.ErrorBarItem(x=x, y=y, top=top, bottom=bottom, beam=0.5, pen={'color':'w', 'width':2})
plt.addItem(err)
plt.plot(x, y, symbol='o', pen={'color': 0.8, 'width': 2})

here is the result...

Sorry for the notification spam; managed to solve this by casting x as a float64

err = pg.ErrorBarItem(x=np.arange(4).astype(np.float64), y=data.mean(axis=1), height=data.std(axis=1), beam=0.5, pen={'color':'w', 'width':2})

Disregard, was on the wrong branch

seems to be an issue limited to connect='pairs' ... modifying some of the line plots with thick lines does not seem to impact this.... what remarkably odd behavior.

EDIT: appears that finite behavior is also problematic

Slight modification to examples/Plotting.py

p4 = win.addPlot(title="Parametric, grid enabled")
x = np.cos(np.linspace(0, 2*np.pi, 1000))
x[500] = np.nan
y = np.sin(np.linspace(0, 4*np.pi, 1000))
p4.plot(x, y, connect='finite', pen={'color': (255,0,0), 'width': 2})
p4.showGrid(x=True, y=True)

This took a really long time to display (like 1-2 minutes)

May I suggest that the Qt6-only "finite" and "pairs" codepaths be moved out from the scope of this PR? Don't let them block the merge of the optimized "all" codepath.

From what I can see, no part of the library makes use of "finite". The places that make use of "pairs" only do so with a small number of points, so speed shouldn't be an issue.

May I suggest that the Qt6-only "finite" and "pairs" codepaths be moved out from the scope of this PR? Don't let them block the merge of the optimized "all" codepath.

From what I can see, no part of the library makes use of "finite". The places that make use of "pairs" only do so with a small number of points, so speed shouldn't be an issue.

I agree. I was hoping to make use of pairs in a change to the downsample='peak' use case in PlotDataItem, but I can worry about that later.

I'll revert/update the commits accordingly.

Would it be better to remove the version check and just unconditionally do the non-finite back-filling? The case qtver < [5, 12, 3] is no longer being tested by the CI.

    qtver = [int(x) for x in QtVersion.split('.')]
    if qtver >= [5, 12, 3]:

I am working on an alternate QPolygonF "finite" implementation: "https://github.com/pijyoi/pyqtgraph/tree/split_finite
Needs more testing but for the (hopefully!) common case where non-finite elements are the exception rather than the norm, it performs better than the operator<< codepath.

Will wait for #1807 and #1796 to be merged first.

As I understand it, there are 2 use-cases for "finite":

1. plots with a bit of missing data represented by nans
   • the QPolygonF codepath will be activated by the heuristic
2. nans used as a delimiter to plot disjoint shapes with a single PlotCurveItem (this is actually what I use it for)
   • user can force the QPolygonF codepath with finiteCheck=False

@pijyoi your use case of pyqtgraph is something I've been really curious about. At some point I'd love to hear more about how yo use the library in your own work.

Those PRs will be merged shortly (end of the day today, this time for sure 😆 ) ... also I think we should do another release with removing Qt 6.0 support (and officially supporting 6.1+) before much longer... I was wanting to do a 0.13.0 release, but we have quite a few deprecation warnings that we say we will remove in 0.13 version of the library... so maybe 0.12.2 ... or maybe I should move this topic to the discussions section.

@pijyoi thank you so much for your help with this

@PierreRaybaut thanks for showing how to construct a QPolygonF object straight from a numpy array. If you ever look to bring Qt6 functionality to PythonQwt, feel free to see how we implemented it.

Merging this PR, yay for faster line plots!