Hey !

I'm using pyqtgraph to display very large datasets and I find the autodownsample and cliptoview very useful. However, for larger datasets (>10⁷ pts) it is very slow and I found simple fixes which increase tremendously the performances, bypassing np.isfinite and np.searchsorted when they are not necessary. After those little fixes I very smoothly navigate in huge timeseries.

This is how I increase a lot the performance in my fork :

About downsampling

if self.opts['autoDownsample']:

            if containsNonfinite:  #MRIV31 EDIT : avoids recomputing isfinite if not necessary
                finite_x = x[np.isfinite(x)]  # ignore infinite and nan values
            else:
                finite_x = x

            if view_range is not None and len(finite_x) > 1:
                dx = float(finite_x[-1]-finite_x[0]) / (len(finite_x)-1)
                if dx != 0.0:
                    width = self.getViewBox().width()
                    if width != 0.0:  # autoDownsampleFactor _should_ be > 1.0
                        ds_float = max(1.0, abs(view_range.width() / dx / (width * self.opts['autoDownsampleFactor'])))
                        if math.isfinite(ds_float):
                            ds = int(ds_float)
                        if ds>3:
                            ds = int(np.exp(int(np.log(ds))))  # MRIV31 EDIT : only allows discrete values of ds spaced logarithmically and prevents being caught in a loop between viewrange updates and small ds variations.

About ClipToView

[EDIT : as @pijyoi noted the poor performance of np.searchsorted was due to the fact that my xArray was of type int. Using np.searchsorted with a int array while the second argument is a float is extremely unefficient, hence my problem. So just make sure your xarray has a type np.float64. This should also be corrected in the code of pyqtgraph, replacing np.arange(len(y)) by np.arange(len(y),dtype=np.float64) in the initialization of PlotDataItemwhen no x array is passed].

If the xArray is sorted and linear (proportional to indices), I do not use np.searchsorted anymore. To do that I added a property xArrayLinSorted to the class PlotDataset. It can be set to True either by the user by passing the argument xArrayLinSorted=True to the initlization of the PlotDataItem or is automatically be set in the case of the absence of xArray passed by the user (when initialization of a PlotDataItem directly use np.arange)

I also added the properties xOrigin and xIncrement which contains respectively x[0] and x[1]-x[0]

        if self.opts['clipToView']:
            if view is None or view.autoRangeEnabled()[0]:
                pass # no ViewBox to clip to, or view will autoscale to data range.
            else:

                if view_range is not None and len(x) > 1:

                    if (self._datasetMapped.xArrayLinSorted == True): #MRIV31 Edit
                        x0 = (view_range.left() - self._datasetMapped.xOrigin)/self._datasetMapped.xIncrement
                        x0 = int(x0) - ds
                        x0 = max(0,x0)

                        x1 = (view_range.right() - self._datasetMapped.xOrigin)/self._datasetMapped.xIncrement
                        x1 = int(x1) + ds

                    else:
                        x0 = np.searchsorted(x, view_range.left()) - ds
                        x0 = fn.clip_scalar(x0, 0, len(x)) # workaround
                        # x0 = np.clip(x0, 0, len(x))

                        x1 = np.searchsorted(x, view_range.right()) + ds
                        x1 = fn.clip_scalar(x1, x0, len(x))
                        # x1 = np.clip(x1, 0, len(x))
                    x = x[x0:x1]
                    y = y[x0:x1]

PlotDataItem update problem bypass

Also there is a problem when you start a plot. The PlotDataItem gets updated before the autodownsample or downsample properties get passed to it by the PlotItem. To avoid that I must repeat the arguments autoDownsample=1,downsample=None,downsampleMethod="subsample" in the plot function even if those properties are set in the PlotItem.

Not Fixed : many calls to _getDisplayDataset

In general when you start a plot, _getDislayDataset is called around 5 times because of many functions calling updateItems. I do not think it is necessary and maybe you can find a workaround to increase performance.

Further not tested ideas

Downsambling with "subsample" does not take much time but for the others method you might want to avoid to repeat that operation at each update, for example by saving resampled arrays somewhere (in _datasetMapped ?) for later reuse

If the autosampling mode is off and the cliptoview mode is on, only compute the resampled array once and apply cliptoview on the resampled array. Do not resample because of cliptoview.

add more restrictive conditions before recomputing all, like a parameter ("xViewRangeWasChangedALot") in here in the function _getDisplayDataset :

 if( self._datasetDisplay is not None and
            not (self.property('xViewRangeWasChanged') and self.opts['clipToView']) and
            not (self.property('xViewRangeWasChanged') and self.opts['autoDownsample']) and
            not (self.property('yViewRangeWasChanged') and self.opts['dynamicRangeLimit'] is not None)
        ):

Note that the condition above which forces the update has a redundancy. It it is also forced by the self._dataDisplay = None in viewRangeChanged