Skip to content

Skip nan values in mesh#3114

Merged
j9ac9k merged 5 commits intopyqtgraph:masterfrom
sldesnoo-Delft:nan_values_in_PColorMeshItem
Oct 13, 2024
Merged

Skip nan values in mesh#3114
j9ac9k merged 5 commits intopyqtgraph:masterfrom
sldesnoo-Delft:nan_values_in_PColorMeshItem

Conversation

@sldesnoo-Delft
Copy link
Copy Markdown
Contributor

Added support for NaN values in PColorMeshItem.
Sometimes there is no valid data for some polygons in the mesh. The modification does not draw a polygon when the Z value is not a finite number (inf, nan, ...). It also ignores the not-finite numbers when determining the range of Z values.

@pijyoi
Copy link
Copy Markdown
Contributor

pijyoi commented Jul 31, 2024

Some comments:

  1. To handle nans means that the code no longer assumes finite input. I.e. To handle a special case, we are penalising all usual cases. While this is true for PlotCurveItem and ImageItem, those are integral to pyqtgraph and their behaviour can't be changed without breaking compatibility. The question is, do we really want to introduce this penalty for yet another GraphicsItem?

  2. The assumption that nan gets converted to an out-of-bounds color index may not be true on all architectures.

  3. PColorMeshItem: implement opengl rendering #3090 implements OpenGL painting of PColorMeshItem and is significantly faster (usable for millions of pixels) than the QPainter implementation (usable only for thousands of pixels). This PR only affects the QPainter implementation.

@sldesnoo-Delft
Copy link
Copy Markdown
Contributor Author

ad 2. Good point. Numpy cast of NaN to int has different behavior on different architectures. On some architectures the scaling function will assign color 0 for NaN values.

ad 3. Unfortunately I have no experience with OpenGL. My naive approach would be to remove the triangles for the NaN values from the buffer after filling it with an expression like buffer = buffer[~np.isnan(Z), ...] applied on the right axis. I assume this is possible for both ibo and not-ibo.

I wouldn't mind to add another argument to setData() to specify that there could be NaN values in Z. Or, a property filter_nan could be added to PColorMeshItem. If this argument of the property is set, then the invalid entries are removed from the buffer.

setData(..., filter_nan=True):
    ...
    if filter_nan:
        z_valid = ~np.isnan(z)
    ...

On the right places this mask is applied on a numpy array.

Shall I make a new pull request using this approach?

@pijyoi
Copy link
Copy Markdown
Contributor

pijyoi commented Aug 1, 2024

What is the use case of presenting nan values as see-through holes in the mesh?

As an aside, ignoring the question of whether nans are a good idea, handling nans differently could be done in the OpenGL shaders. For example, the level scaling code is done in the shader, which nicely avoids issues of rescaleData being slow on the cpu.

@sldesnoo-Delft
Copy link
Copy Markdown
Contributor Author

The use case is that we have scientific data were some points in the mesh do not have a value, because the point is not (yet) measured, or because the computation returns NaN for the point.

I'm sorry but I don't know how to work with OpenGL shaders.

@pijyoi
Copy link
Copy Markdown
Contributor

pijyoi commented Aug 2, 2024

I'm sorry but I don't know how to work with OpenGL shaders.

It's okay, someone else can implement it if they want to.

Changed type of polygon array from list to numpy.ndarray for fast filtering of NaN.
@sldesnoo-Delft
Copy link
Copy Markdown
Contributor Author

Modified code to filter the NaN from Z array and remove corresponding polygons.
Tested performance example of PColorMeshItem using 50x40 and 500x400 points. No significant change in performance was measured.

if skip_nans:
valid_z = self.z[~z_invalid]
else:
valid_z

Check notice

Code scanning / CodeQL

Statement has no effect

This statement has no effect.
@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Aug 18, 2024

Sorry took me a bit to run this, looks like the test suite is having issues with this:

UnboundLocalError: local variable 'valid_z' referenced before assignment

@sldesnoo-Delft
Copy link
Copy Markdown
Contributor Author

The unassigned valid_z was a serious mistake. It has been fixed.

@pijyoi
Copy link
Copy Markdown
Contributor

pijyoi commented Aug 19, 2024

I get the following error with the test script further down. Do you have a working example?

Traceback (most recent call last):
  File "D:\repos\pyqtgraph\pyqtgraph\graphicsItems\PColorMeshItem.py", line 423, in paint
    self.qpicture = self._drawPicture()
  File "D:\repos\pyqtgraph\pyqtgraph\graphicsItems\PColorMeshItem.py", line 327, in _drawPicture
    polys = polys[~z_invalid]
IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed
import numpy as np
import pyqtgraph as pg

Z = np.linspace(0, 1, 12).reshape((3, 4))
Z[1,1] = np.nan

pg.mkQApp()
win = pg.PlotWidget()
pcmi = pg.PColorMeshItem(Z)
win.addItem(pcmi)
win.show()
pg.exec()

Also, is an all-nans 2d-array a supported use-case?

Handle all-nan case.
@sldesnoo-Delft
Copy link
Copy Markdown
Contributor Author

The work I committed two weeks ago was not as it should be. Don't know how this happened.

I believe I've fixed the code. I've also added handling of the all-nan case.

@sldesnoo-Delft
Copy link
Copy Markdown
Contributor Author

@j9ac9k , @pijyoi I fixed the code. Do you think it can be merged for the next release?

@j9ac9k
Copy link
Copy Markdown
Member

j9ac9k commented Oct 13, 2024

Thanks for the PR @sldesnoo-Delft, LGTM, merging!

@j9ac9k j9ac9k merged commit a9b868e into pyqtgraph:master Oct 13, 2024
j9ac9k pushed a commit to j9ac9k/pyqtgraph that referenced this pull request Mar 25, 2025
* Skip nan values in mesh

* Remove NaN values from Z array and remove corresponding polygons.
Changed type of polygon array from list to numpy.ndarray for fast filtering of NaN.

* Fixed bug in previous commit

* Fixed filtering polys.
Handle all-nan case.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants