Skip to content

Conversation

@TomDLT
Copy link
Contributor

@TomDLT TomDLT commented Apr 7, 2022

When averaging over voxels to produce a flatmap, NaNs can be ignored using nanmean=True.
However, if the data is masked (e.g. with a "thin" mask), masked voxels are not ignored like NaNs.

Example with data filled with 1s and NaNs:
"thick" mask: correct behavior, averaging 1s into 1s, and NaNs into NaNs.
thick

"thin" mask: incorrect behavior, averaging 1s into value in [0, 1].
thin

The fix is to use masked_data.filled(), which fills masked numpy arrays with the value in fill_vallue.

Details
import cortex
import numpy as np
from matplotlib import pyplot as plt

subject = 'S1'
transform = 'fullhead'

for type_ in ["thin", "thick"]:
    # create dataset with volume from 0 to n_voxels
    mask = cortex.db.get_mask(subject, transform, type=type_)
    vol = cortex.Volume(np.ones(mask.sum()), subject, transform, cmap="viridis", vmin=0, vmax=1)

    # set 50% of the values in the dataset to NaN
    vol.data[np.random.rand(*vol.data.shape) > 0.5] = np.nan

    _ = cortex.quickshow(vol, nanmean=True, with_curvature=False)
    plt.show()

@TomDLT
Copy link
Contributor Author

TomDLT commented Apr 7, 2022

Actually, this fix is incomplete, because the same problem arises with nanmean=False:
thick mask:
thick
thin mask:
thin

@TomDLT
Copy link
Contributor Author

TomDLT commented Apr 7, 2022

Full fix
Before:
before
After:
after

@mvdoc
Copy link
Contributor

mvdoc commented Apr 7, 2022

Thanks for this! the docs/tests are failing, and It looks like it might be failing when a non-masked numpy array is passed

@mvdoc
Copy link
Contributor

mvdoc commented Apr 7, 2022

awesome, thank you!

@mvdoc mvdoc merged commit 1f23d57 into gallantlab:main Apr 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants