We have some paradigms coming in where epochs are of variable duration (e.g., sentences are used). The current fixed tmax is unsuitable; [EDIT: if I have a set of all_tmax values in my paradigm, then using in the Epochs constructor] tmax=min(all_tmaxs) will cut off otherwise useful data from some epochs, and tmax=max(all_tmaxs) will lead to TOO_SHORT at the end of MEG runs.
Potential solutions:
- Allow a variable
tmax. This is probably much too hard to deal with given how our IO and class is structured.
- Add a
too_short='omit' | 'nan' kwarg to the Epochs constructor. In other words, if there are parts of the epoch (probably after the baseline period, which should probably still be required to be available in the Raw instance if baseline is not None) that are missing from the end, then just put np.nan in for the unavailable values.
- This way users can use
tmax=max(all_tmaxs) and be okay.
- In this case, employing methods using the
Epochs object can be considered to be caveat emptor, and we can make this clear in the docstring.
- Over time we can make some of our functions smarter with a
_check_finite function, which we should probably do anyway, really.
- We'll be killed on
epochs.save() storage a bit this way, but it's easily by using .fif.gz as those chunks will effectively disappear.
It seems like the (hopefully) little bit of added complexity from (2) is worth the enhanced functionality we gain. Thoughts?
We have some paradigms coming in where epochs are of variable duration (e.g., sentences are used). The current fixed
tmaxis unsuitable; [EDIT: if I have a set ofall_tmaxvalues in my paradigm, then using in theEpochsconstructor]tmax=min(all_tmaxs)will cut off otherwise useful data from some epochs, andtmax=max(all_tmaxs)will lead toTOO_SHORTat the end of MEG runs.Potential solutions:
tmax. This is probably much too hard to deal with given how our IO and class is structured.too_short='omit' | 'nan'kwargto theEpochsconstructor. In other words, if there are parts of the epoch (probably after the baseline period, which should probably still be required to be available in theRawinstance ifbaseline is not None) that are missing from the end, then just putnp.nanin for the unavailable values.tmax=max(all_tmaxs)and be okay.Epochsobject can be considered to be caveat emptor, and we can make this clear in the docstring._check_finitefunction, which we should probably do anyway, really.epochs.save()storage a bit this way, but it's easily by using.fif.gzas those chunks will effectively disappear.It seems like the (hopefully) little bit of added complexity from (2) is worth the enhanced functionality we gain. Thoughts?