Skip to content

Gracefully handle invalid EDF dates#8283

Merged
larsoner merged 10 commits intomne-tools:masterfrom
cbrnr:edf-datetime
Sep 22, 2020
Merged

Gracefully handle invalid EDF dates#8283
larsoner merged 10 commits intomne-tools:masterfrom
cbrnr:edf-datetime

Conversation

@cbrnr
Copy link
Copy Markdown
Contributor

@cbrnr cbrnr commented Sep 21, 2020

Fixes #8281.

@agramfort
Copy link
Copy Markdown
Member

agramfort commented Sep 21, 2020 via email

@cbrnr
Copy link
Copy Markdown
Contributor Author

cbrnr commented Sep 21, 2020

Is this some known issue (two jobs fail)?

  Cloning https://github.com/larsoner/python-pytest-harvest (to revision doctest) to /tmp/pip-req-build-b8_nwlwr
  WARNING: Did not find branch or tag 'doctest', assuming revision or ref.
ERROR: Command errored out with exit status 1: git checkout -q doctest Check the logs for full command output.

@larsoner
Copy link
Copy Markdown
Member

Is this some known issue (two jobs fail)?

Fixed in master, will restart jobs

@cbrnr
Copy link
Copy Markdown
Contributor Author

cbrnr commented Sep 21, 2020

Azure jobs fail because they can't create the temp file. Is there a recommended mechanism that works?

@agramfort
Copy link
Copy Markdown
Member

@cbrnr try using our _TempDir() function?

@cbrnr
Copy link
Copy Markdown
Contributor Author

cbrnr commented Sep 22, 2020

Thanks, this should fix the problem. FWIW, the problem with tempfile.NamedTemporaryFile was that I opened it for writing and in the same block I wanted to read that file. This works on Unix, but not on Windows (and I think it is ugly anyway).

@cbrnr cbrnr changed the title Use 1900-01-01 00:00:00 for invalid EDF dates Gracefully handle invalid EDF dates Sep 22, 2020
Copy link
Copy Markdown
Member

@agramfort agramfort left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cbrnr can you just add an entry in latest.inc ?

@cbrnr
Copy link
Copy Markdown
Contributor Author

cbrnr commented Sep 22, 2020

can you just add an entry in latest.inc ?

Sure. Have we already converged on a new format? I think we agreed to include the GitHub PR number, so my entry would look like this:

- :func:`mne.io.read_raw_edf` now supports EDF files with invalid recording dates (by `Clemens Brunner`_, [#8283](https://github.com/mne-tools/mne-python/pull/8283))

If you agree I could change the only other entry by @larsoner accordingly.

@agramfort
Copy link
Copy Markdown
Member

agramfort commented Sep 22, 2020 via email

@cbrnr
Copy link
Copy Markdown
Contributor Author

cbrnr commented Sep 22, 2020

remember we use rst and not md. I am fine adding url to the PR

Right, so this would be:

- :func:`mne.io.read_raw_edf` now supports EDF files with invalid recording dates (by `Clemens Brunner`_, `#8283 https://github.com/mne-tools/mne-python/pull/8283`_)

Or leaving the contributor outside parens like before:

- :func:`mne.io.read_raw_edf` now supports EDF files with invalid recording dates by `Clemens Brunner`_ (`#8283 https://github.com/mne-tools/mne-python/pull/8283`_)

@agramfort
Copy link
Copy Markdown
Member

agramfort commented Sep 22, 2020 via email

@cbrnr cbrnr added the backport-candidate on-merge: backport to maint/1.12 label Sep 22, 2020
Copy link
Copy Markdown
Member

@larsoner larsoner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And if we really want the # in front, you or I could update the substitution code easily to use that instead of the gh- lead-in. gh- was chosen as the lead-in to match SciPy's documentation of github links, but we could change it, especially since github-changelog-generator uses the # terminology (and so do we typically).

cbrnr and others added 2 commits September 22, 2020 13:26
Co-authored-by: Eric Larson <larson.eric.d@gmail.com>
Co-authored-by: Eric Larson <larson.eric.d@gmail.com>
@cbrnr
Copy link
Copy Markdown
Contributor Author

cbrnr commented Sep 22, 2020

And if we really want the # in front, you or I could update the substitution code easily to use that instead of the gh- lead-in.

I think this would be a good idea. Where do I change this?

@cbrnr
Copy link
Copy Markdown
Contributor Author

cbrnr commented Sep 22, 2020

Found it, let's see if this works.

@hoechenberger
Copy link
Copy Markdown
Member

And if we really want the # in front, you or I could update the substitution code easily to use that instead of the gh- lead-in.

I think this would be a good idea. Where do I change this?

Can we have "GH-XXXX"? Seems more generic and, should we ever move to another platform, will help us distinguish

@cbrnr
Copy link
Copy Markdown
Contributor Author

cbrnr commented Sep 22, 2020

I'd prefer #XXX because it looks more familiar and other platforms (e.g. GitLab) also use this notation. Actually, the platform is not relevant for users, so showing GH-XXX might raise questions why GH is important, and should we switch to GitLab using GL-XXX would be similar. Instead, if we use #XXX we don't include the specific platform at all.

@cbrnr
Copy link
Copy Markdown
Contributor Author

cbrnr commented Sep 22, 2020

@larsoner could it be that the current substitution mechanism only works for issues?

@larsoner
Copy link
Copy Markdown
Member

larsoner commented Sep 22, 2020

See comment a few lines down:

 # issue/PR mode (issues/PR-num will redirect to pull/PR-num)

and try it (have to use code-mode so that GitHub doesn't substitute; copy-paste enter and it should redirect here):

https://github.com/mne-tools/mne-python/issues/8283

@cbrnr
Copy link
Copy Markdown
Contributor Author

cbrnr commented Sep 22, 2020

Sorry I don't get it. I'd like to link to a PR, how could the substitution distinguish between issues and PRs if both use identical (but separate) ranges starting from 1? We already have some issue links in our roadmap, so we can't just change everything to point to PRs.

@larsoner
Copy link
Copy Markdown
Member

Sorry I don't get it. I'd like to link to a PR, how could the substitution distinguish between issues and PRs if both use identical (but separate) ranges starting from 1

They are not separate really. They don't overlap. Try this link and see where it goes:

https://github.com/mne-tools/mne-python/issues/8283

@larsoner
Copy link
Copy Markdown
Member

Or directly try the output links

https://22434-1301584-gh.circle-artifacts.com/0/dev/whats_new.html

@cbrnr
Copy link
Copy Markdown
Contributor Author

cbrnr commented Sep 22, 2020

They are not separate really. They don't overlap.

🤯 TIL!

@cbrnr
Copy link
Copy Markdown
Contributor Author

cbrnr commented Sep 22, 2020

All green 🚀 !

@larsoner
Copy link
Copy Markdown
Member

@cbrnr try using our _TempDir() function?

IMO _TempDir use in tests should be discouraged, the pytest tmpdir fixture is more standard and cleaner. We can fix this later though

@larsoner larsoner merged commit e6a22e8 into mne-tools:master Sep 22, 2020
@larsoner
Copy link
Copy Markdown
Member

Thanks @cbrnr

@cbrnr cbrnr deleted the edf-datetime branch September 22, 2020 13:03
@larsoner
Copy link
Copy Markdown
Member

@hoechenberger I'll backport this at the same time as #8287

larsoner added a commit that referenced this pull request Sep 22, 2020
* Use 1900-01-01 00:00:00 for invalid EDF dates

* Set invalid date to None

* Add test

* Use _TempDir

* Add changes/latest.inc entry

* Include GH PR numbers

* Forgot angle brackets

* Use :gh:

Co-authored-by: Eric Larson <larson.eric.d@gmail.com>

* More :gh:

Co-authored-by: Eric Larson <larson.eric.d@gmail.com>

* Change gh- to #

Co-authored-by: Eric Larson <larson.eric.d@gmail.com>
@larsoner larsoner added backported and removed backport-candidate on-merge: backport to maint/1.12 labels Sep 22, 2020
marsipu pushed a commit to marsipu/mne-python that referenced this pull request Oct 14, 2020
* Use 1900-01-01 00:00:00 for invalid EDF dates

* Set invalid date to None

* Add test

* Use _TempDir

* Add changes/latest.inc entry

* Include GH PR numbers

* Forgot angle brackets

* Use :gh:

Co-authored-by: Eric Larson <larson.eric.d@gmail.com>

* More :gh:

Co-authored-by: Eric Larson <larson.eric.d@gmail.com>

* Change gh- to #

Co-authored-by: Eric Larson <larson.eric.d@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Read EDF files with dummy zero year/month/day

5 participants