Add GE scanner physiological file functionality by nw-duncan · Pull Request #424 · physiopy/phys2bids

nw-duncan · 2022-04-23T10:18:15Z

Functionality to read and process GE scanner physiological recording files is added to the tool.

Users can enter a single scanner generated file (with the name given to it by the scanner). Note that these do not have any suffix. If there are other files corresponding to other measurement modalities (i.e., PPG, respiration belt, or EKG) then these will be found automatically and added.

Proposed Changes

New function to check if an input file has the filename convention of a GE file. If so all other relevant files are found and a ".gep" suffix appended to each (preserving the processing pipeline implemented for other supported file types)
New function to read in the relevant files. This also adds a simulated trigger channel as GE scanners do not, to my knowledge, output such information natively
All other steps proceed as with currently supported files

Change Type

Checklist before review

I added everything I wanted to add to this PR.
[Code or tests only] I wrote/updated the necessary docstrings.
[Code or tests only] I ran and passed tests locally.
[Documentation only] I built the docs locally.
My contribution is harmonious with the rest of the code: I'm not introducing repetitions.
My code respects the adopted style, especially linting conventions.
The title of this PR is explanatory on its own, enough to be understood as part of a changelog.
I added or indicated the right labels.

I added information regarding the timeline of completion for this PR.

Please, comment on my PR while it's a draft and give me feedback on the development!

codecov · 2022-08-17T13:04:58Z

Codecov Report

Merging #424 (635cb56) into master (8185967) will increase coverage by 0.04%.
The diff coverage is 90.10%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #424      +/-   ##
==========================================
+ Coverage   94.77%   94.82%   +0.04%     
==========================================
  Files           8        8              
  Lines         881      966      +85     
==========================================
+ Hits          835      916      +81     
- Misses         46       50       +4

Impacted Files	Coverage Δ
phys2bids/io.py	`96.63% <82.50%> (-3.37%)`	⬇️
phys2bids/utils.py	`98.09% <95.23%> (-0.35%)`	⬇️
phys2bids/phys2bids.py	`90.10% <100.00%> (+2.46%)`	⬆️
phys2bids/physio_obj.py	`94.03% <100.00%> (ø)`

smoia · 2022-11-29T10:57:08Z

@nw-duncan once #430 is tested and merged, could you rebase this PR on the new master? I switched back CI to Circle CI, so we can exclude missed runs due to CI providers.

…nner physiological file. If so, adds a .gep filename extension.

…nner physiological file. If so, adds a .gep filename extension. Do nothing if extension is already present.

…input for trigger_idx. Make this input optional with default of 0. If "None" is set as the input then it records this in the log

…s an optional input in their BlueprintInput calls (need to follow up with this so that it's truly optional? At the moment it's hardcoded that they all will actually have an input)

…as visualisation is based upon there being a trigger channel. Changed load_gep to insert a column of zeros for the trigger channel

…rches for other modalities based on the filename the user enters. Need to modify check_ge in io.py before proceeding

… files and adds the .gep extension to them too

…cal files

…les that might be produced if phys2bids is run without a heuristics file or no subject id. Also stopped it adding .gep to files that already end in that

…lename contains data that isn't consistent with GE physiological format. Catches strings and files with more than one column

…ta where the scan is running (i.e., from 30s into the data to the end)

…files

smoia

LGTM!

smoia · 2022-12-23T09:55:04Z

@nw-duncan if you agree, we merge this part in now, and we wait for documentation later?

nw-duncan · 2022-12-23T10:49:15Z

Yes, let's do that.

smoia · 2022-12-23T10:52:53Z

🚀 PR was released in 2.8.0 🚀

- Replaced deprecated numpy aliases (np.int, np.float, np.bool, np.object) with builtins or explicit types - Updated/added tests for dtype handling - Added GitHub Actions CI matrix to test Python 3.8-3.10 and numpy 1.22-1.24 - Ensures phys2bids works on multiple numpy versions Closes physiopy#424

smoia added the Minormod This PR generally closes an `Enhancement` issue. It increments the minor version (0.+1.0) label Apr 23, 2022

nw-duncan and others added 27 commits November 29, 2022 12:02

Test github push

58a3e38

Delete test file

4c991c1

Add test ppg and resp files

a1e2abe

Added new function to utils.py to check if the input file is a GE sca…

f70488a

…nner physiological file. If so, adds a .gep filename extension.

Added new function to utils.py to check if the input file is a GE sca…

c069d73

…nner physiological file. If so, adds a .gep filename extension. Do nothing if extension is already present.

Added check_ge function to phys2bids function.

74916d8

Make ".gep" a supported file type in utils.py

1dfc1ec

Start implementing "load_gep" function in "io.py"

a64f642

First version of load_gep done

504081b

Altered BlueprintInput object to be able to handle "None" as a valid …

772be1f

…input for trigger_idx. Make this input optional with default of 0. If "None" is set as the input then it records this in the log

Modified load_mat, load_txt, load_acq functions so that trigger_idx i…

f374918

…s an optional input in their BlueprintInput calls (need to follow up with this so that it's truly optional? At the moment it's hardcoded that they all will actually have an input)

Modified phys2bids.py to handle chtrig = None

e570469

Fixed start time offset to 30s and recording units to mV

60e1260

Reverted trigger_idx from being optional in load functions. No point …

568f6c1

…as visualisation is based upon there being a trigger channel. Changed load_gep to insert a column of zeros for the trigger channel

Reverted trigger_idx from being optional in physio_obj

413c607

Small details in load_gep

72d609c

Minor tidy up

0500a18

Began altering load_gep so that it loads all physiological files. Sea…

3f50a81

…rches for other modalities based on the filename the user enters. Need to modify check_ge in io.py before proceeding

Updated check_ge so that it looks for other matching GE physiological…

c1d53d2

… files and adds the .gep extension to them too

Updated check_ge so that it looks for other matching GE physiological…

92798ed

… files and adds the .gep extension to them too

Updated load_gep so that it will add in any extra matching physiologi…

113b093

…cal files

Fixed problem in check_ge where it added .gep to .json and .tsv.gz fi…

c328024

…les that might be produced if phys2bids is run without a heuristics file or no subject id. Also stopped it adding .gep to files that already end in that

Added catch in check_ge that will detect if a file with a GE-style fi…

c368a65

…lename contains data that isn't consistent with GE physiological format. Catches strings and files with more than one column

Altered trigger in load_gep so that it indicates the period in the da…

df0bf39

…ta where the scan is running (i.e., from 30s into the data to the end)

Created test that ensures check_ge can catch improperly formatted GE …

0471e83

…files

Created test that ensures check_ge creates files with the .gep suffix

70b13f8

Created test that ensure load_gep loads the data correctly

500a71a

smoia and others added 15 commits November 29, 2022 12:03

Attempt at fixing files imports from OSF

5fdc553

Create pytest fixtures for downloading different GE files

676610c

Updated integration test for GE

448a303

Updated integration test for GE

fef9131

Cosmetic change in io

3baa765

Updated utils tests

78eda6c

Updated utils tests

1e94ff1

Updated io tests

cab7454

Updated integration test for GE

bab3747

Updated integration test for GE

d0977ea

Updated integration test for GE

05a64c2

Updated integration test for GE

79aed12

Fixed problem in physio_obj

cae80f6

Delete PPGData_epiRT_0509202010_27_51_971

d06cfaa

Delete RESPData_epiRT_0509202010_27_51_971

f5d1229

smoia force-pushed the test-ge branch from 4919a6b to f5d1229 Compare November 29, 2022 11:07

nw-duncan and others added 4 commits December 20, 2022 20:30

Merge branch 'physiopy:master' into test-ge

2c955ee

Added additional tests for io.py

9832a13

Fixed additional tests for io.py

7d2ee20

Updated additional tests for io.py

173d493

smoia mentioned this pull request Dec 20, 2022

Update code to support numpy >= 1.24 #434

Open

Altered numpy version requirements.

635cb56

smoia marked this pull request as ready for review December 23, 2022 09:51

smoia approved these changes Dec 23, 2022

View reviewed changes

smoia merged commit 4464c84 into physiopy:master Dec 23, 2022

smoia added the released This issue/pull request has been released. label Dec 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add GE scanner physiological file functionality#424

Add GE scanner physiological file functionality#424
smoia merged 49 commits intophysiopy:masterfrom
nw-duncan:test-ge

nw-duncan commented Apr 23, 2022 •

edited by smoia

Loading

Uh oh!

codecov bot commented Aug 17, 2022 •

edited

Loading

Uh oh!

smoia commented Nov 29, 2022

Uh oh!

smoia left a comment

Uh oh!

smoia commented Dec 23, 2022

Uh oh!

nw-duncan commented Dec 23, 2022

Uh oh!

smoia commented Dec 23, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

nw-duncan commented Apr 23, 2022 • edited by smoia Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed Changes

Change Type

Checklist before review

Uh oh!

codecov bot commented Aug 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

smoia commented Nov 29, 2022

Uh oh!

smoia left a comment

Choose a reason for hiding this comment

Uh oh!

smoia commented Dec 23, 2022

Uh oh!

nw-duncan commented Dec 23, 2022

Uh oh!

smoia commented Dec 23, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

nw-duncan commented Apr 23, 2022 •

edited by smoia

Loading

codecov bot commented Aug 17, 2022 •

edited

Loading