Skip to content

[ENH] replacing CoordinateSystem replacement with space + examples#12

Merged
chrisgorgo merged 19 commits intoenh/derivativesfrom
enh/space_examples
Mar 30, 2019
Merged

[ENH] replacing CoordinateSystem replacement with space + examples#12
chrisgorgo merged 19 commits intoenh/derivativesfrom
enh/space_examples

Conversation

@chrisgorgo
Copy link
Copy Markdown
Owner

@chrisgorgo chrisgorgo commented Mar 16, 2019

  • Remove CoordinateSystem due to redundancy with space keyword
  • Added anat volumetric Coordinate System.
  • Clarified when ReferenceMap is mandatory
  • Added examples

I know this does not cover all possible cases, but it should be useful for 80% of the cases and it's a start.

@chrisgorgo
Copy link
Copy Markdown
Owner Author

Pinging @satra @oesteban @effigies and @edickie for comments.

@satra
Copy link
Copy Markdown

satra commented Mar 16, 2019

@chrisfilo - this makes me think we should either have space or CoordinateSystem - it seems the two are redundant in all these examples. CoordinateSystem is more correct, but takes up a lot of space!

also, if space is encoded in the filename, i would say CoordinateSystem is not needed (or has to match).

@chrisgorgo
Copy link
Copy Markdown
Owner Author

All great points! I will update the PR replacing CoordinateSystem with space (this keyword is already under the same meaning in iEEG) and make it a filename-only keyword.

@chrisgorgo chrisgorgo changed the title [ENH] CoordinateSystem clarifications and examples [ENH] CoordinateSystem replacement with space and examples Mar 17, 2019
@chrisgorgo chrisgorgo changed the title [ENH] CoordinateSystem replacement with space and examples [ENH] replacing CoordinateSystem replacement with space + examples Mar 18, 2019
@chrisgorgo
Copy link
Copy Markdown
Owner Author

@dorahermes I'm replying here because its a better space to have this discussion:

In the current state of the PR ReferenceMap is indeed similar to IntendedFor in _coordsystem (although the definition is slightly different due to the root of the relative path - to allow for group-level custom templates in the future). Currently ReferenceMap is REQUIRED only if space is set to anat defined as

Participant specific anatomical space (for example derived from T1w and/or T2w images). This coordinate system require specifying an additional participant specific file to be fully defined.

This new anat keyword is probably the biggest change that might influence iEEG/EEG specs, but it might be actually a useful addition (considering bids-standard#169 and https://neurostars.org/t/questions-about-appendix-viii-preferred-names-of-coordinate-systems/3758). Would love to hear your thoughts.

@satra
Copy link
Copy Markdown

satra commented Mar 20, 2019

an individual space may come from any data collection (could be diffusion b0, T1w epi, MPM scans, etc.,.). we are already using fsnative to indicate the native freesurfer mesh. as such i think native may be a better word than anat.

@edickie
Copy link
Copy Markdown

edickie commented Mar 20, 2019

It was already using fsnative too. anat does make some sense because it would map to the bids anat/ folder?

@chrisgorgo
Copy link
Copy Markdown
Owner Author

I don't have a strong opinion. anat was originally proposed by @oesteban to replace T1w with something less specific. native still serves that purpose - happy to switch to that.

Copy link
Copy Markdown

@effigies effigies left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible still to define your own space by using a custom string and defining ReferenceMap?

FWIW I lean more toward native than anat, if we're going to give one or the other the blessing of the spec, but I think either should be doable, and it's the responsibility of the app writer to define what the space is.

| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Description | RECOMMENDED. Free-form natural language description of the nature of the file. |
| Sources | OPTIONAL. A list of paths relative to dataset root pointing to the file(s) that were directly used in the creation of this derivative. For example in a chain of A->B->C, “C” should only list “B” as Sources, and “B” should only list “A” as Sources. However in case X and Y jointly contribute to Z, then “Z” should list “X” and “Y” as Sources. |
| RawSources | OPTIONAL. A list of paths relative to dataset root pointing to the BIDS-Raw file(s) that were used in the creation of this derivative. | |
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's an extra | at the end here?

| Description | RECOMMENDED. Free-form natural language description of the nature of the file. |
| Sources | OPTIONAL. A list of paths relative to dataset root pointing to the file(s) that were directly used in the creation of this derivative. For example in a chain of A->B->C, “C” should only list “B” as Sources, and “B” should only list “A” as Sources. However in case X and Y jointly contribute to Z, then “Z” should list “X” and “Y” as Sources. |
| RawSources | OPTIONAL. A list of paths relative to dataset root pointing to the BIDS-Raw file(s) that were used in the creation of this derivative. | |
| ReferenceMap | REQUIRED when a custom template or file is used. A path to a file that was used as, or can be used as, a reference image for determining the coordinate space of this file. The path should start with a `/` and should be relative to the root of the dataset. |
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible for this to be a list, e.g., in the case of CIFTIs?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i would think so, and we should clarify.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking about it and shouldn't a ReferenceMap for a CIFTI file be another single CIFTI file?

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what I call T1wfsLR is the that I think you asked about in another thread, is the hardest.

example: (for now)

sub-{subject}/anat/sub-{subject}_hemi-L_space-T1wfsLR32k_pial.surf.gii
{
ReferenceMap : ['./sub-{subject}_desc-preproc_T1w.nii.gz', 'fsLR32k']
}

is this making any sense.
P.S. I tried to list some cifti/hcp derivatives and what there space would be in this sheet https://docs.google.com/spreadsheets/d/1uJKQWoqGFD0FJE8Tm9hQVFj2-xOgo5zN1dO-ALwkT0M/edit?usp=sharing

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, more to the point. I don't think it makes sense to have one cifti file reference. There's 4 template files in template flow (L and R sphere + the MNI template + the subcortical parcellation template that end up being used..) I can't picture a nice what to combined those into 1 file...

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

templateflow is young and still evolving so I am not sure it is the best reference.

I know that you can definitely pack the two surfaces and subcortical part of the template in a CIFTI file. I am not an expert, but I thought that in connectome workbench one could easily load such CIFTI file and use it as a background to visualize another CIFTI file in the same space. Is that correct?

Would be great to get a clarification on the T1wfsLR. Thanks in advance.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CIFTI files hold surface mapped data (scalar and timeseries) but the geometry (surf.gii) files have always needed to be loaded as two (or more) separate files in order to work with the data. also a volume template (i.e. T1w.nii.gz) was always loaded together as a background as well.

..actually, workbench gets around this by listing all surfaces and volumes that are in the same space within a ".spec" file (that workbench reads to load all files at once). We could recommend that the .json instead points to a ".spec" file for cifti?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

urg..yeah I see. These use hybrid spaces cases probably need their own table in the spec..

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for clarifying.

The most useful case for this (correct me if I am wrong) I can see is the native hybrid space (combination of native volume and fsnative surface). I think a list is better than adding a new data type (.space). This will only work if we require that when space is set to native for a CIFTI file ReferenceMap needs to be a 4 element list that includes surfaces for two hemispheres, volumetric template, and a subcortical parcellations (all valid BIDS derivatives files). Alternatively, we can prescribe a dictionary in such a situation. WDYT?

@satra
Copy link
Copy Markdown

satra commented Mar 20, 2019

these two were removed from the document. they should be added to the appendix.

| Unknown        | The coordinate system is unknown.                                                                                           |
| Custom         | A custom coordinate system that is not in alignment (dimensions, axis orientation, unit) with any device coordinate system. 

@effigies
Copy link
Copy Markdown

Are you suggesting the literal string "Custom", or using it as a stand-in for a custom string? I'm not a fan of the literal string "Custom", as it tells someone looking at the data nothing that any otherwise-unspecified string would tell you.

@satra
Copy link
Copy Markdown

satra commented Mar 20, 2019

the literal string custom was intended to indicate any map that did not fit in with any of the other named systems. so it's literally a custom coordinate system. this would force a referencemap to clarify details.

naming the system would not give it any more specificity other than filename disambiguation. so i would be fine with custom becoming <custom>, and being allowed to replace with a stand-in.

@edickie
Copy link
Copy Markdown

edickie commented Mar 20, 2019

The literal string custom is problematic for me. I end up having more than one 'custom' space (i.e. things not listed in this table) because I concatenate together the ReferenceMap list, just to disambiguate.. (i.e. T1wfsLR, MNIfsLR...is ths now nativefsnative...)

What;s the option when more than two custom spaces exist within one dataset?

Also I'm not realizing that space-nativefsnative is (volume native + surface fsnative) is a weird looking tag.

effigies and others added 2 commits March 20, 2019 15:10
Co-Authored-By: chrisfilo <krzysztof.gorgolewski@gmail.com>
Co-Authored-By: chrisfilo <krzysztof.gorgolewski@gmail.com>
@chrisgorgo
Copy link
Copy Markdown
Owner Author

@satra I am not very fond of hybrids of free and controlled vocabularies (when space could be any of the controlled names or a custom name). It makes checking for typos very hard (fsLR32 is a brand new space or misspelled fsLR32k?) and can lead to confusion when the controlled list evolves taking over a name someone uses in their private dataset. Thus I would prefer to keep space to restricted values.

@edickie in my head custom would be for rare cases when users use study-specific templates and it would be very rare for someone to use two such templates. In those rare cases desc could be used to avoid file name collisions.

Nonetheless, your comment highlights some missing hybrid spaces. Perhaps we should add native (combination of fsnative for surface and native for volume to the hybrid table)?

Additional questions:

@satra
Copy link
Copy Markdown

satra commented Mar 20, 2019

@chrisfilo - fine with me to restrict it to a controlled vocabulary but we should add the literal "custom" and unknown back in.

@edickie
Copy link
Copy Markdown

edickie commented Mar 20, 2019

HCPMNIfsLR is for cifti files where HCP's subcortical parcellation is present in the file's metadata (i.e. subcortical data has been resampled within subcortical parcels - i.e. 91k)

MNIfsLR is for gifti surfaces (i.e. white surface, pial surface) where the surface vertices are aligned to fsLR and 3D coordinates are aligned to the MNI template (i.e. the surfaces can be overlayed on with the sub-{subject}_space-MNI_desc-preproc_T1w.nii.gz)

@chrisgorgo
Copy link
Copy Markdown
Owner Author

Ok Thanks for clarifying @edickie. Followup question: what is the 3D coordinate alignment of the fsLR32k GIFTI/surface space?

@edickie
Copy link
Copy Markdown

edickie commented Mar 22, 2019

space-individual is more characters than my preference. But otherwise, I have no objections.

To clarify, what would the ReferenceMap point to in the case of fsnative? Is it the fsnative sphere or the T1w image?

@satra
Copy link
Copy Markdown

satra commented Mar 22, 2019

@edickie - one thought process would be to remove fsnative and just use individual . the original consideration for fsnative was the surface mesh (so lh/rh.[pial/white/smoothwm], which would be a list (in the context of a cifti fsnative).

but fsnative can get complicated (even for fmriprep) depending on which command is used to project say functional on to the mesh. if mris_expand or mri_vol2surf (current default) is used, then there will be a specific mesh with different 3d coordinates than any of the above default freesurfer outputs.

@edickie
Copy link
Copy Markdown

edickie commented Mar 22, 2019

Putting aside how we would capture information about functional volume to surface - that's the p-word again. I was more trying to figure out what the space metadata for a pial.surf.gii file would look in space-individual (does ReferenceMap point to itself?)

One other thing to consider is that (as far as i can tell) ciftify and fmriprep are both using fsnative because they are actually the same derivative (i.e. the same surface mesh). They were both created by converting the freesurfer (fsnative) surfaces to gifti. In my mind fsnative implied the individual surface mesh as created by freesurfer. (ok I guess that is also provenance..)

@edickie
Copy link
Copy Markdown

edickie commented Mar 22, 2019

I guess my ask is if we can leave fsnative alone (as the freesurfer derived individual space) and leave space-individual as a volume (3D) space tag. Because I imagine that "individual" space surfaces from different pipelines (caret, CIVET, BrainVoyager) might be very different in their geometry.

@satra
Copy link
Copy Markdown

satra commented Mar 22, 2019

@edickie - i would not limit the space-individual to volumes for precisely the reason that you stated. someone using civet or other surfaces can still use it.

regarding your question about pointing to itself, that's a good question. this would not only be true of pial.surf.gii but also of a realigned bold image or diffusion b0 map or an averaged T1w. these all create a new "space". as such i would be completely fine with calling them space-individual and pointing to themselves.

also the functional to surface was not completely about provenance but about the space itself. since we are defining space as (coordinate system + object), where object is some image in 3d or a mesh or streamlines, or sth else. when projecting volume to surface, often the projection is done not to a specific surface (which would be simple), but to a space in between surfaces (computed differently by different projectors). in such a situation, this intermediate surface would have to be generated alongside a derivative. currently this is not happening in fmriprep and we are simply calling it fsnative (this should perhaps change).

@chrisgorgo
Copy link
Copy Markdown
Owner Author

To clarify, what would the ReferenceMap point to in the case of fsnative? Is it the fsnative sphere or the T1w image?

For meshes (.surf.gii) in space-fsnative I would recommend having only one value of the ReferenceMap field - the volume defining the 3D coordinates it is aligned with (for example T1w volume). I don't see the point of adding another value pointing to itself.

For derivatives such as cortical thickness in space-fsnative I would recommend pointing to the mesh gifti file (any of the pial, smoothwm etc. - as long as this is consistent across the dataset it should not matter).

@satra I was struggling to see an example of a commonly used derivative that would require more than one value in ReferenceMap - could you provide one? I'm trying to see if we can avoid having to use a list - a list is not great for objects of inconsistent nature (a surface and a volume, surfaces for two hemispheres etc.), since looking up a specific object requires you to iterate over the whole list and check each object. If we need multiple ReferenceMap we should probably use a dictionary. From I gather so far I can avoid it - at least for now.

fsnative is not part of the current release so I am happy to switch it to individual.

I agree with @edickie that we should probably leave the provenance of surface mapping to another PR.

@chrisgorgo
Copy link
Copy Markdown
Owner Author

To help thinking about the latest updates, here's a new conversion table for ciftify outputs:

CIFTI files:

space-fsLR32k -> space-HCPMNIfsLR32k (ReferenceMap not needed)
space-MNIfsLR32k -> space-HCPMNIfsLR32k (ReferenceMap not needed)
space-fsLR164k -> space-HCPMNIfsLR164k (ReferenceMap not needed)
space-MNIfsLR164k -> space-HCPMNIfsLR164k (ReferenceMap not needed)

.surf.gii files:

space-T1wfsnative -> space-individual_volspace-individual (ReferenceMap points to T1w nifti file)
space-T1wfsLR32k -> space-fsLR32k_volspace-individual (ReferenceMap points to T1w nifti file)
space-T1wfsLR164k -> space-fsLR164k_volspace-individual (ReferenceMap points to T1w nifti file)
space-MNIfsLR32k -> space-fsLR32k_volspace-MNI152NLin6AsymConte69 (ReferenceMap not needed)
space-MNIfsLR164k -> space-fsLR164k_volspace-MNI152NLin6AsymConte69 (ReferenceMap not needed)

Nifti files:

space-MNI -> space-MNI152NLin6AsymConte69 (ReferenceMap not needed)

@satra
Copy link
Copy Markdown

satra commented Mar 24, 2019

@satra I was struggling to see an example of a commonly used derivative that would require more than one value in ReferenceMap - could you provide one?

how about the cifti output of fmriprep - since the user can choose any of the fsaverage/native as well as the mni template?

@chrisgorgo
Copy link
Copy Markdown
Owner Author

how about the cifti output of fmriprep - since the user can choose any of the fsaverage/native as well as the mni template?

fmriprep does not produce such outputs and as far as I know, there were no user requests to change that. Right now fmriprep creates cifti files with a rare combination of fsaverage and MNI spaces not compatible with HCP pipelines outputs. Switching to HCP compatible outputs has been under consideration (see nipreps/fmriprep#1537).

@satra
Copy link
Copy Markdown

satra commented Mar 24, 2019

fmriprep does not produce such outputs

https://github.com/poldracklab/fmriprep/blob/master/fmriprep/cli/run.py#L610

it will produce output in whatever --template and --output-space are set to. the default is a specific combination of the default template and fsaverage5.

@chrisgorgo
Copy link
Copy Markdown
Owner Author

The code might be a bit confusing - this is just a hack for the workflows to work if someone is requesting cifties, without also requesting template (which can only be set to MNI) or fsaverage4/5.

More clarification: https://github.com/poldracklab/fmriprep/blob/7f119454adc6b20a4cfc3b14c89349c716ed7660/docs/outputs.rst

In short fsnative/T1w combination is not supported - @oesteban can correct me if I am wrong.

@chrisgorgo
Copy link
Copy Markdown
Owner Author

Next - and hopefully final order of business for this PR. @oesteban proposed to rename ReferenceMap to Reference. Any objections?

@dorahermes
Copy link
Copy Markdown

'Reference' has a very different meaning in electrophysiology.

@edickie
Copy link
Copy Markdown

edickie commented Mar 25, 2019 via email

@edickie
Copy link
Copy Markdown

edickie commented Mar 25, 2019 via email

@oesteban
Copy link
Copy Markdown

XFMReference has the same issues as ReferenceMap

AlignedTo could be fine. An alternative I proposed is SpatialReference

@oesteban
Copy link
Copy Markdown

'Reference' has a very different meaning in electrophysiology.

Please note that this defined within the scope of the metadata of a transform, I hardly see a conflict.

@edickie
Copy link
Copy Markdown

edickie commented Mar 25, 2019 via email

@chrisgorgo
Copy link
Copy Markdown
Owner Author

Indeed - good point @dorahwrmes.

I like AlignedTo. We could also go with IntendedFor which makes it more consistent with iEEG, but might be less intuitive.

@edickie
Copy link
Copy Markdown

edickie commented Mar 25, 2019

AlignedTo also works more nicely with the current from-{} to-{} nomenclature currently proposed for transforms.

@oesteban
Copy link
Copy Markdown

Thinking this through, AlignedTo is a really bad choice because it implies that you pull info back from the target domain (terminology shamelessly stolen from J. Ashburner in our Transforms file meeting) into the origin domain and thus (gridded) data seem aligned.

Sorry but I have to reiterate SpatialReference if you are so against Reference, which I guess it is already in the stable branch of the specs.

@oesteban
Copy link
Copy Markdown

On the other hand, @satra reminds me that this is not metadata of the transforms, this is metadata of one derived image.

I thought we had Reference only for .x5 files.

@chrisgorgo
Copy link
Copy Markdown
Owner Author

I don't think that AlignTo conveys the message that the same gridding is used, but I am also happy with SpatialReference.

@chrisgorgo
Copy link
Copy Markdown
Owner Author

I've put SpatialReference for now.

@chrisgorgo chrisgorgo merged commit 6fac7ec into enh/derivatives Mar 30, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants