[ENH] Additional age related columns for participants.tsv file to cover animal data#1839
[ENH] Additional age related columns for participants.tsv file to cover animal data#1839ree-gupta wants to merge 0 commit intobids-standard:masterfrom
Conversation
|
What would be the best way to indicate that these fields are recommended for non-humans, but not recommended for humans? |
Besides birthday I think we can entirely avoid species-specificity here, but I liked phrasing to refer to units section on anonymization. I was looking at the stages and if someone annotates that it is late adult stage - it might help finding subjects of such interest. It is more of a question to me of duplicity of metadata, thus possibly leading to incongruities. |
src/schema/objects/columns.yaml
Outdated
| name: age_type | ||
| display_name: Age type | ||
| description: | | ||
| Type of age measurement (e.g. `postnatal`, `fetal`, `gestational`). If not specified, `postnatal` is assumed. |
There was a problem hiding this comment.
is there some ontology or controlled vocab to reference here as well by any chance?
@lzehl says "No!"
There was a problem hiding this comment.
@satra suggested this ontology https://bioportal.bioontology.org/ontologies/FMA/?p=classes&conceptid=http://purl.org/sig/ont/fma/fma312872 (in #1634) however this looks not like a species independent definition (definitions are also very sparse in this ontology)
There was a problem hiding this comment.
if age_reference, some time points to consider:
- fertilization
- birth
- gestation
- ...
it seems we are boiling down to establishing a list of Enums with clear definitions and possibly adding TermURLs for some of them where there is a clear association.
TODO also to formalize such "TermURL" like linking at the level of schema which seems to lack it ATM.
There was a problem hiding this comment.
fertilization
UBERON: http://purl.obolibrary.org/obo/GO_0009566
Definition: "The union of gametes of opposite sexes during the process of sexual reproduction to form a zygote. It involves the fusion of the gametic nuclei (karyogamy) and cytoplasm (plasmogamy)."
OR
We suggest a new UBERON term for the life cycle temporal boundaries:
start of embryo stage
UBERON: REQUEST TO ADD
Adapted definition for the usage here: "A life cycle temporal boundary that marks the start of the embryo stage of the organism."
start of neonate stage (synonym: birth)
UBERON: http://purl.obolibrary.org/obo/UBERON_0035946
Definition: "A life cycle temporal boundary that marks the start of the neonate stage of the organism."
Gestation is not covered in UBERON, but we can maybe request to add these terms:
gestational stage (life cycle stage)
UBERON: REQUEST TO ADD
Definition: "A life cycle stage that starts with the first day of the mother's last menstrual period and ends with birth."
start of gestational stage (life cycle boundary)
UBERON: REQUEST TO ADD
Definition: "A life cycle temporal boundary that marks the start of the gestational stage of the organism."
There was a problem hiding this comment.
if
age_reference, some time points to consider:* fertilization * birth * gestation * ...it seems we are boiling down to establishing a list of Enums with clear definitions and possibly adding TermURLs for some of them where there is a clear association.
TODO also to formalize such "TermURL" like linking at the level of schema which seems to lack it ATM.
Currently the 'TermURL' is a link in the description of the enums, so I am adding it to the overall PR TODO for discussion in the next BEP meeting or otherwise.
|
@bendichter @yarikoptic I agree that only birth_date causes issues with humans due to GDPR. This list is derived from the UBERON ontology where possible with small additions were needed. Note that openMINDS did not yet integrate all UBERON life cycle stages: age_type in addition to define the reference point of the age value is a good idea. I'm not sure if the ontology https://bioportal.bioontology.org/ontologies/FMA/?p=classes&conceptid=http://purl.org/sig/ont/fma/fma312872 suggested by @satra in #1634 is species independent defined (definitions are also very sparse in this ontology). |
|
@lzehl @yarikoptic @bids-standard/bep032 As discussed in our last meeting, one of my TODOs was to add enums for |
|
@ree-gupta https://openminds-documentation.readthedocs.io/en/latest/instance_libraries/terminologies/ageCategory.html for looking up the ones we use in openMINDS (sometimes we refined definitions compared to the ontology). We focused on the UBERON ontology for life cycle stage. Note that gestational stage is missing from this ontology (we could request to add it). For life cycle temporal boundaries we can refer to some of UBERON, but we will have to contribute to complete this (or just define them ourselves as terminology). These terms should potentially be grouped under age_reference or age_temporal_boundary. This group would include for example: start of neonate stage (synonym: birth), start of embryo stage (synonym: fertilization), start of gestational stage, end of life cycle (synonym: death), etc. |
src/schema/objects/columns.yaml
Outdated
| name: age_type | ||
| display_name: Age type | ||
| description: | | ||
| Type of age measurement (e.g. `postnatal`, `fetal`, `gestational`). If not specified, `postnatal` is assumed. |
There was a problem hiding this comment.
@satra suggested this ontology https://bioportal.bioontology.org/ontologies/FMA/?p=classes&conceptid=http://purl.org/sig/ont/fma/fma312872 (in #1634) however this looks not like a species independent definition (definitions are also very sparse in this ontology)
src/schema/objects/columns.yaml
Outdated
| name: age_type | ||
| display_name: Age type | ||
| description: | | ||
| Type of age measurement (e.g. `postnatal`, `fetal`, `gestational`). If not specified, `postnatal` is assumed. |
There was a problem hiding this comment.
fertilization
UBERON: http://purl.obolibrary.org/obo/GO_0009566
Definition: "The union of gametes of opposite sexes during the process of sexual reproduction to form a zygote. It involves the fusion of the gametic nuclei (karyogamy) and cytoplasm (plasmogamy)."
OR
We suggest a new UBERON term for the life cycle temporal boundaries:
start of embryo stage
UBERON: REQUEST TO ADD
Adapted definition for the usage here: "A life cycle temporal boundary that marks the start of the embryo stage of the organism."
start of neonate stage (synonym: birth)
UBERON: http://purl.obolibrary.org/obo/UBERON_0035946
Definition: "A life cycle temporal boundary that marks the start of the neonate stage of the organism."
Gestation is not covered in UBERON, but we can maybe request to add these terms:
gestational stage (life cycle stage)
UBERON: REQUEST TO ADD
Definition: "A life cycle stage that starts with the first day of the mother's last menstrual period and ends with birth."
start of gestational stage (life cycle boundary)
UBERON: REQUEST TO ADD
Definition: "A life cycle temporal boundary that marks the start of the gestational stage of the organism."
src/schema/objects/metadata.yaml
Outdated
| enum: | ||
| - $ref: objects.enums.fertilization.value | ||
| - $ref: objects.enums.gestation.value | ||
| - $ref: objects.enums.birth.value |
There was a problem hiding this comment.
do we need this? How is this different from the age_reference in the Participants table? @yarikoptic
There was a problem hiding this comment.
This is a result of back/forth recommendations of where to integrate the age reference. This is the comment #1839 (comment) with the hopefully final decision on how to solve this.
There was a problem hiding this comment.
From what I see in the comment this should not be deleted.
|
If {
"age": {"Units": "years", "Reference": "birth"}
}We can add a check in AgeReferenceEnumeration:
issue:
...
selectors:
- columns.age
- type(sidecar.age.Reference) != 'null'
checks:
- intersects([sidecar.age.Reference], ['birth', 'gestational', 'fertilization'])Edit:
bids-specification/src/common-principles.md Lines 541 to 554 in 8e175f4 |
|
Summary of conversation with @effigies bids-specification/src/schema/objects/columns.yaml Lines 35 to 47 in 8e175f4 While this section of the schema specifies that age is in years, this is just default behavior. The description and units are overridden by entries in the participants.json file. If the validator does not respect this then it's a bug in the validator. The field that says "definition" is intended to be default (even though the name might suggest that it is not adjustable). @effigies also recommended that we add "Reference" as a field of "age" in the participants.json file. We can use custom check logic to constrain the value to birth, gestational, or fertilization. We just need to add Reference as a field in bids-specification/src/common-principles.md |
| participants: | ||
| selectors: | ||
| - path == "/participants.json" | ||
| - '"age" in columns' | ||
| fields: | ||
| AgeReference: recommended |
There was a problem hiding this comment.
Does this fit with #1839 (comment) ?
If yes this should also recommend age unit not only the age reference
There was a problem hiding this comment.
Not sure if this is also the place for defining schema.rules.checks.something suggestion from #1839 (comment)
There was a problem hiding this comment.
I can also do the changes but I wanted to have a record of the open TODOs based on the latest decision. Final TODO: update bids-specification/src/common-principles.md according to #1839 (comment)
There was a problem hiding this comment.
birth, fertilization and gestation deleted from the enums
|
STATUS/TODOS:
|
|
Heads up: I accidentally force-pushed and closed this PR, but all changes have been recovered (commit 43c52fc is back on my fork). Contacted GitHub Support to reopen, will create a new PR linking here if that doesn't work. Apologies for any confusion! |
|
In addition to |
* enhancement(units): Remove dimension_unit and add default mm for probe * cleanup: remove resolved comment * cleanup: remove resolved comment * cleanup(schema): mark hemisphere__probes for future unification with base hemispher column * cleanup: comments about unifying probes and electrodes column * fix(schema): use um² for microephys electrode size Microelectrodes require micrometer-scale measurements different from iEEG. Added size__microephys with um² units. * fix(schema): remove use of dimension_unit in microephys * cleanup: remove unused surgery_date column (moved as comment to PR #1839) * fix(desc): include microephys for channel column in events * fix(typo): microeelectrode in events * refactor(schema): use base x/y/z columns with addendums for electrodes * change(units): remove unavailble dimension_unit reference and add um in accordance with ProbeInterface norms * fix(docs): add link to markdown for 2D coordinate systems * cleanup: remove resolved TODO comment by 6284d36 * cleanup(rule): remove unsused older check for coordystem file * future cleanup(schema): add a comment of a potential column desc unification * refactor(schema): use base unit column like iEEG for channels
* Add first BEP032 objects * Add ephys modality, start adding _channels but ran into a question on _acq- * Add _acq- to ephys, add _channels and _coords * Draft for probes TSV file * Fix preferably typo * Comment out not yet defined iephys and add ephys to datatypes where it was missing * Added probes file * Describe modality/datatype as "Microelectrode Electrophysiology" otherwise "Electrophysiology" is too broad. Thanks @dorahermes for the note * Remove iephys entirely since I think it was just inspired by DANDI layout not BEP032 * Initial content for ephys document and initial filename template tree * pacify pre-commit * add to TOC * semantic line break and fix * Addressed minor left over lint complains * Add "Microelectrode" specialization for Electrophysiology in the context menu * Fix up for bad latin and adjust wording a bit * Replace fancy unicode “ with simple " . I think this is what might have freaked out `schemacode_ci / windows-latest with Python 3 (pull_request) ` (windows only!) fails with a bunch of ``` 2024-04-19T19:13:44.2815607Z self = <encodings.cp1252.IncrementalDecoder object at 0x0000028D6A14D850> 2024-04-19T19:13:44.2818292Z input = b'---\nHED:\n name: HED\n display_name: HED Tag\n description: |\n Hierarchical Event Descriptor (HED) Tag.\n ...ed or ideal position along the z axis.\n anyOf:\n - type: number\n - type: string\n enum:\n - n/a\n' 2024-04-19T19:13:44.2820386Z final = True 2024-04-19T19:13:44.2820609Z 2024-04-19T19:13:44.2820821Z def decode(self, input, final=False): 2024-04-19T19:13:44.2821664Z > return codecs.charmap_decode(input,self.errors,decoding_table)[0] 2024-04-19T19:13:44.2823260Z E UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 2105: character maps to <undefined> 2024-04-19T19:13:44.2824229Z 2024-04-19T19:13:44.2824768Z C:\hostedtoolcache\windows\Python\3.12.3\x64\Lib\encodings\cp1252.py:23: UnicodeDecodeError ``` * Use redirected to www.datalad.org and www.nwb.org website URLs and few others * BEP032: ephys -> microephys with {icephys, ecephys} modalities/suffixes (#1806) * RF: to have "microephys" (Microelectrode physiology) for modality and icephys and ecephys for suffixes and datatypes * Reflecting decision of having two separate datatypes under the Microelectrode Electrophysiology #1800 (comment) Consensus reached during working group meeting on 2024-05-15: - modality = "Microelectrode Electrophysiology" - datatypes = "icephys" and "ecephys" - suffixes = "_icephys" and "_ecephys" * Adjust wording to Horea's recommendation * Various fixups and tune ups to wording from code review * Clarify wording and do mention `microephys` as shorthand for Microelectrode Electrophysiology * Adding markdown portion of the Bep032 (#1960) * test * MACROS___make_suffix_table * markdown * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * undoing changes to yaml files * adding chanell * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update src/modality-specific-files/microelectrode-electrophysiology.md Co-authored-by: Yaroslav Halchenko <debian@onerussian.com> * Update src/modality-specific-files/microelectrode-electrophysiology.md Co-authored-by: Yaroslav Halchenko <debian@onerussian.com> * Update src/modality-specific-files/microelectrode-electrophysiology.md Co-authored-by: Yaroslav Halchenko <debian@onerussian.com> * Update src/modality-specific-files/microelectrode-electrophysiology.md Co-authored-by: Yaroslav Halchenko <debian@onerussian.com> * Update src/modality-specific-files/microelectrode-electrophysiology.md Co-authored-by: Yaroslav Halchenko <debian@onerussian.com> * Update src/modality-specific-files/microelectrode-electrophysiology.md Co-authored-by: Yaroslav Halchenko <debian@onerussian.com> * Update src/modality-specific-files/microelectrode-electrophysiology.md Co-authored-by: Yaroslav Halchenko <debian@onerussian.com> * adding General ephys metadata * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Examples of real datasets * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Minor tuneups to formatting --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Yaroslav Halchenko <debian@onerussian.com> * [ENH] Schema changes related to BEP032 (#1826) * [FIX] Minor YAML formatting * [ENH] Added additional birthdate column for participants.tsv * [ENH] Added age_category, age_type and corresponding enums * [ENH] Added surgery_date column * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Reverted changes that better suit in a PR against master * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [ENH] Added tabular_data file and columns for _probes.tsv * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [ENH] Added tabular_data rules and columns for _electrodes.tsv * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [ENH] Added tabular_data rules for _electrodes.tsv * [ENH] Added tabular_data rules and columns for _channels.tsv * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [FIX] Fixed pre-commit errors * [ENH] Added additional ephys columns for events.tsv * [ENH] Added tabular data rules for events.tsv to include ephys related columns * [ENH] Added metadata entities for ephys setup metadata * [ENH] Created sidecar rule file for ephys metadata and added data origin and setup fields * Made schema changes to match microephys data type * Renamed rule files from ephys to microephys * Made schema changes to match microephys data type * [ENH] Added objects and rules for coordinate system sidecar file * [ENH] Added processing microephys metadata field in rules * [ENH] Added pharmaceuticals microephys metadata field in rules * [ENH] Added supplementary microephys metadata field and rules * [ENH] Added sample microephys metadata rules * [ENH] Added task microephys metadata rules * Fixed example language convention * Added microephys in schema rule modality * Corrected SampleThickness to SliceThickness * Add yaml document separator for microephys rules file It is optional, and adding does not solve anything but makes it consistent with the other files in the folder * BF: should be a dict, not a list of dicts --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Yaroslav Halchenko <debian@onerussian.com> * Some fixes of formatting and URLs * Adjusted some formatting and some wording at the beginning of the section * Few fixes to the schema from review Co-authored-by: Peyman Najafi <najafe.peyman@gmail.com> * Adopting the markdown line lengths for BEP032 (#2012) * cutting line lenghts * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * marking json examples * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * microephysChannels --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix tabulation in a .tsv + convert to use yaw, pitch, roll Co-authored-by: Ben Dichter <ben.dichter@gmail.com> * Add types and min/max for roll,pitch,yaw + fix typo for microephysChannels * Replace Pharmaceuticals with already existing fields and fix few more references * Make use of merged fancing for tsvs and add a note about spotted Pharmaceuticals * Remove some "bad latin" * Fix a link to be final, before redirects * using MACROS___make_sidecar_table * numbering * Adding icephys and ecephys to filename template * [ENH] Added microephys tag to electrode related enums * [ENH] Added additional channel enums for microphys * [ENH] Enlisted microphys channel types in type__channels * Various small fixes from code-review Co-authored-by: Lyuba Zehl <lyuba.zehl@ebrains.eu> Co-authored-by: Ben Dichter <ben.dichter@gmail.com> * [BEP032] Moved coord system related entities to correct yaml * Minor tuneups from code review (mostly on wording) Co-authored-by: Ben Dichter <ben.dichter@gmail.com> * A pass to harmonize to use "microephys" in favor over "ephys" Also used appropriate suffix in a few spots when encountered * Encode in schema that ATM we do require _electodes and _probes Also extended for microephys for channels and coordsystems * One more ephys -> microephys * Add .json to data (nwb and nix) and _probes files * [DATALAD RUNCMD] replace fancy unicode double quote with a simpler one === Do not change lines below === { "chain": [], "cmd": "sed -i -e 's,[“”],\",g' src/modality-specific-files/microelectrode-electrophysiology.md", "exit": 0, "extra_inputs": [], "inputs": [], "outputs": [], "pwd": "." } ^^^ Do not change lines above ^^^ * Address many remark lint issues using npx remark src/modality-specific-files/microelectrode-electrophysiology.md --frail --rc-path .remarkrc --output and then subselecting some changes and fixing some manually * One more tune up * items * [DATALAD RUNCMD] Fix up itemized things indentation === Do not change lines below === { "chain": [], "cmd": "sed -i -e 's,^\\([0-9]\\.\\|- \\) *,\\1 ,g' src/modality-specific-files/microelectrode-electrophysiology.md", "exit": 0, "extra_inputs": [], "inputs": [], "outputs": [], "pwd": "." } ^^^ Do not change lines above ^^^ * Address leftover of remark lint warnings * Remove custom paragraph on participants, replaced angles with yaw, pitch, roll Co-authored-by: Lyuba Zehl <lyuba.zehl@ebrains.eu> * [FIX] BEP032 - Correcting validations against real data example (#2132) * fix: schemapath resolution in Windows dev mode installation of bidsschematools * fix: schema validation through testing * chore: resolve conflict * fix: validation rules for events * doc: Reorder required metadata files for microephys sessions and clarify channel information * remove unneeded fields * move to task.yaml with __ * remove from microephys * Update src/modality-specific-files/microelectrode-electrophysiology.md Co-authored-by: Yaroslav Halchenko <debian@onerussian.com> * chore: linting * Add comprehensive stereotaxic coordinate documentation with visual guides - Replace x,y,z coordinates with standardized AP/ML/DV system - Add anatomical reference points (Bregma/Lambda) explanations - Include detailed angle measurement conventions with diagrams - Add 6 illustrative images for coordinate system understanding - Update electrode table example to use new coordinate naming * Add probe position columns for AP and ML axes and standardize units - Add AP__probes column for Anterior-Posterior probe positioning - Add ML__probes column for Medial-Lateral probe positioning - Add missing unit specification (mm) to existing DV__probes column * Update microelectrode electrophysiology documentation - Add ProbeInterface format links and JSON schema requirements for custom probes - Standardize requirement language (MUST/SHOULD) for coordsystem fields - Remove deprecated ProbeContours section * Updated probe tabular rule to match AP, ML, DV * Removed coordystem completely * Removes coordsys macro to fix failing mkdocs build * change the electrode positions back from AP/ML/DV to x,y,z * Update _probes.tsv example with standardized coordinate system and ProbeInterface fields Replace x/y/z coordinates with AP/ML/DV notation, add angle fields, and include probeinterface_manufacturer/model columns for better compatibility with ProbeInterface library * Update microephys entity rules and remove task/run requirements * Fix linter issues * Reorganized to match other modalities * Clarify DV axis description in microelectrode electrophysiology documentation * Update src/modality-specific-files/microelectrode-electrophysiology.md * Adds space entity for microephys electrodes * Add coordinate system metadata fields for microephys * Added rules to 1) override rule for coorsystem to be present for electrode.tsv 2) If coorsystem present, corresponding electrode.tsv must be present 3. Match the space between corresponding coordystem and electrodes files * Added entities for the photo specific to microephys * Rewrote the coordsystem JSON and photo section to align with ieeg but adopt to animal electrophysiology * Minor change to the default probe relative coordinate system * Probe interface exaple file tree * Add omission of microephys datatypes for overall channel file rule * Update src/modality-specific-files/microelectrode-electrophysiology.md * Update src/modality-specific-files/microelectrode-electrophysiology.md * Update src/modality-specific-files/microelectrode-electrophysiology.md * Fix indentation for note and example alerts * [MISC] Use Levels and custom json for probe interface files * Update src/modality-specific-files/microelectrode-electrophysiology.md * Update src/schema/objects/columns.yaml * Update src/modality-specific-files/microelectrode-electrophysiology.md * [MISC] Replace image syntax with HTML for anatomical reference diagrams * Fix formatting in example *_electrodes.tsv for improved readability * Apply suggestion from @bendichter * Remove channel_id def and use name__channels same as ieeg * Update microephys electrode.tsv to match ieeg * Changed probe_id to probe_name to avoid use of the concept ID against BIDS conventions * Add missing probe_height def and rewrite desc to match current markdown * Fixed minor typo failing schema validation * Fix remark linting issues * Added and modified texts for coordsystem requirements * Added icephys and ecephys distinction and restructured first section a little * Fixed initial_column to match AP, ML, DV * Update mock TSV and JSON and fix heading levels to match ieeg * Updated toy data section and links to example to match current spec * Minor formatting and style issues * Fix mock data tsv and JSON * Apply various wording suggestions from code review Co-authored-by: Ben Dichter <ben.dichter@gmail.com> * Apply suggestion from @yarikoptic Co-authored-by: Yaroslav Halchenko <debian@onerussian.com> * Apply suggestion from @bendichter * add back type info that was accidentally removed from whole_blood_radioactivity * Update link to iEEG stimulation documentation for clarity * Update src/modality-specific-files/microelectrode-electrophysiology.md * feat: add anatomical reference point to microelectrode electrophysiology schema * Fix up units and clarify that we need TODO work on the units/space more * Fix typo Update src/schema/objects/suffixes.yaml fixing a typo * rf: move "ProbeInterface Library" subsection down after probes.tsv * Revert "Apply suggestion from @bendichter" to bring back width__probes This reverts commit d1835cf . spotted by seeing na na in the table. We have height and depth so makes sense to keep width, and now we get ❯ git grep __probes -- src/schema/objects/columns.yaml src/schema/objects/columns.yaml:AP__probes: src/schema/objects/columns.yaml:depth__probes: src/schema/objects/columns.yaml:DV__probes: src/schema/objects/columns.yaml:height__probes: src/schema/objects/columns.yaml:hemisphere__probes: src/schema/objects/columns.yaml:manufacturer__probes: src/schema/objects/columns.yaml:material__probes: src/schema/objects/columns.yaml: name: material__probes src/schema/objects/columns.yaml:ML__probes: src/schema/objects/columns.yaml:type__probes: src/schema/objects/columns.yaml:width__probes: Ideally the rendered should have failed and not just added na's row but that shouldn't be fixed here... * docs: consolidate electrode coordinate system documentation Reorganize coordinate system documentation in microelectrode electrophysiology spec by removing standalone "Coordinate Systems for Electrode Positions" section and consolidating the information into a new "Electrode Position Coordinates" subsection under the electrodes description. This improves document flow and places coordinate guidance closer to the relevant table definition. * docs(appendices): add microelectrode surgical coordinates appendix Add new appendix documenting stereotaxic coordinate conventions for intracranial probe placement during surgery, including: - Anatomical reference points (Bregma and Lambda) - AP/ML/DV coordinate system conventions - Angle definitions for probe orientation (AP, ML, rotation) - Reference to BrainSTEM documentation source * Add probes related schema changes * docs: add microelectrode electrophysiology coordinate systems Add new section defining restricted keywords for MicroephysCoordinateSystem field in coordsystem.json for icephys and ecephys datasets. Includes: - Pixels (2D electrode localization) - Stereotaxic (generic animal neuroscience coordinates) - AllenCCFv3 (mouse brain reference space) - WaxholmSpace (rat brain coordinate system) - WistarRatAtlas * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * docs: convert RRID identifiers to hyperlinks in coordinate-systems * docs(ephys): add stream_id column documentation for channels.tsv Document the stream_id column format for linking channels to their corresponding data streams. Include specifications for: - NWB files with internal HDF5 paths to ElectricalSeries objects - NIX files referencing data arrays following NIX/Neo organization - Handling multiple data streams with comma-separated lists - Cross-file references using filename:path syntax * docs(ephys): clarify electrode name uniqueness requirement * Changed hemisphere cardinality from required to recommended * fix: correct typo in description for reference__microephys * fix: clarify electrode position coordinates for single electrode cases * fix: fix the name of matrial: material_probes -> material * fix: remove unnecessary brackets in coordsystem selectors * fix: change tables from pipe syntax to html syntax. This removes linter errors and is easier to maintain. * Fix yamllint and codespell issues * feat(microephys): add channel-level filtering columns and documentation - Add new Filtering Information section to microelectrode-electrophysiology.md explaining global and channel-specific filter parameters - Add software_filter_types column definition to columns.yaml for specifying filter types with Levels defined in channels.json - Add low_cutoff, high_cutoff, and notch optional columns to microephysChannels for channel-specific filter cutoff frequencies (consistent with iEEG spec) - Replace hardware_filters and software_filters columns with the new software_filter_types column in microephysChannels table * fix: add missing closing brace in JSON example for task description * fix: update description of MicroephysCoordinateSystem axes and link to surgical coordinates appendix * feat(schema): add model column for probes and fix electrode descriptions - Add new `model__probes` column to store probe model name/number (e.g., Neuropixels 1.0, A1x32-Poly3-5mm-25s-177) - Mark model__probes as recommended in microephysProbes rules - Fix typos in x__electrodes description ("there in" → "therein") - Clarify coordinate units reference in electrode position description * fix: update description for IntendedFor field in microephysCoordsystemGeneral * fix: correct name and type for associated_brain_region_id field in columns.yaml * fix: remove deprecated hardware_filters and software_filters columns in extracellular electrophysiology example * feat: add micrometer (um) as a valid unit for MicroephysCoordinateUnits * fix(schema): correct stereotaxic coordinate system axis definitions Update axis definitions in Stereotaxic enum to fix incorrect mappings: - X-axis (AP): Change positive direction from posterior to anterior - Y-axis: Change from DV to ML axis with right being positive - Z-axis: Change from ML to DV axis with ventral/inferior being positive * feat(microephys): add recording metadata fields to setup schema Add RecordingDuration, RecordingType, and EpochLength fields to the microephysSetup sidecar rules. RecordingDuration and RecordingType are recommended, while EpochLength is optional but recommended when RecordingType is "epoched". * docs(ephys): correct JSON metadata filename convention Update microelectrode electrophysiology documentation to use the correct file suffixes `*_ecephys.json` or `*_icephys.json` instead of the generic `*_ephys.json` to match the actual naming convention for extracellular and intracellular electrophysiology metadata files. * fix: correct pluralization in TODO comment for PharmaceuticalDoseUnits * fix(schema): update electrode position descriptions for clarity and consistency with space entities * fix(schema): rename x/y/z columns to x/y/z__electrodes in microephys Rename coordinate columns in microephysElectrodes table to include __electrodes suffix for disambiguation. Updates both initial_columns and columns sections to use the new naming convention. * docs(ephys): update coordinate system names and fix Stereotaxic spelling - Change StereoTaxic to Stereotaxic for correct scientific terminology - Replace PaxinosWatson with WaxholmSpace and WistarRatAtlas coordinate systems - Remove outdated TODO comment about adding coordinate systems to appendix - Update file naming examples to use corrected Stereotaxic spelling * feat(microephys): add brain atlas coordinate systems for rodents and primates Add new MicroephysCoordinateSystem options for commonly used stereotaxic atlases: - PaxinosWatson: rat brain atlas - FranklinPaxinos: mouse brain atlas - SwansonRat: rat brain atlas - CHARM: macaque cortical hierarchy atlas - D99: Saleem-Logothetis macaque atlas - PaxinosMacaque: macaque stereotaxic atlas - MarmosetBrainAtlas: marmoset brain atlas These additions expand support for microelectrophysiology recordings across multiple species with widely-used reference coordinate systems. * fix(linter): fix yamllint and remark errors * fix(schema): remove unused column defintions * Reverted commit and readded the removed type column for software_filters * fix: reconciling software_filter defintion with master * Update src/schema/objects/metadata.yaml * Update src/modality-specific-files/microelectrode-electrophysiology.md Co-authored-by: Reema Gupta <59512969+ree-gupta@users.noreply.github.com> * fix: microephys electrodes/coordsystem validation with space entity support * fix(convention): update the schema selector convention from == to using intersects * fix(convention): use __ instead of _ for microephys context * cleanup(schema): remove old unused probe suffix * cleanup(schema): remove unsused column contact_count * fix(convention): fix display name capitalization and hypenation * [BEP032] Review and clean-up suggestions (#2293) * enhancement(units): Remove dimension_unit and add default mm for probe * cleanup: remove resolved comment * cleanup: remove resolved comment * cleanup(schema): mark hemisphere__probes for future unification with base hemispher column * cleanup: comments about unifying probes and electrodes column * fix(schema): use um² for microephys electrode size Microelectrodes require micrometer-scale measurements different from iEEG. Added size__microephys with um² units. * fix(schema): remove use of dimension_unit in microephys * cleanup: remove unused surgery_date column (moved as comment to PR #1839) * fix(desc): include microephys for channel column in events * fix(typo): microeelectrode in events * refactor(schema): use base x/y/z columns with addendums for electrodes * change(units): remove unavailble dimension_unit reference and add um in accordance with ProbeInterface norms * fix(docs): add link to markdown for 2D coordinate systems * cleanup: remove resolved TODO comment by 6284d36 * cleanup(rule): remove unsused older check for coordystem file * future cleanup(schema): add a comment of a potential column desc unification * refactor(schema): use base unit column like iEEG for channels * Update src/modality-specific-files/microelectrode-electrophysiology.md Co-authored-by: Cody Baker <51133164+CodyCBakerPhD@users.noreply.github.com> * Update src/modality-specific-files/microelectrode-electrophysiology.md Co-authored-by: Thomas Wachtler <wachtler@bio.lmu.de> * Update src/modality-specific-files/microelectrode-electrophysiology.md Co-authored-by: Thomas Wachtler <wachtler@bio.lmu.de> * Update src/modality-specific-files/microelectrode-electrophysiology.md Co-authored-by: Thomas Wachtler <wachtler@bio.lmu.de> * Update src/modality-specific-files/microelectrode-electrophysiology.md Co-authored-by: Thomas Wachtler <wachtler@bio.lmu.de> * enh(schema): add electrode_name, make reference optional and generalize probe_name * Remove dimension_unit from electrode table description * Remove redundant statement about required columns order in *_electrodes.tsv file * move description of electrode position columns in microelectrode electrophysiology documentation * cleanup: remove additions to events that are not microephys specific * Remove outdated comments and streamline pharmaceuticals section in microelectrode electrophysiology documentation * update source attribution to BrainSTEM * fix(linter): fix remark linter error * fix(build): add missing closing ticks for TSV in md * Update src/modality-specific-files/microelectrode-electrophysiology.md Co-authored-by: Thomas Wachtler <wachtler@bio.lmu.de> * Update src/modality-specific-files/microelectrode-electrophysiology.md Co-authored-by: Thomas Wachtler <wachtler@bio.lmu.de> * docs(ephys): add electrode_name column to channels table example Add electrode_name column to the extracellular electrophysiology channels.tsv example to show the mapping between channels and their corresponding electrodes. * fix(microephys): move reference__microephys to be optional in channels schema * move reference__microephys * fix(coordinate-systems): update link format for Microelectrode Surgical Coordinates appendix to markdown * fix: Correct formatting in electrode data table for clarity * fix: Correct electrode name formatting in electrophysiology examples for consistency * docs: correct coordinate system descriptions and references - Fix WaxholmSpace species from rat to mouse and update DOI reference - Remove RRID links from SwansonRat, CHARM, D99, and MarmosetBrainAtlas - Add DOI link for SwansonRat reference - Correct PaxinosRhesusMonkey author list (Toga AW -> Evrard, H) * fix: Add ignore rule for SciCrunch URLs in link checker --------- Co-authored-by: Julia Sprenger <julia.sprenger@rwth-aachen.de> Co-authored-by: Yaroslav Halchenko <debian@onerussian.com> Co-authored-by: Horea Christian <chr@chymera.eu> Co-authored-by: Remi Gau <remi_gau@hotmail.com> Co-authored-by: Peyman Najafi <najafe.peyman@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Reema Gupta <59512969+ree-gupta@users.noreply.github.com> Co-authored-by: Ben Dichter <ben.dichter@gmail.com> Co-authored-by: Reema Gupta <reema.gupta@lmu.de> Co-authored-by: Lyuba Zehl <lyuba.zehl@ebrains.eu> Co-authored-by: Chris Markiewicz <markiewicz@stanford.edu> Co-authored-by: christinerogers <christinerogers@users.noreply.github.com> Co-authored-by: Thomas Wachtler <wachtler@bio.lmu.de>
|
@effigies this PR was closed by accident. could you please reopen it again so that we may rescue the discussions and open comments for fixing this? |
|
No, unfortunately. There's nothing to reopen. 
I'm not sure if this is relevant to why this happened, but I strongly recommend against using a |
Thanks for looking into it and good advice (something I learned the hard way) :) This is exactly why it happened. This was one of my first PRs to BIDS, I then started working on a lot of other things and completely missed that the commits on the master branch were useful and accidentally discarded them. I read up a bit more on git practices and how working on the master can cause so many issues for a collaborative projects like BIDS. Opened a new PR #2340 |
The PR adds columns for
age_category,age_type,birthdateas discussed and suggested by @yarikoptic in #1826This is open for discussion.
Related issues:
@bids-standard/bep032
TODO:
AgeReferencemust be used in sidecar JSONAgeReferencerule in schema forparticipant.tsvagecolumn definition that would specify that one must useAgeReferencein sidecar?birthdateoragemust be specified