Skip to content

Add flag to write out all metadata columns in pandas data frame #232

@KirstieJane

Description

@KirstieJane

At the moment (as shown in the tutorial) there's a way to save out the layout of the dataset as a pandas frame.

In discussion at Neurohackademy, @kdestasio and I were talking with @tyarkoni about whether it was possible to keep track of individual research groups keeping notes about (for example) the quality control assessments of individual scans. These notes could be stored in .json sidecars next to the files. For example, the key KirstieInspected could include free text notes on whether I've inspected the files.

This functionality probably shouldn't be incorporated into the BIDS specification - or if it does it should be a BIDS specific discussion (and the key should definitely not be called KirstieInspected 🤣) rather than something that pybids implements without discussion

However there is quite a nice flag that could be added to pybids (suggested by @tyarkoni) to add a flag to the df = layout.as_data_frame() command to add all metadata stored in the .json files to that dataframe.

It would be off by default, but example usage would be layout.as_data_frame(all_metadata=True) and it would add a column for every metadata key that appears in the whole layout. KirstieInspected which would contain NaNs if the key wasn't in a sidecar file, or contain the free text notes if it did, and this would be the same behaviour for other keys (RepetitionTime, EchoTime, Manufacturer etc).

I've had a quick look through the code and I can't find this particular command (sorry!) so I'm not a great person to make a start. But happy to clarify the goal if needed, and can take instruction if that's helpful ✨

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions