BIDS: IP-Freely

[Lestropie]

Title

Free manipulation of BIDS datasets according to the Inheritance Principle

Leaders

Robert E. Smith; https://x.com/Lestropie

Collaborators

No response

Brainhack Global 2024 Event

Brainhack Aus

Project Description

The Inheritance Principle ("IP") has been a feature of the Brain Imaging Data Structure (BIDS) since its inception. It describes how the metadata relevant to one specific data file may in fact originate from multiple metadata files. Encoding some common metadata field just once, and having it be deemed applicable to multiple metadata files, may both reduce redundancy and communicate the intrinsic hierarchical structure of the dataset. This however comes at considerable complexity to the specification itself and the software responsible for interfacing with such data. Indeed nearly a decade after its creation, many software APIs for parsing BIDS data still do not support the Inheritance Principle.

The future of the IP is at a crossroads, particularly as work progresses toward BIDS 2.0:

• On one hand, some would like for the IP to disappear entirely: all metadata relevant to any given data file would need to be stored in a sidecar metadata file alongside that data file, regardless of duplication.
• On the other, not just preserving the IP but augmenting it could better communicate the complex hierarchical dependencies of complex datasets, particularly derivative data.

The goals of this project are twofold.

1. There is a dearth of existing software tools for managing the IP. If a dataset that exploits the IP were provided as input to a BIDS App that is naive to the IP, that App could perform improper association of metadata. Existing datasets may contain substantial metadata redundancy, due to not currently having an easy way through which to exploit the IP. A tool to automate such data manipulations could therefore be useful in the BIDS ecosystem.

2. Consequential decisions regarding the IP and related issues in BIDS 2.0 would be better informed if stakeholders could see the effects that proposed specifications would have on how compliant datasets appear.

Link to project repository/sources

https://github.com/Lestropie/IP-freely

Goals for Brainhack Global

• Primary goal: Produce one or more exemplar BIDS datasets that exemplify the complexity of the Inheritance Principle, which can therefore be used to demonstrate the influence of its removal or exploitation under different rule sets: Create synthetic exemplar datasets Lestropie/IP-freely#5

• Primary goal: Take a BIDS dataset that may include use of the Inheritance Principle, and create a new version of that dataset that removes all usage of the Inheritance Principle: Operation: Remove use of Inheritance Principle Lestropie/IP-freely#1

• Primary goal: Take a BIDS dataset that may (but likely does not) include use of the Inheritance Principle, and create a new version of that dataset that makes maximal use of the Inheritance Principle (without overloading): Operation: Maximal use of Inheritance Principle; first pass Lestropie/IP-freely#2

• Secondary goals: Other issues listed at https://github.com/Lestropie/IP-freely/issues.

Good first issues

For anyone not accustomed to open source projects, perhaps the most accessible entrypoint will be the construction of one or more exemplar datasets: Lestropie/IP-freely#5

Communication channels

TBA

Skills

• Python: Moderate

  The code logic for manipulating metadata will necessitate some competence with Python dictionaries; nothing super complex, but maybe not accessible for someone who has never touched them.
  Expect to make use of the pathlib module.
  If any contributor were to have existing expertise with pybids that may also be beneficial.

• BIDS: Variable

  The necessary fundamentals of BIDS are just filesystem structure (directories, entities and suffices) and the idea of a metadata dictionary.
  However creation of exemplar datasets would be facilitated by someone who:

  • Has experience with BIDS across multiple modalities
  • Has some familiarity with the Inheritance Principle.

Onboarding documentation

Current status of the Inheritance Principle:
https://bids-specification.readthedocs.io/en/stable/common-principles.html#the-inheritance-principle

Original context that motivated pursuit of augmentation of the Inheritance Principle:
bids-standard/bids-bep016#50

Previously proposed (but rejected) extension of the Inheritance Principle:
bids-standard/bids-specification#1003
(and other threads linked therein)

Existing repository that contains data-empty exemplar datasets for the purpose of validation:
https://github.com/bids-standard/bids-examples

Discussion regarding potential modification of Inheritance Principle for BIDS 2.0:
bids-standard/bids-2-devel#65

What will participants learn?

• Experience building a piece of open source scientific software from scratch

• Basic algorithmic design for the various manipulations of metadata

• Understanding correspondence between specification rules as described in plaintext language and programmatic implementation

• Can provide an introduction to the basics of Git for any participants not already familiar with such.

Data to use

Proper evaluation of the proposed tool will necessitate construction of one or more novel datasets.

Some insight may be gained from executing the tool against any existing BIDS datasets, whether public or private. Of particular interest would be any datasets known to already be utilising the Inheritance Principle.

Number of collaborators

1

Credit to collaborators

Intend to set up the GitHub all-contributors bot for this new Project.

Depending on scope and eventual adoption, publication in the Journal of Open Source Science (JOSS) may be pursued.

Image

Leave this text if you don't have an image yet.

Type

coding_methods, data_management

Development status

0_concept_no_content

Topic

reproducible_scientific_methods

Tools

BIDS

Programming language

Python

Modalities

behavioral, DWI, ECG, ECOG, EEG, eye_tracking, fMRI, fNIRS, MEG, MRI, PET, TDCS, TMS

Git skills

0_no_git_skills

Anything else?

No response

Things to do after the project is submitted and ready to review.

• Add a comment below the main post of your issue saying: Hi @brainhackorg/project-monitors my project is ready!
• Twitter-sized summary of your project pitch.

[Lestropie]

Twitter-sized pitch:

In the Brain Imaging Data Structure (BIDS), the Inheritance Principle allows sharing of mutual metadata. It is however a source of uncertainty and division. Here we will build a tool for modifying BIDS datasets under the rules of the Inheritance Principle, improving robustness of the current ecosystem and informing BIDS 2.0 development.

[Lestropie]

Hi @brainhackorg/project-monitors my project is ready!

[crnolan]

Looks good!