diff --git a/docs/source/pages/community/community.md b/docs/source/pages/community/community.md index a18c1850..d56be7c0 100644 --- a/docs/source/pages/community/community.md +++ b/docs/source/pages/community/community.md @@ -8,7 +8,7 @@ during or after a contribution. See the [Contributing Guide](contributing.md) fo We are in general very keen to hear feedback on what works and doesn't work in **datashuttle** and -[NeuroBlueprint](https://neuroblueprint.neuroinformatics.dev) +[NeuroBlueprint](https://neuroblueprint.neuroinformatics.dev). Please get in contact with any thoughts on how these tools can be improved by opening a [GitHub Issue](https://github.com/neuroinformatics-unit/datashuttle/issues) @@ -16,7 +16,7 @@ or at our [Zulip Chat](https://neuroinformatics.zulipchat.com/#narrow/stream/405999-DataShuttle). To see what we have planned in future for **datashuttle**, visit -our [Principles and Roadmap](principles-and-roadmap.md). +our [Guiding Principles](guiding-principles.md) and [Roadmap](roadmap.md). ```{toctree} @@ -25,5 +25,6 @@ our [Principles and Roadmap](principles-and-roadmap.md). :hidden: contributing +guiding-principles roadmap ``` diff --git a/docs/source/pages/community/contributing.md b/docs/source/pages/community/contributing.md index 21700fa7..3f660f40 100644 --- a/docs/source/pages/community/contributing.md +++ b/docs/source/pages/community/contributing.md @@ -10,7 +10,7 @@ and [opening a pull request](pull-requests). The core development team will support you in contributing code, irrespective of your experience! -### Creating a development environment +## Creating a development environment To install **datashuttle** along with all packages necessary for development, see the 'Developers' tab on the @@ -43,7 +43,7 @@ pre-commit install ``` -### Pull requests +## Pull requests In all cases, please submit code to the main repository via a pull request. The developers recommend, and adhere, to the following conventions: @@ -72,7 +72,7 @@ with both the design and implementation of tests so please don't let this stop you opening a PR. We take a similar approach to [documenting new features](#contributing-documentation), discussed further below. -### Formatting and pre-commit hooks +## Formatting and pre-commit hooks Running `pre-commit install` will set up [pre-commit hooks](https://pre-commit.com/) to ensure a consistent formatting style. Currently, these include: * [ruff](https://github.com/astral-sh/ruff) does a number of jobs, including code linting and auto-formatting. diff --git a/docs/source/pages/community/guiding-principles.md b/docs/source/pages/community/guiding-principles.md new file mode 100644 index 00000000..bbcf4237 --- /dev/null +++ b/docs/source/pages/community/guiding-principles.md @@ -0,0 +1,64 @@ +# Guiding Principles + +In this document we lay out our vision for +[**NeuroBlueprint**](https://neuroblueprint.neuroinformatics.dev/) +and **datashuttle** in three guiding principles. + +For more detail, please see our blog posts motivating +[NeuroBlueprint](https://neuroinformatics.dev/blog/neuroblueprint.html) +and +[**datashuttle**](https://neuroinformatics.dev/blog/datashuttle.html). + +The three guiding principles in the development of +**NeuroBlueprint** +and **datashuttle** are: + +## 1) Align as best possible with existing community initiatives (BIDS and NWB) + +[BIDS](https://bids.neuroimaging.io/) +and +[NWB](https://www.nwb.org/) +are the gold-standard for standardising systems neuroscience projects. +Alignment with these specifications should always be the final aim for a project. + +The role of **NeuroBlueprint** is a lightweight standard that can be used to get +started, especially during the early phase of a project when you may be very busy. +The guiding rule is that some standardisation is better than no standardisation. + +It is not a rival or replacement for more developed specifications, +but a stepping stone towards full standardisation. +We aim to align with BIDS specification and in future to provide +conversion to NWB through **datashuttle** wherever possible. + +## 2) Strive to be as lightweight as possible + +In keeping with **(1)**, **NeuroBlueprint** aims to keep the specification as lightweight +as possible. There is no point in the standard proliferating in scope such that it becomes +a duplicate of BIDS and NWB. + +In the first version (**datashuttle 0.4, **NeuroBlueprint 0.2), +the goal is to have a simple organisational +system in which different datatypes (ephys, behaviour) can be +automatically found in any project. + +In future versions as we extend the sophisiatication of automated analysis it will +be necessary to standardise additional features (e.g. sync pulse metadata). +These restrictions will always be goal-orientated and cumstomisable +to achieve your research goal. It is hoped as these 'rules' are +accumulated projects become closer and closer to full BIDS standardisation. + +The role of **datashuttle** will be to enforce these rules in a flexible way. + +## 3) Be versioned and modular + +Linked to **(2)**, **NeuroBlueprint** and **datashuttle** will be +properly versioned and modular, allowing them to develop smoothly over time. +The release of a new versions will strive hard to be backwards compatible, +only breaking backwards compatibility infrequently to improve alignment +with existing standards. + +Typically, new versions will only include new +sets of standardisation rules that +you may or may not choose to follow. **datashuttle** will +enforce these in a flexible way and care will be taken to ensure its API +changes as little as possible. diff --git a/docs/source/pages/community/principles-and-roadmap.md b/docs/source/pages/community/principles-and-roadmap.md deleted file mode 100644 index 4486eb90..00000000 --- a/docs/source/pages/community/principles-and-roadmap.md +++ /dev/null @@ -1,85 +0,0 @@ -# Principles and Roadmap - -In this document we lay out our vision for **datashuttle**, what -it aims to do and it's place in the data standardisation landscape. - - -## Principles of **datashuttle** - -The three guiding principles in the development of **datashuttle** are: - -1) It is not a rival or replacement for more developed specifications, -[BIDS](https://bids.neuroimaging.io/) -and -[NWB](https://www.nwb.org/). -It is instead a lightweight specification to get -started quickly with a standardised framework. BIDS / NWB are the gold -standard for standardised research outputs that should always be the final aim for a project. - -2) It will require only the minimum amount of standardisation required -for specific goals. - -3) It will be versioned and modular, meaning you can start with minimal -standardisation (and errort) and build up depending on your requirements. - - -### 1) Not a rival to existing standardisation / Always use existing standarards / align as closely as possible. - -It is not a rival or replacement for more developed specifications, -[BIDS](https://bids.neuroimaging.io/) -and -[NWB](https://www.nwb.org/). - -It is instead a lightweight specification to get -started quickly with a standardised framework. BIDS / NWB are the gold -standard for standardised research outputs that should always be the final aim for a project. - -The guiding rule is that some standardisation is better than no standardisation. -If you are busy, you can use NeuroBlueprint and datashuttle to maintain -standardisation in your project folder, and consider extension to full -standardisation later. If you are already acquiring data in a fully standardised -format, even if it is not NWB or BIDS, then NeuroBlueprint / **datashuttle** -is not for you. - - -### 2) It will be as minimalistic as possible sufficient for automated preprocessing and analysis - -[NeuroBlueprint]() aims to be a lightweight specification. As times goes on, -we will be careful to maintain few restrictions as possible to keep it's purpose -clear and avoid simple duplication of existing, more detailed standards. - -Developed by the Neuroinformatics unit, our primary objective in standardisation -is to faciliatte automated preprocessing and analysis pipelines. NeuroBlueprint -and DataShuttle will be structured in a way that new restrictions will -be added only if they are required for the automation of preprocessing and -analysis pathways. Further, they will be optional based on the kind of analysis you will run. - -In version 1XX, the initial goal was simple: 'have project folders standardised -sufficiently that it is clear where all collected data is and stored predictably -across projects'. This alone opens many doors for automated preprocessing -of data. - -Later, we will want to support additional analysis that will require additional -standardisation. For example, you may be running an experiment -in which you need to align timestamps across behavioural and electrophysiological data. -For automated analysis, metadata standards for syncrnousatio pulses will be required. -We will add only the requirements required for this goal and document them in -a clear way that only IF you want to do such automated analysis THEN -you should use this rule. - -We hope by an objective-based approach projects will incrementally move -towards full BIDS and NWB standardisation across their projects. - -### It will be versioned and modular - -Linked strongly to point 3) we will ensure that datashuttle and -NeuroBlueprint are designed in a modular way. - - -## Roadmap for the future - -- metadata -- working with BIDS / NWB to align as closely as possible -- support different experiment types -- maintain and add quality of life features as requested. -- add docstring versioning to datashuttle and NeuroBlueprint docs prior to new version release. diff --git a/docs/source/pages/community/roadmap.md b/docs/source/pages/community/roadmap.md new file mode 100644 index 00000000..600ebf8a --- /dev/null +++ b/docs/source/pages/community/roadmap.md @@ -0,0 +1,22 @@ +# Roadmap + +Here we lay out the roadmap for +[NeuroBlueprint](https://neuroblueprint.neuroinformatics.dev/) +and **datashuttle**. The roadmap is largely determined by needs +of the community and we strongly encourage people to get in +contact to outline their needs and priorities! + +The aim of the first version of **NeuroBlueprint** (v0.2) and **datashuttle** (v0.4) +are to provide a lightweight, standardised framework such that raw data +and derivatives of different data types (e.g. behaviour, electrophysiology) +can be automatically found. + +The next aim will be to support more advanced multimodal analysis, requiring +metadata (e.g. sync pulses). The next version will focus on recommending +a general metadata standard based on those already used in the community, +and specifying specific metadata keys for the aim of automating multimodal analysis. + +Aiming for integration with existing standards, we plan to develop +automated methods for converting raw data files stored in a NeuroBlueprint +project to NWB. We will constantly strive for full alignment with the BIDS +standard and update when possible as BIDS expands into systems neuroscience.