Add 'Electrophysiology' page.

.github/workflows/docs_build_and_deploy.yml

@@ -28,6 +28,8 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: neuroinformatics-unit/actions/build_sphinx_docs@v2
+        with:
+          python-version: 3.12
 
   deploy_sphinx_docs:
     name: Deploy Sphinx Docs

.gitignore

@@ -1,3 +1,6 @@
+docs/source/ephys_at_swc/example_pipelines/auto_examples_ephys/
+sg_execution_times.rst
+
 # Byte-compiled / optimized / DLL files
 __pycache__/
 *.py[cod]

docs/requirements.txt

@@ -1,8 +1,11 @@
 linkify-it-py
+matplotlib
 myst-parser
 nbsphinx
 numpydoc
 pydata-sphinx-theme
+
 sphinx
 sphinx-copybutton
 sphinx-design
+sphinx-gallery

docs/source/conf.py

@@ -42,6 +42,7 @@
     "myst_parser",
     "numpydoc",
     "nbsphinx",
+    "sphinx_gallery.gen_gallery",
 ]
 
 # Configure the myst parser to enable cool markdown features
@@ -74,6 +75,10 @@
     # to ensure that include files (partial pages) aren't built, exclude them
     # https://github.com/sphinx-doc/sphinx/issues/1965#issuecomment-124732907
     "**/includes/**",
+    # exclude .py and .ipynb files in auto_examples generated by sphinx-gallery
+    # this is to prevent sphinx from complaining about duplicate source files
+    "ephys_at_swc/example_pipelines/auto_examples_ephys/*.ipynb",
+    "ephys_at_swc/example_pipelines/auto_examples_ephys/*.py",
 ]
 
 # Ignore certain URLs from being checked
@@ -91,6 +96,7 @@
 linkcheck_anchors_ignore_for_url = [
     "https://gin.g-node.org/G-Node/Info/wiki",
     "https://gin.g-node.org/G-Node/info/wiki",  # ignore both spellings
+    "https://github.com/stephenlenzi/npix_lse",  # ignore private repository
 ]
 linkcheck_retries = 2
 
@@ -148,3 +154,9 @@
 # Configure the code block copy button
 # don't copy line numbers, prompts, or console outputs
 copybutton_exclude = ".linenos, .gp, .go"
+
+# Configure sphinx-gallery
+sphinx_gallery_conf = {
+     'examples_dirs': ["ephys_at_swc/example_pipelines/examples"],   # path to your example scripts
+     'gallery_dirs': ["ephys_at_swc/example_pipelines/auto_examples_ephys"],    # path to where to save gallery generated output
+}

docs/source/ephys_at_swc/community.md

@@ -0,0 +1,23 @@
+# Community
+
+This site is maintained by the
+[NeuroInformatics Unit](https://neuroinformatics.dev/).
+Please don't hesitate to contact us
+(in particular Joe Ziminski) on Slack or 
+[Zulip chat](https://neuroinformatics.zulipchat.com/)
+with any questions or feedback.
+
+For information and advice on electrophysiology from the SWC community, the
+best place to go is the `#forum-extracellular-ephys` channel on the SWC Slack.
+
+Outside the SWC, you can address any questions or issues about
+[SpikeInterface](https://github.com/SpikeInterface)
+by raising an issue on their
+[GitHub repository](https://github.com/SpikeInterface/spikeinterface/issues).
+They are very friendly and
+happy to answer any questions!
+
+In addition, the
+[Neuropixels](https://neuropixelsgroup.slack.com/)
+Slack channel is a great resource, with an active community discussing
+extracellular electrophysiology acquisition and analysis.

docs/source/ephys_at_swc/example_pipelines/examples/README.rst

@@ -0,0 +1,43 @@
+:orphan:
+
+Python Pipelines
+================
+
+Multiple probes (Cambridge Neurotech)
+-------------------------------------
+
+Dammy Onih
+(`Akrami lab <https://www.sainsburywellcome.org/web/groups/akrami-lab>`__)
+is running a multi-session paradigm with two
+`Cambridge Neurotech <https://www.cambridgeneurotech.com/neural-probes>`__
+probes. In this multi-session task, mice learn a statistical learning paradigm to a
+reward-associated auditory stimulus, recording from the hippocampus.
+The pipeline uses SpikeInterface for preprocessing,
+sorting, and analysis and can be found `here <https://github.com/AOONIH/ephys/tree/master>`__.
+
+The IBL analysis pipeline
+-------------------------
+
+Nate Miska (`Mrsic-Flogel lab <https://www.sainsburywellcome.org/web/groups/mrsic-flogel-lab>`__)
+is a member of the
+`International Brain Laboratory
+(IBL) <https://www.internationalbrainlab.com/>`_
+running the
+`IBL's standardised behavioural task <https://elifesciences.org/articles/63711>`_
+with acute Neuropixels 1.0 recordings. Details of the
+`analysis pipeline code <https://github.com/int-brain-lab/ibl-neuropixel>`__
+on the IBL data management system can be found
+`here <https://int-brain-lab.github.io/iblenv/index.html>`_.
+
+Integrated analysis with Neuropixels 2.0
+----------------------------------------
+
+Steve Lenzi (`Margie Lab <https://www.sainsburywellcome.org/web/groups/margrie-lab>`__)
+has an integrated pipeline for automated behavioural,
+electrophysiological, and anatomical analysis. Steve works with a
+multi-session escape paradigm, recording stimulus-responsive neurons in the posterior
+striatum. He is using NeuroPixels 2.0 and SpikeGLX with a SpikeInterface-based
+pipeline; the code can be found `here <https://github.com/stephenlenzi/npix_lse>`__.
+
+Python Scripts
+--------------

docs/source/ephys_at_swc/example_pipelines/examples/notes-on-docs-structure.txt

@@ -0,0 +1,5 @@
+Here, the 'Python Examples' page is rendered as a sphinx gallery. The
+first section is python examples that link to outside repos. The second
+section is the sphinx gallery entries. The 'README.rst' is the
+'Python Examples' page, it must be called 'README.rst' for sphinx-related
+reasons.

docs/source/ephys_at_swc/example_pipelines/examples/sara_mederos.py

@@ -0,0 +1,112 @@
+"""
+NP2.0 in SpikeInterface
+=======================
+
+Sara Mederos
+`(Hofer Lab) <https://www.sainsburywellcome.org/web/groups/hofer-lab>`__
+performs chronic electrophysiological
+recordings from subcortical and cortical areas using 4-shank
+Neuropixels 2.0 probes (acquired using
+`Open Ephys <https://open-ephys.org/>`__).
+Recordings are conducted in freely moving mice during behavioral
+paradigms that assess the cognitive control of innate behaviors.
+A pipeline used for pre-processing, sorting, and quality metrics
+can be found below.
+"""
+
+from spikeinterface import extract_waveforms
+from spikeinterface.extractors import read_openephys
+from spikeinterface.preprocessing import phase_shift, bandpass_filter, common_reference
+from spikeinterface.sorters import run_sorter
+from spikeinterface.qualitymetrics import compute_quality_metrics
+from pathlib import Path
+from probeinterface.plotting import plot_probe
+import matplotlib.pyplot as plt
+from spikeinterface import curation
+from spikeinterface.widgets import plot_timeseries
+import numpy as np
+
+
+data_path = Path(
+    r"/ceph/.../100323/2023-10-03_18-57-09/Record Node 101/experiment1"
+)
+output_path = Path(
+    r"/ceph/.../derivatives/100323/"
+)
+
+show_probe = False
+show_preprocessing = True
+
+# This reads OpenEphys 'Binary' format. It determines the
+# probe using probeinterface.read_openephys, which reads `settings.xml`
+# and requires the NP_PROBE field is filled.
+raw_recording = read_openephys(data_path)
+
+if show_probe:
+    probe = raw_recording.get_probe()
+    plot_probe(probe)
+    plt.show()
+
+# Run time shift (multiplex correction) and filter
+shifted_recording = phase_shift(raw_recording)
+filtered_recording = bandpass_filter(shifted_recording, freq_min=300, freq_max=6000)
+
+# Perform median average filter by shank
+channel_group = filtered_recording.get_property("group")
+split_channel_ids = [
+    filtered_recording.get_channel_ids()[channel_group == idx]
+    for idx in np.unique(channel_group)
+]
+preprocessed_recording = common_reference(
+    filtered_recording, reference="global", operator="median", groups=split_channel_ids
+)
+
+if show_preprocessing:
+    recs_grouped_by_shank = preprocessed_recording.split_by("group")
+    for rec in recs_grouped_by_shank:
+        plot_timeseries(
+            preprocessed_recording,
+            order_channel_by_depth=True,
+            time_range=(3499, 3500),
+            return_scaled=True,
+            show_channel_ids=True,
+            mode="map",
+        )
+        plt.show()
+
+# Run the sorting
+sorting = run_sorter(
+    "kilosort3",
+    preprocessed_recording,
+    singularity_image=True,
+    output_folder=(output_path / "sorting").as_posix(),
+    car=False,
+    freq_min=150,
+)
+
+# Curate the sorting output and extract waveforms. Calculate
+# quality metrics from the waveforms.
+sorting = sorting.remove_empty_units()
+
+sorting = curation.remove_excess_spikes(sorting, preprocessed_recording)
+
+# The way spikeinterface is set up means that quality metrics are
+# calculated on the spikeinterface-preprocessed, NOT the kilosort
+# preprocessed (i.e. drift-correct data).
+# see https://github.com/SpikeInterface/spikeinterface/pull/1954 for details.
+waveforms = extract_waveforms(
+    preprocessed_recording,
+    sorting,
+    folder=(output_path / "postprocessing").as_posix(),
+    ms_before=2,
+    ms_after=2,
+    max_spikes_per_unit=500,
+    return_scaled=True,
+    sparse=True,
+    peak_sign="neg",
+    method="radius",
+    radius_um=75,
+)
+
+quality_metrics = compute_quality_metrics(waveforms)
+quality_metrics.to_csv(output_path / "postprocessing")

docs/source/ephys_at_swc/example_pipelines/index.md

@@ -0,0 +1,35 @@
+# Examples
+
+This page shares extracellular electrophysiology pipelines used by researchers across the SWC.
+
+These examples aim to demonstrate the possibilities for ephys
+pipeline building rather than serve as drop-in tools for
+data processing. However, please feel free to
+contact any listed researchers to ask questions
+or seek advice!
+
+
+::::{grid} 1 2 2 2
+:gutter: 3
+
+:::{grid-item-card} {fas}`chart-simple;sd-text-primary` Python Examples
+:link: auto_examples_ephys/index
+:link-type: doc
+:::
+
+:::{grid-item-card} {fas}`chart-simple;sd-text-primary` MATLAB Examples
+:link: matlab_examples
+:link-type: doc
+:::
+
+::::
+
+
+```{toctree}
+:maxdepth: 2
+:hidden:
+
+auto_examples_ephys/index
+matlab_examples
+
+```

docs/source/ephys_at_swc/example_pipelines/matlab_examples.rst

@@ -0,0 +1,31 @@
+.. _matlab_examples:
+
+:orphan:
+
+Matlab Pipelines
+=================
+
+Multi-probe Neuropixels 1.0
+---------------------------
+
+Andrei Khilkevich
+(`Mrsic-Flogel lab <https://www.sainsburywellcome.org/web/groups/mrsic-flogel-lab>`__)
+performs acute recordings in a visual change-detection decision-making task.
+Two implanted probes (NP 1.0) in a head-fixed preparation are used to
+simultaneously record different regions across the whole brain.
+Their pipeline is available
+`here <https://github.com/BaselLaserMouse/Khilkevich_Lohse_2024/tree/main/NPX-postprocessing-pipeline>`__.
+
+Automated pipeline for multimodal integration
+---------------------------------------------
+
+Mateo Velez-Fort
+(`Margie Lab <https://www.sainsburywellcome.org/web/groups/margrie-lab>`__)
+investigates the integration of visual
+and vestibular information in the visual cortex with the
+Margrie lab's 'Translocator' setup. They use
+Neuropixels 2.0 (acquired using [SpikeGLX](https://github.com/billkarsh/SpikeGLX)) for acute recordings from the
+primary visual cortex and other cortical areas as the head-fixed
+mouse is physically displaced along a track.
+The pipeline is available
+`here <https://github.com/SainsburyWellcomeCentre/rc2_analysis>`__.

docs/source/ephys_at_swc/getting_started.md

@@ -0,0 +1,24 @@
+# Getting Started
+
+With numerous acquisition, preprocessing and analysis
+steps to consider, getting started with extracellular electrophysiology
+is not easy!
+
+To get started, the [Resources](resources) section has links to articles
+for a general introduction to extracellular electrophysiology (as well as
+more technical readings for a deeper background).
+
+It is useful to get advice on your particular
+experimental setup and research question early on. See the
+[Community](community) sections for details on where to get help.
+
+We recommend
+[SpikeInterface](https://github.com/SpikeInterface/spikeinterface),
+an open source community toolkit for extracellular electrophysiology,
+for preprocessing and spike sorting. To begin
+building your pipeline, the
+[SpikeInterface](https://spikeinterface.readthedocs.io/en/stable/)
+documentation is a good starting point. See the
+[Example Pipelines](example_pipelines/index)
+page for researchers at the SWC who would be happy
+to answer any questions you might have.

docs/source/ephys_at_swc/index.md

@@ -0,0 +1,52 @@
+# Electrophysiology
+
+This page serves as an information source for researchers using
+extracellular electrophysiology at the SWC. It contains
+useful [resources](resources.md) for learning, as well as
+[example pipelines](example_pipelines/index)
+used by researchers in the building.
+
+We encourage all types of contributions. If you'd like to contribute a
+guide or pipeline, please don't hesitate to get in touch! See the
+[community](community.md)
+page for more details.
+
+This page is maintained by the [Neuroinformatics Unit](https://neuroinformatics.dev).
+We are focused on building community tools to automate
+and standardise electrophysiology analysis.
+
+
+::::{grid} 1 2 2 2
+:gutter: 3
+
+:::{grid-item-card} {fas}`chart-simple;sd-text-primary` Getting Started
+:link: getting_started
+:link-type: doc
+:::
+
+:::{grid-item-card} {fas}`chart-simple;sd-text-primary` Resources
+:link: resources
+:link-type: doc
+:::
+
+:::{grid-item-card} {fas}`chart-simple;sd-text-primary` Example Pipelines
+:link: example_pipelines/index
+:link-type: doc
+:::
+
+:::{grid-item-card} {fas}`chart-simple;sd-text-primary` Community
+:link: community
+:link-type: doc
+:::
+
+::::
+
+```{toctree}
+:maxdepth: 2
+:hidden:
+
+getting_started
+resources
+example_pipelines/index
+community
+```