Error occurred when getting spikes

[zjm199502]

Hi petersen,

I got a problem when i tried to follow the tutorial. After run the command cell_metrics = ProcessCellMetrics('session', session); error occurred as the following picture shows.

Is there anything wrong?

My raw data are continuous binary data (.dat). And the spike sorter is Spyking-circus.

Thank you very much!

Jinming Zhang

[zjm199502]

@petersenpeter

[petersenpeter]

Hi

This typically happens if the spike data is not imported correctly (e.g. the spikes struct is empty). Could you check that the spikes struct is generated correctly? I can see that the file continuous.spikes.cellinfo.mat was generated but check that the content looks OK?

If the spikes struct is empty, try generating it directly with loadSpikes.

Feel free to share the session with me.

[zjm199502]

Hi,

Does this (Blue Box) looks OK? (I didn't find a right template of continuous.spikes.cellinfo.mat)

I tried use loadSpikes, and it firstly showed Red Box 1. Then I tried to convert the Spyking-circus sorting result into a Phy format and then loadSpikes again, it showed Red Box 2 at this time. Do I need editor a Spyking-circus result format (.hdf5) by myself ?

Thank you very much!

[zjm199502]

And in the fields of continuous.spikes.cellinfo.mat, spikes.times is empty. While spikes.processinginfo and spikes.processinginfo.params are as following pictures show.

[petersenpeter]

Hi. Could you share your dataset with me? I must say I forgot, but Spyking circus has not been implemented yet, but I can fix this today if I have your data.

If you want to import the phy format you must provide these parameters to loadSpikes:
From the basepath of the session try:
spikes = loadSpikes('format','spyking circus');

Or for the phy data format:
spikes = loadSpikes('format','phy');

[zjm199502]

Hi,
The raw data is too big to upload here, so I just send it through email to your address ([email protected]). Please check the email.

Thank you again for your help.

[petersenpeter]

Hi
Sorry, it took me some time to look into this. But I found the issues.

• Your recording had 32 channels, you had defined 16 in the session info:
  session.extracellular.nChannels = 32;

• Same issue with your electrode groups. Create two groups, one for the first bad 16 channels, and a second for the rest:
  session.extracellular.electrodeGroups.channels = {[1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16] [17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32]};
  This is not strictly necessary but a good practice.

• Next, you did not select the correct spike data format:
  session.spikeSorting{1}.format = 'Phy'

• Finally, your phy data was in a sub-subfolder, continuous/continuous-merged.GUI. Either define the full relative path or move it up a level:
  session.spikeSorting{1}.relativePath = 'continuous/continuous-merged.GUI';

After fixing these issues it worked.

[zjm199502]

Hi,
Sorry for the late feedback.
Thank you so much for your help. And I followed your construction, but unfortunately the error still occurred in my computer when i try to import the phy format result(as mentioned in the first).
I will try the other spike sorting method that the CellExplorer already supported directly (such as klusta,UltraMegaSort2000). And may I ask you that whether you still have the plan to make Spyking-circus format being supported?

[petersenpeter]

Yes, I implement Spyking Circus soon, but as you are doing manual curation in Phy, its actually Phy you want to import, as this contains your manual curation.

Odd that you still receive an error. What is the error you get?

Here are the spikes, cell metrics, and session struct for your session that I processed:
dataset files.zip

[petersenpeter]

Hi
I added SpyKing Circus as a supported format. Please try it for me and let me know if you experience any issues.

To test it, go to the basepath of your session, and run:

spikes = loadSpikes('format','spyking circus','clusteringpath','continuous','forceReload',true);

You can also import your Phy-data from the same dataset:

spikes = loadSpikes('format','phy','clusteringpath','continuous/continuous-merged.GUI','forceReload',true);

[zjm199502]

Hi,

Thank you so much! First, I can manually curate the sorting results using a matlab-GUI on *.hdf5 files, so it do helps if I can import those directly into CellExplorer. Second, I try the code spikes = loadSpikes('format','spyking circus','clusteringpath','continuous','forceReload',true); and it works, spikes were successfully imported.

But there still some issues that I should remind you:

1. In loadSpikes.m, line870, file name 'continuous.result.hdf5' actually will change if users modified the raw data file name(*.result.hdf5).

2. When I try to follow 'CellExplorer_Tutorial.m', part3.1.1, code cell_metrics = ProcessCellMetrics('session', session);, error occurred as following picture shows.
   

3. After the spike data was imported using the newest code you provide, I try to lanuch the main GUI of CellExplorer using cell_metrics = CellExplorer('metrics',cell_metrics); . Error occurred as fllowing picture shows.
   

[zjm199502]

After spikes = loadSpikes('format','spyking circus','clusteringpath','continuous','forceReload',true);, I got three new *.mat files as following shows, without 'continuous.cell_metrics.cellinfo.mat'. Does it need an extra code to generate?

[petersenpeter]

1: Thanks for pointing this out: I pushed a new version corrected for this.
2: I believe I imported the spikes with incorrect dimensions. Try updating your CellExplorer repo and run the script again.
3: Your third error happens because the cell metrics pipeline resulted in the error in step 2, so you did not create the cell metrics.

[petersenpeter]

Hi, I fixed some bugs and updated the data source for SpyKing Circus. Update your repo and try to run the pipeline again.

Make sure to delete spikes and cell metrics files before rerunning the pipeline.

[zjm199502]

Thank you for that , but the error still occurred. May there be anything still need be modified?

[zjm199502]

I think I have located the problem: mannually curated *.result-merged.hdf5 file becomes an 'array' type which has 2 dimensions.

While the uncurated *.result-merged.hdf5 is 'carray' type which only has 1 dimension.

This could be the reason of the cat error. I try to process the uncurated *.result-merged.hdf5 file with the latest version of CellExplorer and it does works.

Could this problem be solved by modifying current script? Or I need turn to the developer of Spyking-circus for further helping?

Sorry for so many questions.

[zjm199502]

I made some modifications on loadSpikes.m so that it can correctly handle the mannually curated *.result-merged.hdf5 with 'array' data type. Hope this will help the others in using a Spyking-circus format.
Replace code line882 spikes.times{i} = double(h5read(nwb_file,['/spiketimes/',template_names{i}]))/session.extracellular.sr;
with:
times = h5read(nwb_file,['/spiketimes/',template_names{i}]);
spikes.times{i} = double(times(1,:)')/session.extracellular.sr;

Thank you so much!

[petersenpeter]

Great. I will incorporate your suggestion and commit a new version of loadSpikes.