-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add miniscope interface #8
Conversation
This is ready for review now @alessandratrapani |
def __init__(self, folder_path: DirectoryPath): | ||
self.folder_path = Path(folder_path) | ||
|
||
miniscope_videos_folder_path = self.folder_path / "Miniscope" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This may not work with Cai lab data. For FC, Neutral Exposure and Recall session the folder containing miniscope data is "miniscope" (all lower case). For Offline sessions the folder is called "My_V4_Miniscope". I think we should pass directly what you called miniscope_videos_folder_path
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This may not work with Cai lab data. For FC, Neutral Exposure and Recall session the folder containing miniscope data is "miniscope" (all lower case).
So I developed this against the data from this interface but it is good to be aware. This depends on the subject:
Ca_EEG2-1: folder name: Minscope
Ca_EEG3-4: folder name: miniscope
This makes you point stronger though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed.
super().__init__(folder_path=folder_path) | ||
|
||
self.folder_path = Path(folder_path) | ||
miniscope_folder = Path(folder_path) / "Miniscope" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed.
@@ -776,6 +776,8 @@ This is the metadata file in the same directory that the minian videos: | |||
} | |||
``` | |||
|
|||
Something concerning is that the 30FPS does not correspond to the videos. The metadata says 30 FPS but the videos are 60 FPS. | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure that the 60FPS reported in the file property is actually correct.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I checked against OpenCV, why are you suspicious?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(looking at subject EEG3-4) Because 60fps is a bit unusual for onephoton imaging (higher than the usual 20-30 Hz) + the timestamps are 33 ms apart (even if they are not always regular), that is 30Hz + if you account for 30fps, the time span of the calcium imaging recording is the same of the behavioral recording which would make more sense instead of recording with miniscope just half of the time (as it would be accounting for 60fps)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for writing this. Let me take a close look.
timestamps_seconds = timestamps_milliseconds / 1000.0 | ||
|
||
# Shift when the first timestamp is negative | ||
# TODO: Figure why, I copied from miniscope |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess the timestamps refer to the session start time. If this is correct, when shifting all timestamps, we should also shift back the session start time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's discuss this briefly in the meeting. I copied this from the current miniscope code on the corresponding extension. I would rather fix the timing issues in one go when we are at the synch stage.
I checked how the frame rate is in the jsons files that were shared with and that is consistent: My grep search on the data that I have (click to expand)@h-laptop$ pwd
/media/heberto/One Touch/Cai-CN-data-share/Ca_EEG_Experiment
@h-laptop$ grep -r "frameRate" --include="*.json" .
./Ca_EEG3-4/Ca_EEG3-4_Sessions/Ca_EEG3-4_NeutralExposure/2022_09_17/09_23_03/miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Sessions/Ca_EEG3-4_FC/2022_09_19/09_18_41/miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Sessions/Ca_EEG3-4_Recall1/2022_09_20/09_19_43/miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Sessions/Ca_EEG3-4_Recall2/2022_09_21/08_41_45/miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Sessions/Ca_EEG3-4_Recall3/2022_09_22/08_40_12/miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/09_40_40/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/10_10_41/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/10_40_41/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/11_10_42/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/11_40_42/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/12_10_43/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/12_40_43/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/13_10_44/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/13_40_44/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/14_10_45/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/14_40_45/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/15_10_46/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/15_40_46/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/16_10_47/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/16_40_47/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/17_10_48/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/17_40_48/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/18_10_49/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/18_40_49/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/19_10_50/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/19_40_50/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/20_10_51/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/20_40_51/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay1/2022_09_17/21_10_52/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/09_30_07/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/10_00_07/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/10_30_08/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/11_00_08/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/11_30_09/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/12_00_09/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/12_30_10/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/13_00_10/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/13_30_11/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/14_00_11/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/14_30_12/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/15_00_12/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/15_30_13/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/16_00_13/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/16_30_14/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/17_00_14/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/17_30_15/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/18_00_15/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/18_30_16/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/19_00_16/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/19_30_17/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/20_30_18/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/21_00_18/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG3-4/Ca_EEG3-4_Offline/Ca_EEG3-4_OfflineDay2/2022_09_19/20_00_17/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Sessions/Ca_EEG2-1_FC/10_11_24/Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Sessions/Ca_EEG2-1_NeutralExposure/10_43_54_first_attempt/Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Sessions/Ca_EEG2-1_NeutralExposure/10_44_18/Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Sessions/Ca_EEG2-1_Recall1/10_33_20/Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay1/2021_10_12/10_05_57/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay1/2021_10_12/10_08_17/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay1/2021_10_12/10_10_44/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay1/2021_10_12/10_15_50/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay1/2021_10_12/12_07_50/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay2/2021_10_14/09_24_27/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay2/2021_10_14/09_54_27/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay2/2021_10_14/10_24_27/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay2/2021_10_14/10_54_27/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay2/2021_10_14/11_24_26/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay2/2021_10_14/11_54_26/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay2/2021_10_14/12_24_26/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay2/2021_10_14/12_54_26/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay2/2021_10_14/13_24_26/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay2/2021_10_14/13_54_26/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay2/2021_10_14/14_24_25/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay2/2021_10_14/14_54_25/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay2/2021_10_14/15_24_25/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay2/2021_10_14/15_54_25/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay2/2021_10_14/16_24_25/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay2/2021_10_14/16_54_24/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay2/2021_10_14/17_24_24/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay2/2021_10_14/17_54_24/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflineDay2/2021_10_14/18_24_24/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflinePre/2021_10_10/11_05_22/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflinePre/2021_10_10/11_35_22/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflinePre/2021_10_10/12_05_22/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflinePre/2021_10_10/12_35_22/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflinePre/2021_10_10/13_05_21/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflinePre/2021_10_10/13_35_21/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflinePre/2021_10_10/14_05_21/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflinePre/2021_10_10/14_35_21/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflinePre/2021_10_10/15_05_21/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflinePre/2021_10_10/15_35_21/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflinePre/2021_10_10/16_05_20/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
./Ca_EEG2-1/Ca_EEG2-1_Offline/Ca_EEG2-1_OfflinePre/2021_10_10/16_35_20/My_V4_Miniscope/metaData.json: "frameRate": "30FPS",
|
@alessandratrapani I changed the frame rate on the metadata (see previous comment). I can't force timestamps because I only have done those changes for recording interfaces here: We should add that feature for neuroconv in imaging interfaces. |
Should come after #6
As I have been exploring this topic here:
catalystneuro/roiextractors#356
I think that the format at the moment is too heterogeneous. Let's make an interface that works for this conversion so we can move forward with this project. Once we accumulate more examples (working on that) we can propagate this to roiextractors and neuroconv.