pspm_trim error: Matrix index is out of range for deletion.

[evl98]

Hello,

I am trying to import and trim my SCR data from a biosemi device (.bdf file). However, whenever I try to trim the data it gives the following error:

Matrix index is out of range for deletion.
In file "C:\Users\0072494\OneDrive - Universiteit Utrecht\Documents\MATLAB\PsPM\PsPM_course_ESCAN2024\PsPM_v6.1.2\pspm_trim.m" (???), function "pspm_trim" at line 328.
In file "C:\Users\0072494\OneDrive - Universiteit Utrecht\Documents\MATLAB\PsPM\PsPM_course_ESCAN2024\PsPM_v6.1.2\pspm_cfg\pspm_cfg_run_trim.m" (???), function "pspm_cfg_run_trim" at line 45.

This happens regardless of the references I put in. The only way in which the data does trim, is when I only import my SCR channel, without the markers, and then use the time period from the start of the file as a reference. For this reason, I am suspecting there to be an issue with the import of the marker channel from my .bdf file; as even just using the start of the file as a reference gives this error whenever the marker channel is included.

Does anybody have an idea what is going wrong and how I could solve this?
Thanks!

Emily

[dominikbach]

Hi Emily
thanks for reaching out. I agree this is probably a problem with the import. Would you be able to send the problematic raw data file and your import script, via email or direct download? Or any sample file.
Dominik

[evl98]

Dear Dominik, Thank you for your quick response! Attached you can find the script for the import job and the resulting matlab dataset. Below you can also find the link where you can download the .bdf file. I used your e-mail address from the university of Bonn to grant you access. Please let me know whether this works, and whether you need some more information. Kind regards, Emily [​bdf icon] part1.bdf<https://solisservices-my.sharepoint.com/:u:/g/personal/e_a_vanlooy_uu_nl/ETQyN9_YM1NIp8FsyCtwBscBZxBeuIxhq_SkAf7PpTAq0w?email=d.bach%40uni-bonn.de&e=0HjEOr> ­­Emily Vanlooy (she/her) | PhD Candidate | Utrecht University | Department of Psychology | Heidelberglaan 1, 3584 CS Utrecht | Office: Langeveld G1.15 From: Dominik Bach ***@***.***> Sent: maandag 10 juni 2024 17:49 To: bachlab/PsPM ***@***.***> Cc: Vanlooy, E.A. (Emily) ***@***.***>; Author ***@***.***> Subject: Re: [bachlab/PsPM] pspm_trim error: Matrix index is out of range for deletion. (Discussion #719) CAUTION: This email originated from outside of Utrecht University. Do not click links or open attachments unless you recognize the sender and know the content is safe. Hi Emily thanks for reaching out. I agree this is probably a problem with the import. Would you be able to send the problematic raw data file and your import script, via email or direct download? Or any sample file. Dominik — Reply to this email directly, view it on GitHub<#719 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A3RW6SXFAKQOLRM47VOB7XTZGXDF5AVCNFSM6AAAAABI77DPBCVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TOMRZGEYDE>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>

[dominikbach]

Thanks for sending the data; we will be looking into it.

[evl98]

Dear Dominik, Thank you! I noticed that the script attachments were deleted from my previous mail due to Outlook safety protocols. Hopefully the links below give you access to the intended files: C:\Users\0072494\OneDrive - Universiteit Utrecht\Documents\MATLAB\PsPM\import_bdf.m<file:///C:/Users/0072494/OneDrive%20-%20Universiteit%20Utrecht/Documents/MATLAB/PsPM/import_bdf.m> C:\Users\0072494\OneDrive - Universiteit Utrecht\Documents\MATLAB\PsPM\import_bdf_job.m<file:///C:/Users/0072494/OneDrive%20-%20Universiteit%20Utrecht/Documents/MATLAB/PsPM/import_bdf_job.m> C:\Users\0072494\OneDrive - Universiteit Utrecht\Documents\MATLAB\PsPM\pspm_part1.mat<file:///C:/Users/0072494/OneDrive%20-%20Universiteit%20Utrecht/Documents/MATLAB/PsPM/pspm_part1.mat> I also wanted to let you know that I was able to solve the 'Matrix index is out of range' error by transforming my .bdf file into an .edf file (using Brain Vision Analyzer). The issue seemed to be that the marker values had a different number of entries from the marker names. However, despite using the .edf format, the data still does not import in the right way for further processing. That is, I am still unable to trim the data using the marker values or marker names. Upon inspecting the code for the 'trim' function and comparing it to my own imported data, I noticed that the trim function refers to the marker values and names as follows: "data{k}.markerinfo.value", whereas my data can be called upon by using "data{2, 1}.markerinfo.value". My knowledge of Matlab is limited, but this seems to be suggesting that the structure of the dataset specified in the trim function does not match the way my imported data was structured. This holds for both the .bdf and the .edf data. Hopefully this information helps you locate the issue more easily. Below are the links to the .edf file and the import script: C:\Users\0072494\OneDrive - Universiteit Utrecht\Documents\MATLAB\PsPM\PsPM_course_ESCAN2024\import_edf_job.m<file:///C:/Users/0072494/OneDrive%20-%20Universiteit%20Utrecht/Documents/MATLAB/PsPM/PsPM_course_ESCAN2024/import_edf_job.m> C:\Users\0072494\OneDrive - Universiteit Utrecht\Documents\MATLAB\PsPM\PsPM_course_ESCAN2024\import_edf.m<file:///C:/Users/0072494/OneDrive%20-%20Universiteit%20Utrecht/Documents/MATLAB/PsPM/PsPM_course_ESCAN2024/import_edf.m> C:\Users\0072494\OneDrive - Universiteit Utrecht\Documents\MATLAB\PsPM\part13-edf.edf<file:///C:/Users/0072494/OneDrive%20-%20Universiteit%20Utrecht/Documents/MATLAB/PsPM/part13-edf.edf> Kind regards, Emily Emily Vanlooy (she/her) | PhD Candidate | Utrecht University | Department of Psychology | Heidelberglaan 1, 3584 CS Utrecht | Office: Langeveld G1.15 From: Dominik Bach ***@***.***> Sent: maandag 17 juni 2024 18:24 To: bachlab/PsPM ***@***.***> Cc: Vanlooy, E.A. (Emily) ***@***.***>; Author ***@***.***> Subject: Re: [bachlab/PsPM] pspm_trim error: Matrix index is out of range for deletion. (Discussion #719) You don't often get email from ***@***.******@***.***>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> CAUTION: This email originated from outside of Utrecht University. Do not click links or open attachments unless you recognize the sender and know the content is safe. Thanks for sending the data; we will be looking into it. Could you also send the import script - it was not attached? - Reply to this email directly, view it on GitHub<#719 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A3RW6SVAKH2MRH4J6WYSQ6TZH4ERDAVCNFSM6AAAAABI77DPBCVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TOOJWG4ZTG>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>

[dominikbach]

Hi Emily
there was a bug in using markervalues for some data types. This should be fixed in the develop branch - at the very least you should be getting a more expressive error already during import if something goes wrong. Could you perhaps try this?
Fixing the bdf import may take some more time, unfortunately.
Dominik

[dominikbach]

Hi Emily
the root cause of your problem is that for some markers, the recording software records a type but no value (this is the case for type Epoch and CM_in_range). We're working on a solution that converts the non-existing values to NaN.
Dominik

[dominikbach]

Biosemi Import for these edge cases is fixed now in the develop branch and will be included in the next release.

[evl98]

Thank you for fixing the bdf import! I will try it on my data as soon as possible

[evl98]

Dear Dominik, Thank you again for the adjustments to the software. Everything works now up until the estimation of the non-linear SCR model. When running the model estimation, I receive the following error: Error using downsample Expected N to be integer-valued. In file "C:\Program Files\MATLAB\R2023a\toolbox\signal\signal\downsample.m" (???), function "downsample" at line 50. In file "C:\Users\0072494\OneDrive - Universiteit Utrecht\Documents\MATLAB\PsPM_v6.1.2\pspm_dcm.m" (???), function "pspm_dcm" at line 376. In file "C:\Users\0072494\OneDrive - Universiteit Utrecht\Documents\MATLAB\PsPM_v6.1.2\pspm_cfg\pspm_cfg_run_dcm.m" (???), function "pspm_cfg_run_dcm" at line 131. This error is solved when I add a round() function to the second argument of the downsample function in line 376 of the pspm_dcm.m file (see printscreen below). ***@***.*** However, given that I am not familiar with the software, I wanted to make sure whether this is a valid solution; or whether this indicates that there is another error somewhere or compromises the reliability of the model estimation in some way. Thanks in advance! Kind regards, Emily From: Dominik Bach ***@***.***> Sent: woensdag 4 september 2024 7:07 To: bachlab/PsPM ***@***.***> Cc: Vanlooy, E.A. (Emily) ***@***.***>; Author ***@***.***> Subject: Re: [bachlab/PsPM] pspm_trim error: Matrix index is out of range for deletion. (Discussion #719) You don't often get email from ***@***.******@***.***>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> CAUTION: This email originated from outside of Utrecht University. Do not click links or open attachments unless you recognize the sender and know the content is safe. Biosemi Import for these edge cases is fixed now in the develop branch and will be included in the next release. - Reply to this email directly, view it on GitHub<#719 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A3RW6SWNTRNHUQ3574CQNR3ZU2IQXAVCNFSM6AAAAABI77DPBCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANJUGA2DSMA>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>

[dominikbach]

Thanks for flagging up this edge case, in which a non-integer ratio of initial sample rate, and model sample rate (default 10 Hz) lead to an error. Unfortunately, just rounding the ratio of sample rates will produce wrong missing value indices, because the data are downsampled with the correct non-integer ratio using a different procedure.

While we fix this in the code, there are two workarounds:

1. Make sure the model sample rate is a divisor of the initial sample rate. You can change the model sample rate in the filter settings of the model.
2. In the offending line 376 in pspm_dcm, replace downsample(sbs_miss, model.filter.sr/model.sr) with: resample(sbs_miss, model.filter.sr, model.sr)

[evl98]

Thanks! I checked the initial rate which was 2048Hz, so it worked after changing the model sample rate to 16Hz From: Dominik Bach ***@***.***> Sent: maandag 7 oktober 2024 18:12 To: bachlab/PsPM ***@***.***> Cc: Vanlooy, E.A. (Emily) ***@***.***>; Author ***@***.***> Subject: Re: [bachlab/PsPM] pspm_trim error: Matrix index is out of range for deletion. (Discussion #719) CAUTION: This email originated from outside of Utrecht University. Do not click links or open attachments unless you recognize the sender and know the content is safe. Thanks for flagging up this edge case, in which a non-integer ratio of initial sample rate, and model sample rate (default 10 Hz) lead to an error. Unfortunately, just rounding the ratio of sample rates will produce wrong missing value indices, because the data are downsampled with the correct non-integer ratio using a different procedure. While we fix this in the code, there are two workarounds: 1. Make sure the model sample rate is a divisor of the initial sample rate. You can change the model sample rate in the filter settings of the model. 2. In the offending line 376 in pspm_dcm, replace downsample(sbs_miss, model.filter.sr/model.sr) with: resample(sbs_miss, model.filter.sr, model.sr) - Reply to this email directly, view it on GitHub<#719 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A3RW6SST6VMKQIAXB5QR3F3Z2KXEPAVCNFSM6AAAAABI77DPBCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAOBXGAZDAMY>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>