fieldmap-less distortion correction not working

[DVSneuro]

What happened?

We have a few subjects where the fieldmaps weren't collected correctly for all of the runs because the subject got out of the scanner and we didn't collect a second fieldmap when they got back in the scanner. So, we're trying to use --use-syn-sdc and --ignore fieldmaps on these subjects, but FMRIprep is crashing (see error/log information below).

I'm only using the --ignore fieldmaps flag because we haven't yet updated our IntendedFor fields in the json files for these subjects. Ultimately, I'd like to use the fieldmap we collected for the runs that we collected before the subject got out of the scanner, and then use fieldmap-less distortion correction for the runs that were collected after the subject got back into the scanner.

Thanks!
David

What command did you use?

singularity run --cleanenv \
		-B ${TEMPLATEFLOW_DIR}:/opt/templateflow \
		-B ${MPLCONFIGDIR_DIR}:/opt/mplconfigdir \
		-B $maindir:/base \
		-B ~/work/tools/licenses:/opts \
		-B $scratchdir:/scratch \
		~/work/tools/fmriprep-23.1.3.simg \
		/base/bids /base/derivatives/fmriprep \
		participant --participant_label $sub \
		--stop-on-first-crash \
		--nthreads 12 \
		--me-output-echos \
		--ignore fieldmaps \
		--use-syn-sdc \
		--cifti-output 91k \
		--output-spaces fsLR fsaverage MNI152NLin6Asym \
		--bids-filter-file /base/code/fmriprep_config.json \
		--fs-license-file /opts/fs_license.txt -w /scratch

What version of fMRIPrep are you running?

23.1.3

How are you running fMRIPrep?

Singularity

Is your data BIDS valid?

Yes

Are you reusing any previously computed results?

No

Please copy and paste any relevant log output.

Node: fmriprep_23_1_wf.single_subject_10317_wf.func_preproc_task_doors_run_1_echo_1_wf.output_select
Working directory: /scratch/fmriprep_23_1_wf/single_subject_10317_wf/func_preproc_task_doors_run_1_echo_1_wf/output_select

Node inputs:

fmap = ['/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00016/unwarp/clipped_field.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00017/unwarp/clipped_fi
eld.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00018/unwarp/clipped_field.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00019/unwarp/clipp
ed_field.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00020/unwarp/clipped_field.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00021/unwarp/
clipped_field.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00022/unwarp/clipped_field.nii.gz']
fmap_coeff = [['/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00016/bs_filter/fmap_syn0Warp_Hz_trans_coeff000.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_
auto_00016/bs_filter/fmap_syn0Warp_Hz_trans_coeff001.nii.gz'], ['/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00017/bs_filter/fmap_syn0Warp_Hz_trans_coeff000.nii.gz', '/scratch/fmriprep_23
_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00017/bs_filter/fmap_syn0Warp_Hz_trans_coeff001.nii.gz'], ['/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00018/bs_filter/fmap_syn0Warp
_Hz_trans_coeff000.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00018/bs_filter/fmap_syn0Warp_Hz_trans_coeff001.nii.gz'], ['/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap
_preproc_wf/wf_auto_00019/bs_filter/fmap_syn0Warp_Hz_trans_coeff000.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00019/bs_filter/fmap_syn0Warp_Hz_trans_coeff001.nii.gz'], ['/scra
tch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00020/bs_filter/fmap_syn0Warp_Hz_trans_coeff000.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00020/bs_filter/
fmap_syn0Warp_Hz_trans_coeff001.nii.gz'], ['/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00021/bs_filter/fmap_syn0Warp_Hz_trans_coeff000.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_
10317_wf/fmap_preproc_wf/wf_auto_00021/bs_filter/fmap_syn0Warp_Hz_trans_coeff001.nii.gz'], ['/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00022/bs_filter/fmap_syn0Warp_Hz_trans_coeff000.ni
i.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00022/bs_filter/fmap_syn0Warp_Hz_trans_coeff001.nii.gz']]
fmap_mask = ['/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00016/zooms_bmask/sub-10317_T1w_noise_corrected_corrected_xform_rbrainmask_trans_uint8_trans.nii.gz', '/scratch/fmriprep_23_1_wf/
single_subject_10317_wf/fmap_preproc_wf/wf_auto_00017/zooms_bmask/sub-10317_T1w_noise_corrected_corrected_xform_rbrainmask_trans_uint8_trans.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/
wf_auto_00018/zooms_bmask/sub-10317_T1w_noise_corrected_corrected_xform_rbrainmask_trans_uint8_trans.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00019/zooms_bmask/sub-10317_T1w_
noise_corrected_corrected_xform_rbrainmask_trans_uint8_trans.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00020/zooms_bmask/sub-10317_T1w_noise_corrected_corrected_xform_rbrainma
sk_trans_uint8_trans.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00021/zooms_bmask/sub-10317_T1w_noise_corrected_corrected_xform_rbrainmask_trans_uint8_trans.nii.gz', '/scratch/
fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00022/zooms_bmask/sub-10317_T1w_noise_corrected_corrected_xform_rbrainmask_trans_uint8_trans.nii.gz']
fmap_ref = ['/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00016/unwarp/clipped_unwarped.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00017/unwarp/cli
pped_unwarped.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00018/unwarp/clipped_unwarped.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00019
/unwarp/clipped_unwarped.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00020/unwarp/clipped_unwarped.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf
_auto_00021/unwarp/clipped_unwarped.nii.gz', '/scratch/fmriprep_23_1_wf/single_subject_10317_wf/fmap_preproc_wf/wf_auto_00022/unwarp/clipped_unwarped.nii.gz']
key = auto_00000
keys = ['auto_00016', 'auto_00017', 'auto_00018', 'auto_00019', 'auto_00020', 'auto_00021', 'auto_00022']
sdc_method = ['FLB ("fieldmap-less", SyN-based)', 'FLB ("fieldmap-less", SyN-based)', 'FLB ("fieldmap-less", SyN-based)', 'FLB ("fieldmap-less", SyN-based)', 'FLB ("fieldmap-less", SyN-based)', 'FLB ("fieldmap-less", 
SyN-based)', 'FLB ("fieldmap-less", SyN-based)']

Traceback (most recent call last):
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 344, in _send_procs_to_workers
    self.procs[jobid].run(updatehash=updatehash)
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node output_select.

Traceback:
	Traceback (most recent call last):
	  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 400, in run
	    outputs = self.aggregate_outputs(runtime)
	  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 429, in aggregate_outputs
	    predicted_outputs = self._list_outputs()  # Predictions from _list_outputs
	  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/niworkflows/interfaces/utility.py", line 217, in _list_outputs
	    index = self.inputs.keys.index(self.inputs.key)
	ValueError: 'auto_00000' is not in list

Additional information / screenshots

No response

[effigies]

I believe this should have been fixed by #3060. Can you try 23.1.4?

[DVSneuro]

Thanks! That works! And wow, it seems to work better than using the actual fieldmaps...

[effigies]

That's good to hear... I'm finding some cases where SyN is failing pretty badly, and I hope it's just that I'm finding new cases and we haven't somehow broken it for everybody.

[DVSneuro]

I don't have a ton of data points on SyN yet, but maybe something is off with our fieldmaps in some cases. We'll investigate further and post to NeuroStars or file a separate issue if it looks like a potential bug with how fieldmaps are being handled with our data.

[effigies]

I'm also looking into fieldmaps. There's an issue with resampling the fieldmap onto the target space that seems to be affecting phasediff fieldmaps in particular, and I don't yet understand why.