write_raw_bids() seems to add a stray coordinate transform in FIFF

[vmantyne]

Description of the problem

There seems to be a stray coordinate transformation in the recorded file, after acquisition parameters block (117), before projection block (313). May cause problems with MaxFilter and XFit.

show_fiff -v sub-16_ses-01_task-rest_proc-raw_meg.fif
100 = file ID        1.4 0x3033303330333033 Thu 01 Jan 1970 02:00:00 AM EET +2147483647 us
101 = dir pointer    -1
106 = free list      -1
104 = {              100 = measurement   
...
104 = {              109 = HPI result    
213 = dig. point     hpi        1 (  60.7,  79.8,  27.3)
213 = dig. point     hpi        2 (  67.0, -19.0, -54.1)
213 = dig. point     hpi        3 ( -79.8,   0.1, -49.0)
213 = dig. point     hpi        4 ( -50.8,  84.1,  29.5)
213 = dig. point     hpi        5 ( -17.0,  92.2,  55.8)
247 = HPI coil order 5 ints (4 3 2 1 5 )
246 = HPI coils used 4 ints (1 2 3 4 )
240 = HPI moments    2: 3 x 5 (15 floats)
241 = HPI g-values   5 floats (0.995893 0.993551 0.998066 0.999219 0.996491 )
243 = HPI g limit    0.98
244 = HPI dist limit 0.005
242 = HPI fit accept 0
222 = transform      device -> head
       0.994742 -0.100193 -0.021228
       0.048210 0.275214 0.960174
       -0.090361 -0.956148 0.278597
       7.379529 28.639034 -3.697928 (inv. -9.055565 -10.678233 -26.311550)
105 = }              109 = HPI result    
...
 
ERFtriggerMap 1:2:4:8:16:32:3:5:6:7:9:10:11:12:13:14:15:1:1:1:1:1:1:1:1:1:1:1:1:1:1:1
ERFupdate 15
ERFversion 1
TCPcollectorHost alpha
TCPisotrakHost alpha
 
151 = acq. stim seq. 1 50.000000 5000.000000
105 = }              117 = acq. pars.    
222 = transform      device -> head
       0.994742 -0.100193 -0.021228
       0.048210 0.275214 0.960174
       -0.090361 -0.956148 0.278597
       7.379529 28.639034 -3.697928 (inv. -9.055565 -10.678233 -26.311550)
104 = {              313 = projection    
104 = {              314 = proj. item    
200 = nchan          306
...

Steps to reproduce

write_raw_bids(raw, 
                   bids_path, 
                   events_data=events_data,
                   event_id=event_id, 
                   overwrite=True)

Expected results

Copy of the raw data file

Actual results

Additional information

Platform: Linux-5.15.0-86-generic-x86_64-with-glibc2.31
Python: 3.10.9 | packaged by conda-forge | (main, Feb 2 2023, 20:20:04) [GCC 11.3.0]
Executable: /shared/anaconda3/envs/mne1.3/bin/python
CPU: x86_64: 16 cores
Memory: 31.3 GB

mne: 1.3.0
numpy: 1.23.5 {OpenBLAS 0.3.21 with 16 threads}
scipy: 1.10.0
matplotlib: 3.6.2 {backend=QtAgg}

sklearn: 1.2.0
numba: 0.56.4
nibabel: 5.0.0
nilearn: 0.10.0
dipy: 1.6.0
openmeeg: 2.5.5
cupy: 11.5.0
pandas: 1.5.2
pyvista: 0.38.1 {OpenGL 4.5.0 NVIDIA 525.125.06 via Quadro M2000/PCIe/SSE2}
pyvistaqt: 0.9.0
ipyvtklink: 0.2.2
vtk: 9.2.5
qtpy: 2.2.0 {PyQt5=5.15.6}
ipympl: Not found
pyqtgraph: 0.13.1
pooch: v1.6.0

mne_bids: 0.10
mne_nirs: 0.5.0
mne_features: 0.2.1
mne_qt_browser: 0.4.0
mne_connectivity: 0.5.0
mne_icalabel: 0.4

[welcome]

Hello! 👋 Thanks for opening your first issue here! ❤️ We will try to get back to you soon. 🚴🏽‍♂️

[larsoner]

There seems to be a stray coordinate transformation in the recorded file, after acquisition parameters block (117), before projection block (313). May cause problems with MaxFilter and XFit.

One of these is in the HPI_RESULT block and the other is in the INFO block (one level up) and they should be identical so I think in principle it should be okay. We have written it this way in MNE-Python for years and haven't gotten bug reports about it. (I'm 99% sure collaborators have used maxfilter and/or Xfit with these files in the meantime.) Have you experienced problems or is this hypothetical? If just hypothetical I think we can probably close...

[vmantyne]

This is a real problem but most likely very rare. We have a recording with the cardinals digitized in wrong order and the head frame is inverted. Actually this was caused by the misplacement of the Polhemus receiver with the cord NOT POINTING TOWARD FEET but toward head with the subject in supine position on the bed.

On XFit

The initial device -> head transform is also obviously wrong:

MEGIN helped and provided us with a utility program to swap the cardinals. In short, the instructions to correct the digitization and the transform read as follows:

1. swap the cardinals to correct the head coordinate frame and overwrite digitized points because of 'upside down' digitization
2. Redo the hpifit
3. Overwrite the old hpifit in the raw file with the new, right-left swapped hpifit produced in the previous step

I realize we should correct the original raw data files and redo "BIDSifying" because the “BIDSifying” of the erroneous files appears as a possible source of problems. Let’s see how!

The BIDS step of copying the raw data adds a stray coordinate transform, the second one at meas. info block root level. This seems to cause problems with Xfit, at least.

The original transform, and the BIDS copy of it do look weird:

222 = transform device -> head
0.994893 -0.082534 -0.058113
0.077752 0.259449 0.962622
-0.064372 -0.962224 0.264541
5.855902 32.088913 -3.036933 (inv. -8.516451 -10.764319 -29.745789)
222 = transform device -> head
0.994893 -0.082534 -0.058113
0.077752 0.259449 0.962622
-0.064372 -0.962224 0.264541
5.855902 32.088913 -3.036933 (inv. -8.516451 -10.764319 -29.745789)

After swapping the cardinals, hpifit redo, and transform replacement:

222 = transform device -> head
0.994219 -0.104434 -0.024944
0.106942 0.983903 0.143171
0.009591 -0.145011 0.989384
0.926245 -6.471648 58.243248 (inv. -0.787385 14.910131 -56.675255)
222 = transform device -> head
0.994893 -0.082534 -0.058113
0.077752 0.259449 0.962622
-0.064372 -0.962224 0.264541
5.855902 32.088913 -3.036933 (inv. -8.516451 -10.764319 -29.745789)

The first transform, the correct one inside HPI block, was changed into the correct one. Let’s the points, coils and the head model on Xfit!

MNE BIDS copy of the raw file with corrected digitization and HPI fit opened in XFit

In XFIT, coils/helmet still looks rotated with the BIDS copied raw data although the digitization points and the head frame appear properly aligned. Let's see how the original raw behaves. That only contains one transform device -> head.

Original raw data file with corrected digitization and HPI fit opened in XFit

Everything looks properly aligned.

Conclusion is that the Xfit (our version) appears to read the wrong transform, the stray one added in BIDS stage by MNE Python. Of course it would be trivial to correct both transforms at the same time or force the XFit (and other programs) use the HPI block transform as it should but that would be fixing the wrong place wouldn't it? I admit I do not understand why and where this other transform is needed though. I sure hope reading it is not too hard-coded everywhere if it turns out to be worth removing also in your opinion.

The duplicate transform saved by MNE BIDS can apparently cause problems although it’s rare that, e.g., the described digitization errors happen. Apparently there are not too many (MEGIN?) support requests on these two transforms that could be avoided altogether. The device-to-head transform is unique (one head under one helmet at one initial position!) and therefore only one transform should be needed. It also should probably be made clear that the MNE BIDS does not copy the raw data files as it implies but reassembles them different from the original recordings. The problem thus is real, rare so far, and in part philosophical. :)

[sappelhoff]

it also should probably be made clear that the MNE BIDS does not copy the raw data files as it implies but reassembles them different from the original recordings.

a PR that clarifies this behavior in our documentation would be most welcome!