unknown new formatting for imaging ROIs in v1.4.0

[carsen-stringer]

Description

Before pynwb==1.4.0, rois could be saved as 2D, now I write them as 2D but when I read them they are no longer 2D:

IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed

There is no mention of format change in the v1.4.0 release, this works fine in v1.3.3.

Steps to Reproduce

with NWBHDF5IO(fpath, 'r') as fio:
    nwbfile = fio.read()
    rois = nwbfile.processing['ophys']['ImageSegmentation']['PlaneSegmentation']['pixel_mask']
    rois[0][:,0]

Environment

Python Executable: Conda
Python Version: Python 3.7
Operating System: Linux
HDMF Version: pynwb==1.4.0

Checklist

• [X ] Have you ensured the feature or change was not already reported?
• [X ] Have you included a brief and descriptive title?
• [X ] Have you included a clear description of the problem you are trying to solve?
• [X ] Have you included a minimal code snippet that reproduces the issue you are encountering?
• [X ] Have you checked our Contributing document?

[rly]

Hi @carsen-stringer . Thanks for the bug report. What version of h5py do you have installed?

[carsen-stringer]

h5py 2.10.0 pypi_0 pypi
hdf5 1.10.6 nompi_h6a2412b_1114 conda-forge
hdmf 2.3.0 pypi_0 pypi

[carsen-stringer]

okay so I dug into this a little more and now the rois are tuples? is there any reason why they aren't np.ndarrays anymore?

[carsen-stringer]

I've fixed the accessing in suite2p and it should be backward compatible, but other imaging softwares may have similar issues

[bendichter]

Thanks for tracking this down, @carsen-stringer!

[rly]

Hi @carsen-stringer . Sorry for not following up on this issue. The core machinery of PyNWB lives in HDMF, which we also maintain. We found that PyNWB 1.4.0 with HDMF 2.2.0 wrote pixel masks as a dataset of dtype int and shape (N, 3), when instead, based on the NWB spec, it should be written as a dataset of compound type (uint, uint, float) and shape (N, ) (see #1313). This was fixed in HDMF 2.3.0, which changes how pixel mask data are written, which in turn changes how the data can be read. Since PyNWB has not yet released a new version since the update to HDMF 2.3.0, you will get different behavior with PyNWB 1.4.0 and different versions of HDMF. I've raised issue #1343 to make a new PyNWB release and to change our release workflow to prevent this from happening again.

We also need to update PyNWB to have the same API regardless of what version of HDMF the file was written with.

Files written with HDMF 2.2.0 and below contain a pixel mask dataset of dtype int and shape (N, 3). When reading such a file, rois[0] returns an numpy array of dtype int32 and shape (M, 3) corresponding to the M (x, y, weight) values of the first ROI. rois[0][:, 0] returns all the x coordinates. rois[0]['x'] will not work.

Files written with HDMF 2.3.0 and up contain a pixel mask dataset of compound type (uint, uint, float) and shape (N, ). When reading such a file, rois[0] returns an numpy structured array of dtype [('x', '<u4'), ('y', '<u4'), ('weight', '<f4')] and shape (M, ) corresponding to the M (x, y, weight) values of the first ROI. rois[0]['x'] returns all the x coordinates. rois[0][:, 0] will not work.

It looks like suite2p previously used the rois[0][:, 0] format, then updated to rois[0]['x'] here, and is now at the latest version. Lines 43-45, e.g., np.array([rois[n][i][0].astype("int") for i in range(len(rois[n]))]) should work for files written with any version of HDMF.

I am puzzled as to how you got rois[0] as a tuple rather than a numpy ndarray. I think that line 40 in suite2p/io/nwb.py will always be True. Could you please elaborate on this result in case there is another issue?

[carsen-stringer]

Ah you're right it's an np.ndarray that's structured! Sorry I didn't realize that. You're right, one of those should work -- I was checking for an np.ndarray because I assumed the new format was a dictionary :) thanks so much for all the help. The way I've written it now should work for either case:

"ypix": np.array([rois[n][i][0].astype("int") for i in range(len(rois[n]))]),
 "xpix": np.array([rois[n][i][1].astype("int") for i in range(len(rois[n]))]),
"lam": np.array([rois[n][i][-1] for i in range(len(rois[n]))])