Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Support for complex "fancy" indexing #16

Open
manzt opened this issue Jan 16, 2020 · 5 comments
Open

[Feature] Support for complex "fancy" indexing #16

manzt opened this issue Jan 16, 2020 · 5 comments
Labels
enhancement New feature or request

Comments

@manzt
Copy link
Collaborator

manzt commented Jan 16, 2020

I wanted to move this from the docs to here.

In python, zarr supports the ability to index arrays using binary and integer masks:

import zarr

z = zarr.open("foo.zarr")
z.shape
# (5, 50, 50)

arr1 = z[[1,2,3],:,:]
arr1.shape
# (3, 50, 50)

arr2 = z[[False, True, False, True, True]])
arr1.shape == arr2.shape
# True

I am not sure what this would entail implementation-wise, but I'm guessing a new Indexer is needed. I would be interested in hearing what ideas you had in mind!

@gzuidhof
Copy link
Owner

For reference, the zarr documentation on fancy indexing.

Yes, in the Python implementation they have their own indexer, see this one for the boolean indexing, which then calls this. I would say that the implementation is just a matter of typing out the same code but in Typescript and add a bunch of test cases for it (you can just copy the Python zarr ones), it's not very complex.

The coordinate array setting is much more complicated, it relies on numpy broadcasting and a few other numpy functions, so that would have to be ported to TS. See here for the indexer code. One could implement it without supporting broadasting to simplify it a bit. I'm not sure it's worth the hassle of implementing right now, surely the coordinate indexing is easy to convert to a boolean mask for arrays that are not huge - and how often do you really use it?

@manzt
Copy link
Collaborator Author

manzt commented Jan 16, 2020

Thank you for the detailed response! I will look into the python implementation and translating into TS.

I agree. It would be ideal to have both boolean indexing and coordinate indexing, but having the functionality of "fancy" indexing at all is what is desired. Prefer the simpler route for now, and perhaps add the other in the future!

@gzuidhof
Copy link
Owner

Oh, actually the boolean indexing builds upon the coordinate indexing, so it looks like it will be quite complicated to implement either..

@manzt
Copy link
Collaborator Author

manzt commented Jan 17, 2020

I see. Thanks for looking into this.

@alimanfoo
Copy link

alimanfoo commented Jan 17, 2020 via email

@manzt manzt added the enhancement New feature or request label Feb 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants