Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IO] support external data in dictdump ">" links #4054

Open
woutdenolf opened this issue Jan 26, 2024 · 3 comments
Open

[IO] support external data in dictdump ">" links #4054

woutdenolf opened this issue Jan 26, 2024 · 3 comments

Comments

@woutdenolf
Copy link
Contributor

Currently we only support internal and external links:

elif key.startswith(">"):

A use case has come up (Bliss NeXus writer) where an EDF should be linked.

@t20100
Copy link
Member

t20100 commented Mar 21, 2024

Do you mean supporting external datasets?

h5py.Group.create_dataset:

  • external – Store the dataset in one or more external, non-HDF5 files. This should be an iterable (such as a list) of tuples of (name, offset, size) to store data from offset to offset + size in the named file. Each name must be a str, bytes, or os.PathLike; each offset and size, an integer. The last file in the sequence may have size h5py.h5f.UNLIMITED to let it grow as needed. If only a name is given instead of an iterable of tuples, it is equivalent to [(name, 0, h5py.h5f.UNLIMITED)].

@payno
Copy link
Member

payno commented Mar 21, 2024

I don't know by if it can be of any help please be aware that we did some proto for tomography:

@woutdenolf
Copy link
Contributor Author

Do you mean supporting external datasets?

Yes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants