Skip to content

Commit

Permalink
Revert "Switch to LZF as default HDF5 compression"
Browse files Browse the repository at this point in the history 
This reverts commit 3e5b619.
  • Loading branch information
@gipert
gipert committed May 4, 2024
1 parent f26be5b commit 300a0ab
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/lgdo/lh5/_serializers/write/array.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

log = logging.getLogger(__name__)

DEFAULT_HDF5_SETTINGS: dict[str, ...] = {"shuffle": True, "compression": "lzf"}
DEFAULT_HDF5_SETTINGS: dict[str, ...] = {"shuffle": True, "compression": "gzip"}


def _h5_write_array(
Expand Down

3 comments on commit 300a0ab

@lvarriano
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Related to #77

@gipert
Copy link
Member Author

@gipert gipert commented on 300a0ab May 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @lvarriano, I reverted the change because, as you know, LZF is not a HDF5 built-in. We should avoid it for data preservation. Oliver was wondering if we could try SZIP, which is instead built-in.

@lvarriano
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what you mean by "data preservation." SZIP is worse than GZIP for our data (see #77) and also has license/distribution issues if we care about that. https://docs.hdfgroup.org/archive/support/HDF5/doc/RM/RM_H5P.html#Property-SetSzip

Please sign in to comment.