Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dak.to_parquet should default to extensionarray=True, like @2ak.to_parquet` #540

Open
jpivarski opened this issue Sep 5, 2024 · 1 comment

Comments

@jpivarski
Copy link
Collaborator

  1. This option is required for status to round-trip through Parquet.
  2. The ak and dak versions of a function shouldn't have different defaults, since ak dispatches to dak, and this can make it appear to contradict its documentation.
@martindurant
Copy link
Collaborator

This option is required for status to round-trip through Parquet.

I was under the impression that we would be moving away from extension arrays and putting the required metadata into the global parquet k-v store instead. The original reason for False here was, that some combinations in the past caused hard crashes in arrow on read. We really don't want that! Perhaps it has all been fixed, but it still feels like the more complex option that people outside of HEP won't be wanting*.

* these is an argument we can have about who we expect to directly use ak dispatch versus dask-awkward versus akimbo or other avenues for reaching this code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants