Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dynamic bitset blocks size should not be configurable and be a single byte #184

Open
Alex-PLACET opened this issue Aug 23, 2024 · 2 comments

Comments

@Alex-PLACET
Copy link
Collaborator

Alex-PLACET commented Aug 23, 2024

Currently, it's possible to choose any size for the blocks in the Dynamic bitset's template argument, but in the apache arrow documentation: https://arrow.apache.org/docs/dev/format/Columnar.html#validity-bitmaps
But using a block size greater than a byte will lead to endianess issues

@Klaim
Copy link
Collaborator

Klaim commented Aug 26, 2024

dynamic_bitest itself is a generic type, it's not designed to be specific to arrow. It's usage in the library, for example there, here the block size is 8bit, which matches the spec, isnt it? It can't be changed by the user AFAIK.

@Alex-PLACET
Copy link
Collaborator Author

In that case, it requires modifications to works on big endian architectures. I don't know if we really want to add complexity for cases we don't use.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants