You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It seems that the arraybuilder always assumes int64 when setting up the data. It would be nice to set the integer type like uint8 or uint64. This could help reduce memory load when building huge datasets that only contain smaller numbers. the same goes for floats for example
The text was updated successfully, but these errors were encountered:
That's true about ArrayBuilder, but ArrayBuilder is only for ingesting data from untyped sources, such as Python objects or JSON. Those integers don't have any fixed-width type, like int32 or int64.
There are a few functions for setting the type of Awkward Array, after it has been built from an ArrayBuilder. ak.values_astype sets the numeric type of all numbers in an array, wherever they are, and ak.enforce_type provides more control by forcing a whole array to fit a given type description. (You can get the type from one array, either as a ak.types.Type object or as a string, change the int64 to something else, and then enforce-type.)
I think it makes sense to change the type after the fact because the arrays of Python objects or JSON can't be too big if you're able to process them at all. And if you're coming from NumPy arrays ak.from_numpy, the dtype is preserved.)
Is it okay if I turn this from an issue/feature request into a Discussion?
Description of new feature
It seems that the arraybuilder always assumes int64 when setting up the data. It would be nice to set the integer type like uint8 or uint64. This could help reduce memory load when building huge datasets that only contain smaller numbers. the same goes for floats for example
The text was updated successfully, but these errors were encountered: