-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for custom conversion #27
Comments
I think this and issue #26 can be solved together using Cascading 2.x's typed
For sink taps, the Cascading |
I should note that I've started doing some modifications to the project that are related to this, but still fall short from what's desired. My forked branch is here: https://github.com/ldcasillas-progreso/cascading.avro/tree/field_types |
Hi Luis - I agree this would be useful, especially on the source side...I just ran into the issue of only needing a few fields from an Avro file, but having to read everything in. Though I then started looking at using Parquet (with Avro records) as an even more efficient approach to this issue - have you ever tried that? |
I gave |
OK, just curious about parquet-cascading. I'm hoping to roll out a 2.6 release that has Silas's fairly significant changes. These are in the version-2.6 branch; no idea if this would change your approach to the above, but I think it would be worth a look. |
I have a scheme that has many fields as avro "bytes". These are converted to BytesWritable (BTW, why not Byte[] ?). But I want to use them as String. So I need an extra step to convert. And the BytesWritable object is created unnecessarily. Would be nice to be able to specify custom conversions for fields to avoid the generic ones.
The text was updated successfully, but these errors were encountered: