You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It is possible to have multiple Avro Schemas present within the same input directory. Usually this occurs when the data format changes over time. Need to have a utility method to create a pipe which can read multiple schemas (probably as a multi-tap). Also, need to return fields and data types as a normalized union of the two schemas, using the following rules:
If data types of the fields do not conflict, simply take the union of the fields and data types of all input Schemas.
If the data types conflict, but can be promoted, then promote them (i.e. int -> long, float -> double, int -> double).
Otherwise, convert conflicting data types to String (when converting BytesWritable to String, use Hex)
The text was updated successfully, but these errors were encountered:
I think I'd have a requirement like the scheme names have to match, or at least share a prefix, so that you don't wind up with crazy results if you accidentally mix in a file with a completely different scheme.
It is possible to have multiple Avro Schemas present within the same input directory. Usually this occurs when the data format changes over time. Need to have a utility method to create a pipe which can read multiple schemas (probably as a multi-tap). Also, need to return fields and data types as a normalized union of the two schemas, using the following rules:
The text was updated successfully, but these errors were encountered: