CODEWITHSUNDEEP
avro
hshang
hshang

Reputation: 21

Can I Write Two Schemas in One AVRO File

I have data like this: schema1, a1, a2, ..., an, schema2, b1, b2, ...., bm. I know exactly how many data items I have for each schema. Can I write the two schema/data in one AVRO file, instead of two?

dataFileWrite API gives only create(). There is no append() for me to writer the second schema after the last data item of the first schema.

Upvotes: 2

Views: 1746

Answers (1)

miljanm
miljanm

Reputation: 936

You should create a new union schema using Schema.createUnion(schema1,schema2) and use that one as the writer schema for your file. When reading the data you either use the union schema again if you have both types in your file or just the schema that you know is present.

Upvotes: 1

Related Questions