CODEWITHSUNDEEP
protocol-buffersavro
user882447
user882447

Reputation: 41

Transform protobuf to avro

I have data serialized in protobuff format and I want to transform it to an Avro serialization.

I have no problem reading the proto data using

    ProtoTest.Msg msg = buildMessage();
    ProtobufData protobufData = ProtobufData.get();
    Schema protoSchema = protobufData.getSchema(ProtoTest.Msg.class);
    Object o = protobufData.newRecord(msg, protoSchema);

The resulting o is again a protobuf object. Now I want to write o as avro with the same schema

    GenericDatumWriter genericDatumWriter = new GenericDatumWriter(protoSchema);
    OutputStream out = new ByteArrayOutputStream();
    Encoder encoder = EncoderFactory.get().binaryEncoder(out, null );
    genericDatumWriter.write(o, encoder);

But running the code above throws next exception at the write method

java.lang.ClassCastException: example.avro.ProtoTest$Msg cannot be cast to org.apache.avro.generic.IndexedRecord
at org.apache.avro.generic.GenericData.getField(GenericData.java:526)
at org.apache.avro.generic.GenericData.getField(GenericData.java:541)
at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:104)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)
at hermes.mediationOrchestrator.AvroFileWriteTest.testWriter3(AvroFileWriteTest.java:115)

How can I transform proto object into avro object?

Regards, Ronen.

Upvotes: 4

Views: 4714

Answers (1)

Doug Cutting
Doug Cutting

Reputation: 466

You should use a ProtobufDatumWriter instead of a GenericDatumWriter.

http://avro.apache.org/docs/current/api/java/org/apache/avro/protobuf/ProtobufDatumWriter.html

Also, use a ProtobufDatumReader to read protobuf data.

Lastly, Avro questions are more promptly answered on Avro's mailing lists.

http://avro.apache.org/mailing_lists.html

Upvotes: 2

Related Questions