CODEWITHSUNDEEP
javaavro
chitwan
chitwan

Reputation: 263

Apache kafka with AVRO, where in the message does the schema id go?

I do have a number of queries about AVRO schema.

I have read that, we need to pass a schema id and the message in the Kafka event.The body of my Kafka event is like -

{ "componentName": "ABC", //some more fields, "payload": { "name" : "xyz", "age": "23" } }

In payload field, we provide the actual data. Here, where will I provide the schema id. I found one answer related to this at [link][1] 


  [1]: https://stackoverflow.com/questions/31204201/apache-kafka-with-avro-and-schema-repo-where-in-the-message-does-the-schema-id,

which said that there is encoder which takes a schema and finds it's schema id from schema registry. Is this encoder serializer or something different? Do we need to embed schema also in the message we are sending? how will encoder pick the schema?

Do we need to explicitly register schema in schema registry as said on this [link][1] 

Also, how will we associate a schema with a topic name?

Upvotes: 0

Views: 1442

Answers (1)

Vassilis
Vassilis

Reputation: 1054

I was looking at exactly this today and it seems the schema id is encoded as the first bytes of the message.

When the key or value is de-serialized (code):

ByteArrayOutputStream out = new ByteArrayOutputStream();
out.write(0);
out.write(ByteBuffer.allocate(4).putInt(e).array());

When serialized (code):

ByteBuffer e = this.getByteBuffer(payload);
int id1 = e.getInt();
Schema schema = this.schemaRegistry.getById(id1);

Upvotes: 1

Related Questions