CODEWITHSUNDEEP
javascriptapache-kafkaavroconfluent-schema-registry
shytikov
shytikov

Reputation: 9558

How to figure out avro version used?

I have Confluent registry that is out of my control, and producer that is based upon @kafkajs/confluent-schema-registry. Is there any way how I can understand which version of message format is used?

I can get encoded AVRO message, but it's just stream of bytes. Is there any way to understand which version of a message it actually is?

Upvotes: 0

Views: 1200

Answers (1)

Oliver
Oliver

Reputation: 36453

I see you are using Confluent who have their own version of the wire format.

Embedded in the leading bytes is the Schema ID which can be used to fetch the schema from their Schema Registry:

https://docs.confluent.io/platform/current/schema-registry/serdes-develop/index.html#wire-format

I am not sure how to manipulate the bytes in Javascript but what we've done in Scala :

  1. drop the first byte
  2. read the next 4 bytes turn that into an integer
  3. remaining bytes can be deserialized with the schema fetched from Schema Registry.

/Side Note: What is really annoying is this a vendor specific format even though the Apache Avro spec denotes a Single-object encoding which includes schema information.

Furthermore it also looks like Confluent seem uninterested in supporting the Apache Avro format. https://github.com/confluentinc/schema-registry/issues/1294

Upvotes: 1

Related Questions