CODEWITHSUNDEEP
javaavro
Marcin_S
Marcin_S

Reputation: 539

How to Pass Null value while serializing Avro message

I have field as defined as Map<String, String> map; , I taking request from a client and sending this to Kafka Topic having schema defined in schema registry.

In schema I have defined this as:

 union{null, map<string>} map;

But when someone is sending me a value as null I am getting this error while pushing the message to Kafka topic :

Caused by: org.apache.kafka.common.errors.SerializationException: Error serializing Avro message
Caused by: java.lang.NullPointerException: null of string of map of union of event.AvroFlumeEvent
    at org.apache.avro.generic.GenericDatumWriter.npe(GenericDatumWriter.java:184)
    at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:178)
    at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:83)
    at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
    at io.confluent.kafka.serializers.AbstractKafkaAvroSerializer.serializeImpl(AbstractKafkaAvroSerializer.java:95)
    at io.confluent.kafka.serializers.KafkaAvroSerializer.serialize(KafkaAvroSerializer.java:53)
    at org.apache.kafka.common.serialization.ExtendedSerializer$Wrapper.serialize(ExtendedSerializer.java:65)
    at org.apache.kafka.common.serialization.ExtendedSerializer$Wrapper.serialize(ExtendedSerializer.java:55)
    at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:841)
    at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:803)
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
    at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException

Input :

"dimensions": {
      "countryCode": "US",
      "Number": NULL

}

How to make take care of nulls can it be defined it in schema itself ?

Upvotes: 0

Views: 2519

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191681

That schema says map values are not unions, so they must be non-null strings. The map itself can be null, though

I'm not if this is valid in IDL, but you can try, assuming you didn't want the map to be nullable

map<union{null, string}>} map;

Otherwise, you could define your deserializers to treat empty strings or some default value as a nullable field

Upvotes: 1

Related Questions