CODEWITHSUNDEEP
javaavro
lalit
lalit

Reputation: 21

Optional field in avro schema are not working

I am trying to use optional field in avro message. I am using avro 1.8.2.

If the optional field is missing in the JSON message then it doesn't work and throws following exception.

org.apache.avro.AvroTypeException: Expected field name not found: username

You can reproduce the issue with below files.

Step :1 -> Create a json file (test.json) - Here field username is missing in the JSON file

{
   "tweet":"Simple message.",
   "timestamp":7866844999
}

Step :2 -> Create a schema file (schema.avsc) - Here username is declared as optional field

{
   "type":"record",
   "name":"test_schema",
   "namespace":"com.test.avro",
   "fields":[
      {
         "name":"username",
         "type":[
            "null",
            "string"
         ],
         "default":null,
         "doc":"doc 1"
      },
      {
         "name":"tweet",
         "type":"string",
         "doc":"doc 2"
      },
      {
         "name":"timestamp",
         "type":"long",
         "doc":"doc 3"
      }
   ],
   "doc:":"A basic schema"
}

Step 3 - Run this command - (You need to download avro-tools jar)

java -jar avro-tools-1.8.2.jar fromjson --schema-file schema.avsc test.json

Upvotes: 0

Views: 1946

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191963

Avro defaults are only applicable at consumption/reading, not when serializing or writing Avro

Therefore, you need

{
   "tweet":"Simple message.",
   "username": null, 
   "timestamp":7866844999
}

Or actually add a real username since tweets should have them, anyway

Upvotes: 2

Related Questions