CODEWITHSUNDEEP
enumsapache-kafkaunionavro
Deepu
Deepu

Reputation: 21

Issue while including enum type in unions within avro schema

I am working with Apache Kafka to send messages to Kafka topics. I am trying to use unions in Avro Schemas including enum types for message validation. But I am facing an issue with the usage of enum types within union. I am using Kafka REST API through POSTMAN tool to post a record/message to a topic with schema validation. Below is the request payload including schema and records inline - 

{
    "key_schema": "{\"type\": \"record\", \"name\": \"key\", \"fields\": [{\"name\": \"keyInput\", \"type\": \"string\"}]}",
    "value_schema": "{\"type\": \"record\", \"name\": \"value\", \"fields\": [{\"name\": \"valueInput1\", \"type\": \"string\"},{\"name\": \"valueInput2\",\"type\":[{\"type\":\"enum\",\"name\":\"actorobjType\",\"symbols\":[\"Agent\",\"Group\"]},\"null\"],\"default\":null}]}",
    "records": [
        {
            "key": {
                "keyInput": "testUser-key"
            },
            "value": {
                "valueInput1": "testUser-value",
                "valueInput2": "Agent"
            }
        }
    ]
}

I am getting the following error when I am trying to insert a record using above request payload -

{
  "error_code": 42203,
  "message": "Conversion of JSON to Avro failed: Failed to convert JSON to Avro: Expected start-union. Got VALUE_STRING"
}

After searching in different sites including stackoverflow, I came through a suggestion

asking to explicitly specify the type while passing the record as below - 

{
    "key_schema": "{\"type\": \"record\", \"name\": \"key\", \"fields\": [{\"name\": \"keyInput\", \"type\": \"string\"}]}",
    "value_schema": "{\"type\": \"record\", \"name\": \"value\", \"fields\": [{\"name\": \"valueInput1\", \"type\": \"string\"},{\"name\": \"valueInput2\",\"type\":[{\"type\":\"enum\",\"name\":\"actorobjType\",\"symbols\":[\"Agent\",\"Group\"]},\"null\"],\"default\":null}]}",
    "records": [
        {
            "key": {
                "keyInput": "testUser-key"
            },
            "value": {
                "valueInput1": "testUser-value",
                "valueInput2": {
                    "enum": "Agent"
                }
            }
        }
    ]
}

But then I face the below error -

{
  "error_code": 42203,
  "message": "Conversion of JSON to Avro failed: Failed to convert JSON to Avro: Unknown union branch enum"
}

The same suggestion worked fine for unions with other types like string and map, but with unions including enum, that does not seem to work.

I also thought there may be some other type which needs to be used for enum specification, Hence I tried some other words like below -

"valueInput2": {
                    "string": "Agent"
                }

and

"valueInput2": {
                    "enumeration": "Agent"
                }

But none of them seem to work. Please help me resolve this.

Upvotes: 2

Views: 3258

Answers (2)

Martyn Davis
Martyn Davis

Reputation: 624

I ended up here, and davis michael's answer gave a hint, which helped me eventually figure it out.

Within the context of the question,

"valueInput2": {
    "actorobjType": "Agent"
}

Upvotes: 1

davis michael
davis michael

Reputation: 1

As ENUM type is not exist in JSON format, value type should be changed to correct one: namespace + type name In your case, it will be namespace + actorobjtype : "agent"

Upvotes: 0

Related Questions