CODEWITHSUNDEEP
apache-nifiavro
Mindaugo
Mindaugo

Reputation: 87

Nifi ConvertJSONToAvro doesn't parse records with nulls

I have two steps defined in my process:

  1. In JoltTransformJson processor, I append columns if not exist
[{
    "operation": "modify-default-beta",
    "spec": {
      "my_key": null,
  1. In ConvertJSONToAvro Im converting to Avro by specifiyng schema:
{
    "type": "record",
    "name": "my_name",
    "fields": [
        {
            "name": "my_key",
            "type": "string",

The problem is that ConvertJSONToAvro processor sends records to the failure queue if it detects null value. If it is an empty string, it works well. With strings, I can provide an empty string value, but with booleans, I need to use null.

How can I enable ConvertJSONToAvro processor to parse null values?

Upvotes: 1

Views: 280

Answers (1)

qaziqarta
qaziqarta

Reputation: 1944

For nullable values you need to use union of your Avro type and null value in your Apache Avro schema, like so:

{
    "type": "record",
    "name": "my_name",
    "fields": [
        {
            "name": "my_key",
            "type": ["null", "string"],

Upvotes: 1

Related Questions