CODEWITHSUNDEEP
jsonapache-nifijolt
Andrew Chisholm
Andrew Chisholm

Reputation: 6567

Support for concat in JSON Jolt in Apache NiFi

Using Apache NiFi I want to add a new field to all the elements within a JSON flow file based on the concatenation of two other fields. I am trying to use the JoltTransformJSON processor for this, however, the Jolt transform I want to use works OK using online tools but does not work within NiFi. I suspect a version issue but there may be something stupid with my Jolt specification.

The input JSON looks like this...

[
  {
    "id": 485842,
    "cc": 1,
    "x": 0,
    "y": null
  },
  {
    "id": 281733,
    "cc": 1,
    "x": 0,
    "y": 10
  },
  {
    "id": 721412,
    "cc": 12,
    "x": 0,
    "y": null
  }
]

The desired output I want is this...

[ {
  "id" : 485842,
  "cc" : 1,
  "x" : 0,
  "y" : null,
  "id_cc" : "485842_1"
}, {
  "id" : 281733,
  "cc" : 1,
  "x" : 0,
  "y" : 10,
  "id_cc" : "281733_1"
}, {
  "id" : 721412,
  "cc" : 12,
  "x" : 0,
  "y" : null,
  "id_cc" : "721412_12"
} ]

And the Jolt transform I use on the online site is...

[{
  "operation": "modify-default-beta",
  "spec": {
    "*": {
      "id_cc": "=concat(@(1,id),'_',@(1,cc))"
    }
  }
}]

In NiFi, I configure the JoltTransformJSON processor to have Modify-Default and I use this slightly modified Jolt Specification...

{
    "operation": "modify-default",
    "spec": {
      "*": {
        "id_cc": "=concat(@(1,id),'_',@(1,cc))"
      }
    }
}

NiFi validates this OK and the process runs. The output JSON consists of a single record only and a new field is added like this

"operation": "modify-default"

Is there a quick fix to the modify-default operation that will resolve this or is there an even easier way using a shift operation?

Thanks in advance for any pointers.

Upvotes: 2

Views: 2313

Answers (1)

Andrew Chisholm
Andrew Chisholm

Reputation: 6567

After some help from colleagues, I have found a way to make this work.

Firstly, set the Jolt Transformation DSL property of the JoltTransformJSON processor to be Chain.

Secondly, set the Jolt specification to the following...

[{
    "operation": "modify-default-beta",
    "spec": {
      "*": {
        "id_cc": "=concat(@(1,id),'_',@(1,cc))"
      }
    }
}]

The [] are vital as is the beta in the operation.

Thirdly, ensure the JSON fed into the Jolt processor is an array.

Get all this correct and the expected output will be produced.

Upvotes: 3

Related Questions