CODEWITHSUNDEEP
apache-nifi
raghu
raghu

Reputation: 398

Nifi - atttributes to json - not able to generate the required json from an attribute

The flowfile content is

{ "resourceType": "Patient", "myArray": [1, 2, 3, 4] }

I use EvaluateJsonPath processor to load the "myArray" to an attrribute myArray.

enter image description here Then I use the processor AttributesToJSON to create a json from myArray. enter image description here

But in the flowfile content, what I get is

{"myArray":"[1,2,3,4]"}

I expected the flowfile to have the following content.

{"myArray":[1,2,3,4]}

Here are the flowfile attributes enter image description here How can I get "myArray" as an array again in the content?

Upvotes: 4

Views: 8024

Answers (2)

user3523883
user3523883

Reputation: 1

Not all credit goes to me but I was pointed to a better simpler way to achieve this. There are 2 ways.

Solution 1 - and the simplest and elegant Use Nifi JoltTransformJSON Processor. The processor can make use of Nifi expression language and attributes in both left or right hand side of the specification syntax. This allows you to quickly use the JOLT default spec to add new fields (from flow-file attributes) to a new or existing JSON.

Ex:

{"customer_id": 1234567, "vckey_list": ["test value"]}

both of those fields values are stored in flow-file attributes as a result of a EvaluateJSONPath operation. Assume "customer_id_attr" and ""vckey_list_attr". We can simply generate a new JSON from those flow-file attributes with the "default" jolt spec and the right hand syntax. You can even add addition expression language functions to the processing

[
  {
    "operation": "default",
    "spec": {
      "customer_id": ${customer_id_attr},
      "vckey_list": ${vckey_list_attr:toLower()}
    }
  }
]

This worked for me even when storing the entire JSON, path of "$", in a flow-file attribute.

Solution 2 - complicated and uglier Use a sequence Nifi ReplaceText Processor. First use a ReplaceText processor to append the desired flow-file attribute to the file-content. replace_text_processor_1

If you are generating a totally new JSON, this would do it. If you are trying to modify an existing one, you would need to first append the desired keys, than use ReplaceText again to properly format as a new key in the existing JSON, from

{"original_json_key": original_json_obj}{"customer_id": 1234567, "vckey_list": ["test value"]}

to

{"original_json_key": original_json_obj, "customer_id": 1234567, "vckey_list": ["test value"]}

using replace_text_processor_2

Then use JOLT to do further processing (that's why Sol 1 always makes sense)

Hope this helps, spent about half a day figuring out the 2nd Solution and was pointed to Solution 1 by someone with more experience in Nifi

Upvotes: 0

notNull
notNull

Reputation: 31470

Use record oriented processors like Convert Record processor instead of using EvaluateJsonPath,AttributesToJSON processors.

RecordReader as JsonPathReader

JsonPathReader Configs: Path reader

AvroSchemaRegistry:

{
    "namespace": "nifi",
    "name": "person",
    "type": "record",
    "fields": [     
        { "name": "myArray", "type": {
            "type": "array",
            "items": "int"
        }}
    ]
}

JsonSetWriter: Use the same AvroSchemaRegistry controller service to access the schema. To access the AvroSchema you need to set up schema.name attribute to the flowfile.

Output flowfile content would be

[{"myArray":[1,2,3,4]}]

please refer to this link how to configure ConvertRecord processor

(or)

if your deserved output is {"myArray":[1,2,3,4]} without [](array) then use ReplaceText processor instead of AttributesToJson Processor.

ReplaceText Configs: enter image description here

Upvotes: 7

Related Questions