CODEWITHSUNDEEP
jsoncsvapache-nifi
meh
meh

Reputation: 273

NiFi convert json to csv using ConvertRecord

i have a stream of json in apache nifi that contain dynamic fields (maximum 11 fields) and i want to convert it to csv file.

sample json:

{  
   "field1":"some text",
   "field2":"some text",
   "field3":"some text",
   "field4":"some text",
   "field5":"some text",
   "field6":"some text",
   "field7":"some text"
}

i don't wanna using replace or json evaluate; how i do it with ConvertRecord? using this processor is so odd and hard to work...

Clear expression about dynamic fields: i have 11 fields at total. one record may have contain 7 fields, and next record may contain 11 fields and next 9 fields...

Upvotes: 1

Views: 5666

Answers (1)

Sivaprasanna Sethuraman
Sivaprasanna Sethuraman

Reputation: 4132

The steps provided below will help you in getting this done.:

  • Connect your source processor which generates/outputs the JSON files to ConvertRecord.
  • Configure ConvertRecord and set 'Record Reader' to use JsonTreeReader controller service and 'Record Writer' to use CSVRecordSetWriter controller service
  • Configure both the controller services and set Schema Registry property to use AvroSchemaRegistry
  • Configure AvroSchemaRegistry. Go to 'Properties' tab and click the + button which lets you add a dynamic property.
  • Give some property name (ex: mySchema) and for the value, give the Avro schema expected for your input JSON. (You can use InferAvroSchema processor to generate Avro schema for your JSON)
  • Configure both JsonTreeReader and CsvRecordSetWriter and set the 'Schema Name' property to the name provided above, in this case, mySchema.
  • Connect the relationships of ConvertRecord to downstream processors according to your need.

Upvotes: 10

Related Questions