CODEWITHSUNDEEP
apache-nifi
ilovetolearn
ilovetolearn

Reputation: 2060

How do I parse a date field and generate a date in string format in NiFi

Each of my flow file contains 2000 records. I would like to parse 01/01/2000 into a column year = 2000, column month = Jan and column day = 01

i.e. the input column 01/01/2000 into 3 values separated by commas 01,Jan,2000

Upvotes: 3

Views: 3120

Answers (2)

mattyb
mattyb

Reputation: 12083

You can use UpdateRecord for this, assuming your input record has the date column called "myDate", you'd set the Replacement Value Strategy to Record Path Value, and your user-defined properties might look something like:

/day format(/myDate, "dd") /month format(/myDate, "MMM") /year format(/myDate, "YYYY")

Your output schema would look like this:

{
 "namespace": "nifi",
 "name": "myRecord",
 "type": "record",
 "fields": [
  {"name": "day","type": "int"},
  {"name": "month","type": "string"},
  {"name": "year","type": "int"}
 ]
}

Upvotes: 3

Bryan Bende
Bryan Bende

Reputation: 18630

Lets say you have a schema like this for a person with a birthday and you want to split out the birthday:

{
  "name": "person",
  "namespace": "nifi",
  "type": "record",
  "fields": [
    { "name": "first_name", "type": "string" },
    { "name": "last_name", "type": "string" },
    { "name": "birthday", "type": "string" }
  ]
}

You would need to modify the schema so it had the fields you want to add:

{
  "name": "person",
  "namespace": "nifi",
  "type": "record",
  "fields": [
    { "name": "first_name", "type": "string" },
    { "name": "last_name", "type": "string" },
    { "name": "birthday", "type": "string" },
    { "name": "birthday_year", "type": ["null", "string"] },
    { "name": "birthday_month", "type": ["null", "string"] },
    { "name": "birthday_day", "type": ["null", "string"] }
  ]
}

Lets say the input record has the following text:

bryan,bende,1980-01-01

We can use UpdateRecord with a CsvReader and CsvWriter, and UpdateRecord can populate the three fields we want by parsing the original birthday field.

enter image description here

If we send the output to LogAttribute we should see the following now:

first_name,last_name,birthday,birthday_year,birthday_month,birthday_day
bryan,bende,1980-01-01,1980,01,01

Here is the link to the record path guide for details on the toDate and format functions:

https://nifi.apache.org/docs/nifi-docs/html/record-path-guide.html

Upvotes: 6

Related Questions