CODEWITHSUNDEEP
jsonapache-nifi
Maykid
Maykid

Reputation: 517

How to convert epoch to timestamp in NiFi within a JSON file?

I'm having issues with getting an epoch conversion to Timestamp to work properly. So far my example timestamp looks like the following:

{"createTime": 1510932843000}

What my end goal is to make it look like the following:

2017-11-17 3:34:03.000

The things I've tried so far are the UpdateRecord and JoltTransformation Processor. For the UpdateRecord I have tried various ways but all end in an error. The current code I have for this is:

${field.value:format("yyyy-MM-dd HH:mm:ss.SSS")}

Which results in the following error:

JSON Object due to java.lang.NumberFormatException: For input string: "2017-11-17 15:34:03.000": For input string: "2017-11-17 15:34:03.000"

I have also tried the code without the multiply(1000) to the same effect.

I have also tried a Jolt Transformation of the following code:

{
  "createTime": "${createTime:append('000'):format('yyyy-MM-dd HH:mm:ss.SSS')}"
}

This however results in the following:

"createTime": "1970-01-01 00:00:00.000"

Which isn't what I'm looking for as its the incorrect date result. Am I doing something wrong within my code itself or is another factor occurring? I've been working with this and searching all over for different kind of results and have tried multiple different formats with no success. Any help with this would be greatly appreciated!

Upvotes: 1

Views: 1132

Answers (1)

tonykoval
tonykoval

Reputation: 1232

My preferred solution:

Use a ScriptedTransformRecord processor:

  • Record Reader: JsonTreeReader
  • Record Writer: JsonRecordSetWriter
  • Script Language: Groovy
  • Script Body:
import java.time.Instant;
import java.time.format.DateTimeFormatter;
import java.time.ZoneId;

def formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss.SSS").withZone(ZoneId.of("Europe/Bratislava"))

record.setValue("createTimeFormatted", formatter.format(Instant.ofEpochMilli(record.getAsLong("createTime"))))
return record

Output json:

{
  "createTime" : 1510932843000,
  "createTimeFormatted" : "2017-11-17 16:34:03.000"
}

Another approach

Use 2 processors: JoltTransformJSON (convert type from Long to String) -> UpdateRecord (convert date).

JoltTransformJSON processor:

  • Jolt Transformation DSL: Chain
  • Jolt specification:
[
 {
    "operation": "modify-overwrite-beta",
    "spec": {
      "createTime": "=toString"
    }
  }
]

UpdateRecord processor:

  • Record Reader: JsonTreeReader
  • Record Writer: JsonRecordSetWriter
  • Replacement Value Strategy: Literal Value
  • /createTime (dynamic property): ${field.value:format("yyyy-MM-dd HH:mm:ss.SSS")}

Output json:

{
  "createTime" : "2017-11-17 16:34:03.000"
}

Upvotes: 2

Related Questions