CODEWITHSUNDEEP
apache-nifi
Justin
Justin

Reputation: 886

Nifi json efficiency parsing

I am not really sure how to go about this, I am trying to find the best way to get the data i want and i cant figure out how to do it without 3 different sets of evaluateJsonPath processors.

My data looks like this.

{
  "@timestamp": "2020-05-29T18:06:53.965Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "_doc",
    "version": "7.7.0"
  },
  "agent": {
    "ephemeral_id": "691847ce-87fa-49de-a635-555b31ed28f4",
    "hostname": "acal",
    "id": "c3336311-de5a-4563-b24b-655613b399aa",
    "version": "7.7.0",
    "type": "filebeat"
  },
  "log": {
    "offset": 3759198,
    "file": {
      "path": "/opt/zactivity.log"
    }
  },
  "message": "{\"eventDate\":\"2020-05-29 18:06:50.913\",\"status\":\"SUCCEEDED\",\"activityType\":\"AUTHENTICATION\",\"user\":\"admin\",\"accountId\":\"4dadb154\",\"ip\":\"111.11.1.111\",\"userRoles\":[\"ROLE_DELETE_ALL\",\"ROLE_INVOKE_ACTIONS\",\"ROLE_SAVE\",\"ROLE_CREATE\",\"ROLE_READ\",\"ROLE_SAVE\",\"ROLE_ADMINISTER\",\"ROLE_SHARE\",\"ROLE_MANAGE\",\"ROLE_EXPORT\",\"ROLE_MANAGE_ALL\",\"ROLE_MANAGE_ACTION\",\"ROLE_EDIT\",\"ROLE_ADMIN\",\"ROLE_MANAG\",\"ROLE_RAW_DATA_ACCESS\",\"ROLE_ADMINIST\"],\"userGroups\":[\"1fb\"],\"userType\":\"USER\",\"authenticationType\":\"USERNAME_PASSWORD\"} ",
  "input": {
    "type": "log"
  },
  "ecs": {
    "version": "1.5.0"
  },
  "host": {
    "mac": [
      "50:6b:8d:db:04:2e"
    ],
    "hostname": "local",
    "architecture": "x86_64",
    "os": {
      "version": "7 (Core)",
      "family": "redhat",
      "name": "CentOS Linux",
      "kernel": "3.10.0-1062.12.1.el7.x86_64",
      "codename": "Core",
      "platform": "centos"
    },
    "id": "a29a885a0aba48418d46414ecbe449e1",
    "containerized": false,
    "name": "local",
    "ip": [
      "11.11.11.11"
    ]
  }
}

I want need a few values from message and also log.file.path and maybe even the @timestamp.

I currently have an EvaluateJsonPath just for getting message, log.file.path, and @timestamp. I then route that based on the file path and send it to another EvaluateJsonPath just to take the message and make it a flowfile to pass it into another EvaluateJsonPath to parse out the values within message.

Is there a better way to do this instead of using 3 different EvaluateJsonPath's?

Upvotes: 0

Views: 42

Answers (1)

maxime G
maxime G

Reputation: 1771

you should be able to use only one EvaluateJsonPath. set Destination to flowfile-attribute and define 3 dynamic properties, each one for each values you want.

Upvotes: 1

Related Questions