CODEWITHSUNDEEP
jsonapache-kafkaapache-nifi
Ayoub E. Awabde
Ayoub E. Awabde

Reputation: 1

How to send message key with unique identiffier (eventID/UUID/filename) from Nifi to Kafka topic and see in kafka logs

I am trying to send a message (json file) to Kafka topic (publish), and to use the filename or the uuid as a kafka key so I could trace it in the logs. the message is sent and consumed later on but I am not able to trace the key in the logs so I could trace and identify the messages. The goal is to have a unique identifier for each message (using different filename/uuid for each event)

I have tried to use the evaluteJsonPath process in Nifi in order to add the filename as an attribute so it could be used, but still not able to find anything in kafka logs. In addition I have changed the configuration to debug mode in kafka logs =, but it didn't help

enter image description here

enter image description here

Upvotes: 0

Views: 2515

Answers (1)

Bryan Bende
Bryan Bende

Reputation: 18670

All flow files already have an attribute named 'filename', so you don't need to use EvaluateJsonPath unless you want to overwrite the filename attribute with something else coming from the json.

All flow files also have a 'uuid' attribute, so you could use either of those in the publish kafka processor.

The config of publish kafka looks correct in terms of referencing ${filename}, but I have no idea what kafka logs.

Upvotes: 1

Related Questions