CODEWITHSUNDEEP
hadoopapache-kafkaflume
har777
har777

Reputation: 503

Writing to custom HDFS directory based on message data for Kafka -> Flume -> hdfs ingestion

How can I write to custom hdfs directory using flume based on message type in Kafka message ?

Say kafka message: {"type": "A", "data": "blah"} having "A" in type field should be written to /data/A, message: {"type": "B", "data": "blah"} having "B" in type field should be written to /data/B, etc

Upvotes: 1

Views: 157

Answers (1)

Guo
Guo

Reputation: 1803

I think you need Custom flume sink. Apache Flume custom sink

Upvotes: 1

Related Questions