CODEWITHSUNDEEP
etlapache-nifidruidbigdata
Bo.
Bo.

Reputation: 675

Apache Nifi: load data from REST to Druid

I'm using Apache NiFi.

What is the proper way to load data from REST API to Druid?

Upvotes: 0

Views: 940

Answers (2)

Slim Bouguerra
Slim Bouguerra

Reputation: 359

You have 2 options:

  1. Option 1 A simple way to solve this it to first make NiFi dump the data into a Kafka Queue Then point Druid Realtime nodes to pull data out of the Kafka Queue to index data on realtime.
  2. Option 2 is to bring the data from NiFi to Storm cluster apply some transformation (eg Stream joins ..) if needed then use Tranquility to push data to Druid.

I am not an NiFi expert, but i think you will always need something like kafka to buffer events since i don't think NiFi does offer a buffering queue (again am not 100% sure about this claim).

Upvotes: 1

Joe Witt
Joe Witt

Reputation: 2172

There are not any apache community based connectors for Druid yet but we really should make one soon. Hopefully someone will contribute one soon.

Upvotes: 1

Related Questions