CODEWITHSUNDEEP
apache-kafkadruid
Hiba Rehman
Hiba Rehman

Reputation: 137

How to enable druid kafka ingestion?

I am following druid documentation but as I run the command

curl -XPOST -H'Content-Type: application/json' -d @quickstart/tutorial/wikipedia-kafka-supervisor.json http://localhost:8081/druid/indexer/v1/supervisor

I get the error 

Warning: Couldn't read data from file 
Warning: "quickstart/tutorial/wikipedia-kafka-supervisor.json", this 
makes an 
Warning: empty POST.

so where this file exists wikipedia-kafka-supervisor.json?

Upvotes: 2

Views: 604

Answers (1)

Giorgos Myrianthous
Giorgos Myrianthous

Reputation: 39820

You can find the content of this file on Github.

You'd need to first donwload Kafka: 

curl -O https://archive.apache.org/dist/kafka/0.10.2.2/kafka_2.12-0.10.2.2.tgz
tar -xzf kafka_2.12-0.10.2.2.tgz
cd kafka_2.12-0.10.2.2

Then fire up a Kafka broker: 

./bin/kafka-server-start.sh config/server.properties

and create a wikipedia topic to which the data will be sent: 

./bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic wikipedia

Finally, you can start the service to enable Druid Kafka ingestion:

curl -XPOST -H'Content-Type: application/json' -d @quickstart/tutorial/wikipedia-kafka-supervisor.json http://localhost:8090/druid/indexer/v1/supervisor

If the supervisor is successfully created, you will get the ID of the supervisor as a response; for example: 

 {"id":"wikipedia-kafka"}

Upvotes: 1

Related Questions