CODEWITHSUNDEEP
hadoopapache-kafkaapache-kafka-connectconfluent-platform
Mario Stefanutti
Mario Stefanutti

Reputation: 262

Need to move small JSON messages from Kafka to HDFS with Kafka Connect but without using Confluent libs, if not completely free

I'd like to use Kafka Connect to move JSON messages from Kafka to HDFS and then Impala, only using OpenSource libs.

I was trying to understand if I can use the Confluent Sink library for Kafka Connect, without the need to use the entire Confluent distribution.

Are there other and/or better options to achieve this?

Upvotes: 0

Views: 152

Answers (1)

Robin Moffatt
Robin Moffatt

Reputation: 32110

The Kafka Connect HDFS 2 Sink is available under the Confluent Community Licence. It is a plugin for Apache Kafka; you do not have to run Confluent Platform to use it.

Upvotes: 1

Related Questions