CODEWITHSUNDEEP
apache-kafkahdfsapache-kafka-connectambariconfluent-platform
yuliansen
yuliansen

Reputation: 520

add connector plugin kafka by ambari

im new in apache environment and it have something that makes me really wonder. My kafka was installed via ambari. so everything can be started by ambari's GUI. When kafka installed via ambari it includes kafka-connect. But what if i the connector i need is not included . here are some the connectors that included.

/etc/kafka-connect-activemq
/etc/kafka-connect-elasticsearch
/etc/kafka-connect-ibmmq
/etc/kafka-connect-jdbc
/etc/kafka-connect-jms
/etc/kafka-connect-replicator
/etc/kafka-connect-s3
/etc/kafka-connect-storage-common
/etc/kafka-connect-activemq/ActiveMQSourceConnector.properties
/etc/kafka-connect-activemq/connect-avro-localhost.properties
/etc/kafka-connect-elasticsearch/quickstart-elasticsearch.properties
/etc/kafka-connect-ibmmq/IbmMQSourceConnector.properties
/etc/kafka-connect-ibmmq/connect-avro-localhost.properties
/etc/kafka-connect-jdbc/sink-quickstart-sqlite.properties
/etc/kafka-connect-jdbc/source-quickstart-sqlite.properties
/etc/kafka-connect-jms/ActiveMQSourceConnector.properties
/etc/kafka-connect-jms/JMSSourceConnector.properties
/etc/kafka-connect-jms/connect-avro-localhost.properties
/etc/kafka-connect-replicator/quickstart-replicator-unicluster.properties
/etc/kafka-connect-replicator/quickstart-replicator.properties
/etc/kafka-connect-replicator/replicator-connect-distributed.properties
/etc/kafka-connect-replicator/replicator-connect-standalone.properties
/etc/kafka-connect-replicator/replicator-log4j.properties
/etc/kafka-connect-replicator/replicator.properties
/etc/kafka-connect-s3/quickstart-s3.properties
/etc/kafka-connect-s3/quickstart-schema-source-for-s3.properties

but what i need is hdfs3-sink. How do i achieve it without confluent?

any suggestion will be appreciated so much. Thank you

Upvotes: 0

Views: 211

Answers (1)

Robin Moffatt
Robin Moffatt

Reputation: 32090

The only HDFS 3 connector for Kafka Connect that I'm aware of is this one from Confluent: https://www.confluent.io/hub/confluentinc/kafka-connect-hdfs3/.

To install it you can download the JAR directly, or use confluent-hub CLI tool to install it.

Upvotes: 2

Related Questions