CODEWITHSUNDEEP
amazon-s3apache-kafkaapache-kafka-connectconfluent-platform
clay
clay

Reputation: 20390

Kafka Confluent S3 Connector "Failed to find class"

I'm trying a simple quickstart example and I get:

Caused by: org.apache.kafka.connect.errors.ConnectException: Failed to find any class that implements Connector and which name matches io.confluent.connect.s3.S3SinkConnector, available connectors are: io.confluent.connect.jdbc.JdbcSinkConnector, io.confluent.connect.jdbc.JdbcSourceConnector, org.apache.kafka.connect.tools.MockSinkConnector, org.apache.kafka.connect.file.FileStreamSourceConnector, org.apache.kafka.connect.sink.SinkConnector, io.confluent.connect.elasticsearch.ElasticsearchSinkConnector, org.apache.kafka.connect.tools.MockConnector, io.confluent.connect.hdfs.HdfsSinkConnector, org.apache.kafka.connect.file.FileStreamSinkConnector, org.apache.kafka.connect.tools.VerifiableSourceConnector, org.apache.kafka.connect.tools.VerifiableSinkConnector, org.apache.kafka.connect.tools.SchemaSourceConnector, org.apache.kafka.connect.source.SourceConnector, org.apache.kafka.connect.tools.MockSourceConnector, io.confluent.connect.hdfs.tools.SchemaSourceConnector

This is a brand new VM running the latest Confluent Platform 3.2.0 installed via Debian apt package. I can see that it definitely is running CP 3.2.0. Why wouldn't the S3 connector class be available?

Not sure if this is related, but if I do:

ls -l /etc/ | grep kafka
drwxr-xr-x 2 root root           4096 Apr  5 04:41 kafka
drwxr-xr-x 2 root root           4096 Apr  5 04:39 kafka-connect-elasticsearch
drwxr-xr-x 2 root root           4096 Apr  5 04:39 kafka-connect-hdfs
drwxr-xr-x 2 root root           4096 Apr  5 04:39 kafka-connect-jdbc
drwxr-xr-x 2 root root           4096 Apr  5 04:39 kafka-rest

The docs reference a kafka-connect-s3 directory that clearly doesn't exist in the installation.

I'm using the Open Source variant of the Confluent Platform but according to https://www.confluent.io/product/compare/, the open source version includes the Confluent Amazon S3 Connect connector.

Upvotes: 4

Views: 2672

Answers (2)

Hans Jespersen
Hans Jespersen

Reputation: 8335

This was a packaging error in 3.2.0 and is fixed in the latest Confluent version which as of today (May 17th, 2017) is 3.2.1

There is also a bug fix to the S3 Connector in the 3.2.1 version so it's better to use that version anyway.

Upvotes: 1

Kelvin Low
Kelvin Low

Reputation: 746

As a workaround, you can download the zip or tar version from http://confluent.io

From there you'll find kafka-connect-s3 in the etc and share/java directory.

Upvotes: 1

Related Questions