CODEWITHSUNDEEP
apache-kafkaapache-kafka-connectopenedge
theSword
theSword

Reputation: 13

Kafka Connect won't pick up custom connector

I am trying to use kafka connect in a docker container with a custom connector (PROGRESS_DATADIRECT_JDBC_OE_ALL.jar) to connect to an openedge database.

I have put the JAR file in the plugin path (usr/share/java) but it won't load as a connector.

COPY Openedge/PROGRESS_DATADIRECT_JDBC_OE_ALL.jar /usr/share/java/progress

I can load another (standard) connector by putting it in the plugin path. This works

COPY confluentinc-kafka-connect-jdbc-10.3.2 /usr/share/java/confluentinc-kafka-connect-jdbc-10.3.2

A little lost on how to move forward and I'm very new to kafka. My main sources of information are openedge to kafka streaming and How to use Kafka connect

Upvotes: 0

Views: 1122

Answers (2)

theSword
theSword

Reputation: 13

@OneCricketeer had the solution. As a retro for me and hopefully helpful to someone else, here are my steps to make this work. Copy the JDBC Connector to CONNECT_PLUGIN_PATH and install with confluent hub install:

COPY confluentinc-kafka-connect-jdbc-10.3.2.zip /usr/share/java
RUN confluent-hub install --no-prompt /usr/share/java/confluentinc-kafka-connect-jdbc-10.3.2.zip

Copy the driver (I ended up using openedge.jar) to the path where other jars are located (like sqllite) according to @OneCricketeer suggestion.

COPY Openedge/openedge.jar /usr/share/confluent-hub-components/confluentinc-kafka-connect-jdbc/lib

Verify with this by enabling DEBUG as suggested by this page Finally add a .properties file to create the connector. In my case based on the one in “openedge to kafka streaming” link above

Upvotes: 1

OneCricketeer
OneCricketeer

Reputation: 191993

JDBC Drivers are not Connect plugins, nor are they connectors themselves.

You'd need to set the JVM CLASSPATH environment variable for detecting JDBC Drivers, as with any Java process.

The instructions on the linked site suggest you should copy the JDBC Drivers into the directory for the existing Confluent JDBC connector. While you could use a Docker COPY command, the better way would be to use confluent-hub install

Upvotes: 0

Related Questions