CODEWITHSUNDEEP
postgresqlapache-kafkaapache-kafka-connectdebezium
Anirudha Nanda
Anirudha Nanda

Reputation: 43

Kafka Connect Debezium postgres

I am trying to connect a Postgres SQL db hosted in Amazon RDS with a Kafka topic using Debezium.

I am following the following tutorial :

http://debezium.io/docs/tutorial/

My kafka and kafka connect services start up fine and the kafka connect service also picks up my debezium postgres connector jars in /usr/share/java dir.

However on trying to attach the postgres config json via the kafka connect API using the following curl command :

curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d @connector1.json

I end up getting the following error:

 [2018-06-13 23:45:44,749] ERROR Uncaught exception in REST call to /connectors/ (org.apache.kafka.connect.runtime.rest.errors.ConnectExceptionMapper:61)
org.apache.kafka.connect.errors.ConnectException: Could not intialize type registry
    at io.debezium.connector.postgresql.connection.PostgresConnection.<init>(PostgresConnection.java:68)
    at io.debezium.connector.postgresql.PostgresConnector.validate(PostgresConnector.java:95)
    at org.apache.kafka.connect.runtime.AbstractHerder.validateConnectorConfig(AbstractHerder.java:277)
    at org.apache.kafka.connect.runtime.distributed.DistributedHerder$6.call(DistributedHerder.java:534)
    at org.apache.kafka.connect.runtime.distributed.DistributedHerder$6.call(DistributedHerder.java:531)
    at org.apache.kafka.connect.runtime.distributed.DistributedHerder.tick(DistributedHerder.java:267)
    at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:216)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.postgresql.util.PSQLException: The connection attempt failed.
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:275)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
    at org.postgresql.Driver.makeConnection(Driver.java:431)
    at org.postgresql.Driver.connect(Driver.java:247)
    at io.debezium.jdbc.JdbcConnection.lambda$patternBasedFactory$1(JdbcConnection.java:161)
    at io.debezium.jdbc.JdbcConnection.connection(JdbcConnection.java:585)
    at io.debezium.connector.postgresql.connection.PostgresConnection.<init>(PostgresConnection.java:65)
    ... 11 more
Caused by: java.net.SocketTimeoutException: connect timed out
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at org.postgresql.core.PGStream.<init>(PGStream.java:62)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:144)
    ... 18 more

Could some please advice on the exception and how to resolve it?

Is there a property/configuratuion that I may be missing here?

Upvotes: 2

Views: 4183

Answers (1)

Jiri Pechanec
Jiri Pechanec

Reputation: 1976

Do you have postgres database port configured to be accessible from outside? IIRC RDS database uses SSL for connection so it might be necessary to configure SSL also in postgres connector.

Upvotes: 2

Related Questions