CODEWITHSUNDEEP
apache-kafkaapache-kafka-connectconfluent-platform
user3066281
user3066281

Reputation: 63

Kafka connect confluent jdbc does not control session pool in MSSQL database

I am working with Kafka connect and confluent jdbc. Integrate a source connector with Mssql and a few days ago the operating area warned us that there is a high number of sessions in the "sleeping" state in the database. I need to control those sessions but apparently the connector (confluent jdbc) doesn't have those properties in its configuration.

Do you have any ideas to correct this problem?

Upvotes: 1

Views: 1294

Answers (1)

Robin Moffatt
Robin Moffatt

Reputation: 32090

Kafka Connect will run a minimum of one task per connector. Each connector is isolated from the other and other than sharing a runtime environment is isolated from the others.

Therefore if you have 27 connectors sourcing from the same database, you will have a minimum of 27 connections to the database.

If you can't reduce the number of connectors (e.g. by have one connector pull from multiple tables), then the only option I think you have is to speak to your DBA about enforcing some kind of resource management on the RDBMS side. For example, on Oracle the Resource Manager option can be used for this.

Upvotes: 1

Related Questions