CODEWITHSUNDEEP
sql-serverapache-kafkaconfluent-platformcdc
Mário de Sá Vera
Mário de Sá Vera

Reputation: 388

Kafka Connect CDC to MS SQL sourceOffset exception

We are using Confluent MS SQL CDC connector and the connection descriptor is : 

curl -X POST -H \
  "Content-Type: application/json" --data '{
    "name" : "yury-mssql-cdc1",
    "config" : {
      "connector.class" : "io.confluent.connect.cdc.mssql.MsSqlSourceConnector",
      "tasks.max" : "1",
      "initial.database" : "test2",
      "username" : "user",
      "password" : "pass",
      "server.name" : "some-server.eu-west-1.rds.amazonaws.com",
      "server.port" : "1433",
      "change.tracking.tables" : "dbo.foobar"
    }
  }' \
   http://ip-10-0-0-24.eu-west-1.compute.internal:8083/connectors

the whole infrastructure is deployed at AWS... and the exception is :

ERROR Exception thrown while querying for ChangeKey

{databaseName=test2, schemaName=dbo, tableName=foobar} (io.confluent.connect.cdc.mssql.QueryService:94) java.lang.NullPointerException: sourceOffset cannot be null.

any help would be greatly appreciated.

Upvotes: 3

Views: 384

Answers (1)

user10838797
user10838797

Reputation: 11

I found the answer, I think the problem is the way SQL server CDC is configured. We should not use the old way of setting CDC ( EXEC sys.sp_cdc_enable_db and EXEC sys.sp_cdc_enable_table )

Instead, use the following command to configure SQL server CDC

ALTER DATABASE [db name] SET CHANGE_TRACKING = ON (CHANGE_RETENTION = 2 DAYS, AUTO_CLEANUP = ON) GO ALTER DATABASE [db name] SET ALLOW_SNAPSHOT_ISOLATION ON GO ALTER TABLE [talbe name ] ENABLE CHANGE_TRACKING WITH (TRACK_COLUMNS_UPDATED = ON) GO

Upvotes: 1

Related Questions