CODEWITHSUNDEEP
apache-kafka-connectdebezium
Gignac
Gignac

Reputation: 210

Replication mysql via Debezium: missing table

I have three tables in MySQL database:

table1
table2
table3

and Debezium MySQL connector:


{
    "name": "debezium_mysql_1",
    "config": {
        "connector.class": "io.debezium.connector.mysql.MySqlConnector",
        "tasks.max": "1",
        "database.hostname": "xxx.xxx.xxx",
        "database.port": "3306",
        "database.user": "debezium",
        "database.password": "${file:/path/to/credentials:password}",
        "database.server.name": "mysql",
        "heartbeat.interval​.ms": 5000,
        "snapshot.mode": "when_needed",
        "database.include.list": "database",
        "table.include.list": "database.table1,database.table2,database.table3",
        "database.history.kafka.topic": "database_history",
        "database.history​.kafka.recovery​.poll.interval.ms": 5001,
        "database.history.kafka.bootstrap.servers": "xxx.xxx.xxx:9092",
        "database.serverTimezone": "Europe/Warsaw"
    }
}

Two of three tables are loaded to Kafka without any problems:

kafka-topics --list --zookeeper xxx.xxx:2181

mysql.database.table1
mysql.database.table3

but table2 is missing.

I've tried new connector just for table2 - same. Connector is RUNNING, no logs on Cluster and no new topic is created.

There is only one interesting log in mysql:

Aborted connection 8775 to db: 'unconnected' user: 'debezium' host: 'xxx.xxx.xxx' (failed on flush_net())

How table2 can be different from tables 1, 3 that is not replicated to Kafka? Why the connector is RUNNING and there are no logs? What can I do to "force" connector, to replicate table2?

Upvotes: 0

Views: 344

Answers (1)

Gignac
Gignac

Reputation: 210

It turned out, that the missing table does not have PRIMARY KEY:

mysql> SHOW KEYS FROM table2 WHERE Key_name = 'PRIMARY';

Empty set (0.01 sec)

I need to set message.key.columns

Upvotes: 0

Related Questions