Reputation: 784
I'm working with CDC using a debezium connector managed by Confluent.
I was trying to use the topics creation group feature offered by debezium, to create topics automatically based on the size of each table.
This is my config:
{
"name": "sample_connector_name",
"topic.prefix": "sample_connector_prefix",
"database.dbname": "servicedb",
"snapshot.mode": "schema_only",
"snapshot.locking.mode": "none",
"signal.enabled.channels": "source",
"signal.data.collection": "servicedb.debezium_signal",
"incremental.snapshot.allow.schema.changes": "true",
"incremental.snapshot.chunk.size": 1024,
"tombstones.on.delete": "true",
"database.ssl.mode": "preferred",
"poll.interval.ms": "1000",
"max.batch.size": "1000",
"output.data.format": "AVRO",
"topic.creation.enable": "true",
"topic.creation.default.partitions": 1,
"topic.creation.default.replication.factor": -1,
"topic.creation.groups": "small,medium",
"topic.creation.small.retention.ms": 432000000,
"topic.creation.small.include": "sample_connector_prefix.servicedb.users",
"topic.creation.medium.retention.ms": 86400000,
"topic.creation.medium.include": "sample_connector_prefix.servicedb.orders,sample_connector_prefix.servicedb.cities",
"tasks.max": "1",
"status": "RUNNING"
}
I was expecting that Debezium creates the following topics:
small
, a topic sample_connector_prefix.servicedb.users
with a retention time of 432000000 ms
. (5 days)medium
, the topics ample_connector_prefix.servicedb.orders
and sample_connector_prefix.servicedb.cities
with a retention time of 86400000 ms
(1 day)But Debezium is ignoring all the topics under the small
and medium
groups, and it's just creating the topics that are not listed in those groups.
Do you have any hints about the potential problem?
Upvotes: 1
Views: 238