ljuk
ljuk

Reputation: 784

Debezium automatic topic creation not working as expected

I'm working with CDC using a debezium connector managed by Confluent.

I was trying to use the topics creation group feature offered by debezium, to create topics automatically based on the size of each table.

This is my config:

{
    "name": "sample_connector_name",
    "topic.prefix": "sample_connector_prefix",
    "database.dbname": "servicedb",
    "snapshot.mode": "schema_only",
    "snapshot.locking.mode": "none",
    "signal.enabled.channels": "source",
    "signal.data.collection": "servicedb.debezium_signal",
    "incremental.snapshot.allow.schema.changes": "true",
    "incremental.snapshot.chunk.size": 1024,
    "tombstones.on.delete": "true",
    "database.ssl.mode": "preferred",
    "poll.interval.ms": "1000",
    "max.batch.size": "1000",
    "output.data.format": "AVRO",
    "topic.creation.enable": "true",
    "topic.creation.default.partitions": 1,
    "topic.creation.default.replication.factor": -1,
    "topic.creation.groups": "small,medium",
    "topic.creation.small.retention.ms": 432000000,
    "topic.creation.small.include": "sample_connector_prefix.servicedb.users",
    "topic.creation.medium.retention.ms": 86400000,
    "topic.creation.medium.include": "sample_connector_prefix.servicedb.orders,sample_connector_prefix.servicedb.cities",
    "tasks.max": "1",
    "status": "RUNNING"
}

I was expecting that Debezium creates the following topics:

But Debezium is ignoring all the topics under the small and medium groups, and it's just creating the topics that are not listed in those groups.

Do you have any hints about the potential problem?

Upvotes: 1

Views: 238

Answers (0)

Related Questions