CODEWITHSUNDEEP
mongodbapache-kafkatime-seriesdebezium
Hanieh
Hanieh

Reputation: 23

I have problems converting debezium topic to mongodb timeseries

I have a PostgreSQL partitioned table -partitioned by timestamp field- which I want to replicate in MongoDB as a timeseries collecion. I have a source debezium connecor which reads all partitions of PostgreSQL table in one single kafka topic through debezium's reroute transform. I Have a sink connector which writes the mentioned topic in mongoDB. I have set Kafka Time Series Properties in connector configs to make the collection as a timeseries one, but it fails.

I think it is because of this: in Time Series Collection Limitations it is mentioned that update command must not set upsert: true and debezium sets upsert: true. does anyone have a suggestion to solve this problem? thank you for your time.

Upvotes: 1

Views: 93

Answers (1)

Hanieh
Hanieh

Reputation: 23

fortunately I managed to solve my problem. the main problem was that I was using "at.grahsl.kafka.connect.mongodb.MongoDbSinkConnector" as my connector class. when I changed it to "com.mongodb.kafka.connect.MongoSinkConnector", I could get a timeseries collection in mongodb using timeseries.timefield key (I used the docker image "confluentinc/cp-kafka-connect:5.3.0" to build my connect service which provides the mentioned connector class).

Upvotes: 0

Related Questions