CODEWITHSUNDEEP
postgresqlamazon-web-servicesamazon-rdsdebezium
shrawan_lakhe
shrawan_lakhe

Reputation: 356

AWS Postgres RDS logical replication for Change Data Capture using Debezium

I am trying to implement logical replication with AWS RDS Postgres sql for Change Data Capture using Debezium. Mostly the implementation is working fine and i see the changes being streamed to the consumer services. But, I have been facing couple of other issues.

In the RDS database, in some of the tables, composite keys are being used and when creating publication in postgres, if true is set to update, delete actions, then i cannot delete/update records in those tables as it wants to set REPLICA IDENTITY FULL in those tables.

Without creating publications, is there any way to make the logical replication work? Based on my research till now, i don't think that is possible for my scenario but I may be wrong as well. And setting REPLICA IDENTITY FULL in all the tables where composite keys are used is not preferable as it will impact on performance. Can you suggest on this what might be the optimized way to handle this?

Also, another issue that i am facing is after creating replication slot in the db, i am having issues with the db size. Db size grows uncontrollably and when i delete the replication slot, db size comes to normal. Is there any way that this can be solved?

Thank You

Upvotes: 1

Views: 836

Answers (1)

Frank Lee
Frank Lee

Reputation: 2748

Not sure about the REPLICA IDENITY FULL, but AFAIK the database size issue seems RDS specific. It happens when you have no / very few changes. It can only clean up the WAL after there is an actual change. To mitigate that, you can set up a heartbeat.

Check the WAL Disk Space Consumption part here: https://debezium.io/documentation/reference/stable/connectors/postgresql.html

Upvotes: 1

Related Questions