CODEWITHSUNDEEP
pythonmysql
codie-fz
codie-fz

Reputation: 126

How to catch new inserts in a MySQL Table in Python

I would like to catch new inserts committed to a MySQL Table and process these records in Python.

For example: In an "Order" Table a new record is inserted each time a customer orders something. I would like to catch this change made to the Table in real time and process it in Python in order to for example write this record to a csv file.

What would be the best approach to do this? I was thinking about running a script every second, track the overall count of the table and whenever the count goes up I pull the latest x rows. But maybe there is an easier, more reliable way?

Cheers!

Upvotes: 0

Views: 95

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191728

The easier way is to enable/read binlogs on the database.

Since you have , then you could use Debezium to load CDC records into a Kafka topic, which you can then use any Kafka client (including Python, not necessarily Spark) to consume.

Upvotes: 1

Related Questions