CODEWITHSUNDEEP
google-cloud-platformgoogle-cloud-pubsubgoogle-cloud-spanner
codergirl
codergirl

Reputation: 1

What is the best way to read data from Spanner to PubSub Queue?

I need something like a listener which listens to changes in the database. As soon as some change occurs, it inserts the updated row in the pubSub queue. What is the best way to do this?

Upvotes: 0

Views: 1340

Answers (1)

Knut Olav Løite
Knut Olav Løite

Reputation: 3512

It's still in alpha stage, but you could have a look at this: https://github.com/cloudspannerecosystem/spanner-change-watcher

It's a generic framework for watching a Spanner database for changes by polling tables for new commit timestamps. It does require the tables that you want to watch for changes to contain a column with the option ALLOW_COMMIT_TIMESTAMP=TRUE and for this column to be filled.

It contains three main modules:

  1. google-cloud-spanner-watcher: This a generic framework that watches a Spanner database for changes.
  2. google-cloud-spanner-publisher: This uses the framework from the point above and publishes all changes to one or more Pubsub topics.
  3. google-cloud-spanner-archiver: This is an example application for consuming the messages from the Publisher which archives all changes in Google Cloud Storage.

Upvotes: 1

Related Questions