How to backup Google Storage buckets?

Question

I need to backup our Google Storage buckets. Versioning is not enough. I was thinking about:

• backup to s3 - is there an automated bucket sync from GS->S3 or out-of-the-box solution for scheduled transfers between buckets?
• backup to another GS bucket - in the same gc project, a coldline bucket "replica" with read-only privs to most users and some automated process to replicate/sync the data?
• any other ideas?

thanks:)

Answer 1

As mentioned in a comment, GCS Transfer is what you are looking for, at least for the part: "backup to another GS bucket".

From the doc:

Transfer data to your Cloud Storage buckets from Amazon Simple Storage Service (S3), HTTP/HTTPS servers, or other buckets. You can schedule one-time or daily transfers, and you can filter files based on name prefix and when they were changed.

Answer 2

You could use gsutil rsync to do this:

gsutil -m rsync -rd gs://your-bucket s3://your-bucket

(similarly for syncing between GCS buckets).

You would need to set up a cron job or something similar to cause this to run periodically.