CODEWITHSUNDEEP
heroku
DonMB
DonMB

Reputation: 2718

Store heroku pg backup on own S3 bucket

Heroku offers automatic and scheduled backups of your PG database. https://devcenter.heroku.com/articles/heroku-postgres-data-safety-and-continuous-protection

GBackups will launch a dedicated dyno to take a dump of your database and upload it to S3

Simple question: Is it possible to upload a scheduled PG backup to one's OWN S3 Bucket? Simply to have control over the backup files and to not be limited in Storage space. Researching this topic did not provide me with an answer if this is possible.

Upvotes: 7

Views: 1820

Answers (3)

pawurb
pawurb

Reputation: 526

You can do it by using Heroku scheduler and a bash script. 

# Set the script to fail fast if there
# is an error or a missing variable

set -eu
set -o pipefail

#!/bin/sh

# Download the latest backup from
# Heroku and gzip it

heroku pg:backups:download --output=/tmp/pg_backup.dump --app $APP_NAME
gzip /tmp/pg_backup.dump

# Encrypt the gzipped backup file
# using GPG passphrase

gpg --yes --batch --passphrase=$PG_BACKUP_PASSWORD -c /tmp/pg_backup.dump.gz

# Remove the plaintext backup file

rm /tmp/pg_backup.dump.gz

# Generate backup filename based
# on the current date

BACKUP_FILE_NAME="heroku-backup-$(date '+%Y-%m-%d_%H.%M').gpg"

# Upload the file to S3 using
# AWS CLI

aws s3 cp /tmp/pg_backup.dump.gz.gpg "s3://${S3_BUCKET_NAME}/${BACKUP_FILE_NAME}"

# Remove the encrypted backup file

rm /tmp/pg_backup.dump.gz.gpg

You can check out this tutorial for detailed step by step explanation.

Upvotes: 3

Matt Sanders
Matt Sanders

Reputation: 10705

Here is a buildpack for doing this on a regular schedule. It hasn't been updated in a bit, but you could easily update / adapt it as needed.

Upvotes: 0

Du3
Du3

Reputation: 1422

One option is to create a backup (you can even create a follower database to created it from for performance reasons), then download the backup via stream to your server, and then upload it into your own S3 bucket.

If you wanted a quick Rail app to do this, you can setup https://github.com/kjohnston/pgbackups-archive. It does everything aside from creating a follower database, but if you are not too concerned with performance 24/7, then this should do fine. I don't know why Heroku doesn't offer storage to your own S3 buckets, as they store them on S3 themselves.

Upvotes: 0

Related Questions