CODEWITHSUNDEEP
mysqlbashmariadb
David W
David W

Reputation: 565

Removing old MySQL / MariaDB backups in Bash Backup Script

I've written a bash script, initiated on cron, that backups all databases on a particular machine nightly and weekly. The script correctly removes old databases, except for those cases when there's been a change in month.

As an example, let's say is November 2nd. The script runs at 11:00pm, and correctly removes the backup made from November 1st. But come December 1st, the script gets confused, and does not correctly remove the backup made from November 30th.

How can I fix this script to correctly remove the old backups in this case?

DATABASES=$(echo 'show databases;' | mysql -u backup --password='(password)' | grep -v ^Database$)
LIST=$(echo $DATABASES | sed -e "s/\s/\n/g")
DATE=$(date +%Y%m%d)
DAYOLD=$(($DATE-1))
SUNDAY=$(date +%a)
WEEKOLD=$(($DATE-7))

for i in $LIST; do
if [[ $i != "mysql" ]]; then
        mysqldump --single-transaction $i > /mnt/backups/mariadb/daily/$i.$DATE.sql
        if [ -f /mnt/backups/mariadb/daily/$i.$DAYOLD.sql ]; then
                rm -f /mnt/backups/mariadb/daily/$i.$DAYOLD.sql
        fi
        if [[ $SUNDAY == "Sun" ]]; then
                cp /mnt/backups/mariadb/daily/$i.$DATE.sql /mnt/backups/mariadb/weekly/$i.$DATE.sql
                rm -f /mnt/backups/mariadb/weekly/$i.$WEEKOLD.sql
        fi
fi
done

Upvotes: 0

Views: 1450

Answers (2)

David W
David W

Reputation: 565

I was making this too complicated. Instead of using the date at all, I'm just searching for the age of the file backup with:

find /mnt/backups/mariadb/weekly/* -type f -mtime +8 -exec rm -f {} \;

So the entire script becomes:

DATABASES=$(echo 'show databases;' | mysql -u backup --password='foo' | grep -v ^Database$)
LIST=$(echo $DATABASES | sed -e "s/\s/\n/g")
DATE=$(date +%Y%m%d)
SUNDAY=$(date +%a)

for i in $LIST; do
if [[ $i != "mysql" ]]; then
        /bin/nice mysqldump --single-transaction $i > /mnt/backups/mariadb/daily/$i.$DATE.sql
        find /mnt/backups/mariadb/daily/* -type f -mtime +1 -exec rm -f {} \;

        if [[ $SUNDAY == "Sun" ]]; then
                cp /mnt/backups/mariadb/daily/$i.$DATE.sql /mnt/backups/mariadb/weekly/$i.$DATE.sql
                find /mnt/backups/mariadb/weekly/* -type f -mtime +8 -exec rm -f {} \;
        fi
fi
chown -R backup.backup /mnt/backups
done

Upvotes: 0

danius
danius

Reputation: 2764

If you know the number of backups performed in a specific range of time, let's say you know from 2nd Nov until 2nd Dec you know that exactly 30 backups have been made and you now want to erase those, just use the number of backups, it's super simple to do and you don't have to deal with dates which is pretty complex in bash:

$ (ls -t|head -n 30;ls)|grep -v ^Database|sort|uniq -u|xargs rm -rf

You can then easily automate this script by removing each day the older one so you only get the fix number of backups you want:

#! /bin/bash
# Create new full backup
BACKUP_DIR="/path-to-backups/"
BACKUP_DAYS=1

# Prepare backup
cd ${BACKUP_DIR}
latest=`ls -rt | grep 201 | head -1`

# Change latest reference
ln -sf ${BACKUP_DIR}${latest} latest

# Cleanup older than one week (n days)
to_remove=`(ls -t | grep 201 | head -n 3;ls)|sort|uniq -u`
echo "Cleaning up... $to_remove"
(ls -t|head -n ${BACKUP_DAYS};ls)|sort|uniq -u|xargs rm -rf

echo "Backup Finished"
exit 0

Then you can link it to daily cron. This is explained in this blog entry, how to do this stuff in a very straightforward fashion (but with hot backups, no mysqldump): http://codeispoetry.me/index.php/mariadb-daily-hot-backups-with-xtrabackup/

Upvotes: 2

Related Questions