CODEWITHSUNDEEP
linuxbashrsync
Niksr
Niksr

Reputation: 529

Possible to add current date to rsync working path?

I have an every-hour rsync cron task that is used to add new files to the backup server. Directory structure is the following: /myfiles/year/month/date where year, month and date are actual dates of files. Cron task is defined as the file in the /etc/cron.d

The problem is that I have to indicate a "root" /myfiles directory to make rsync replicate my folder structure in the backup location with every new day. Amount of files is substantial - up to 1000 files a day, so rsync needs to iterate through all yearly files to build a copy list while it's not needed at all because I need to copy today's only files. As of April, it takes ~25 minutes even with --ignore-existing option.

Can someone help me to create a script or whatever to add a current year, month and date to the working rsync path in the cron task, if possible? The final result should look like that:

0 * * * *  root rsync -rt --ignore-existing /myfiles/2020/04/26 user@myserver:/myfiles/2020/04/26

where /2020/04/26 is variable part that is changing every day.

I have very limited experience with *nix systems so I feel that is possible but basically have no clue how to start.

Upvotes: 1

Views: 2211

Answers (1)

Jetchisel
Jetchisel

Reputation: 7791

To add an actual date to the path one can use the date utility or the builtin printf from the bash shell.

Using date

echo "/myfiles/$(date +%Y/%m/%d)"

Using printf

echo "/myfiles/$(printf '%(%Y/%m/%d)T')"

In your case when using the builtin printf you need to define the shell as bash in the cron entry.

0 * * * *  root rsync -rt --ignore-existing "/myfiles/$(printf '\%(\%Y/\%m/\%d)T')" "user@myserver:/myfiles/$(printf '\%(\%Y/\%m/\%d)T')"

Using date either define the PATH to include where the date utility is or just use an absolute path

0 * * * *  root rsync -rt --ignore-existing "/myfiles/$(/bin/date +\%Y/\%m/\%d)" "user@myserver:/myfiles/$(/bin/date +\%Y/\%m/\%d)"

  • The date syntax should work on both GNU and BSD date.

  • The % needs to be escaped inside the cron entry.

  • See the local documentation on your cron(5) on how to add the PATH and SHELL variables. Although the SHELL normally can be SHELL=/bin/bash and PATH to PATH=/sbin:/bin:/usr/sbin:/usr/bin

Upvotes: 1

Related Questions