Can I use modified date and time as suffix in rsync?

Question

I'm using rsync to synchronize folders, and make backups of existing files if they have been modified. Currently all modified files are backed to a separate directory, with the synchronization time as suffix. This is with the following command:

rsync --times --backup --backup-dir=OldVersions --suffix=`date +'.%y%m%d%H%M'` /SourceDir /DestDir

Now what I would like to do is use the modified date and time of each file that has to be backed up, instead of the time of the synchronization. Any ideas how I would be able to achieve this?

Answer 1

The approach below may work - it was tested on a Linux system. Just run another script against the OldVersion diretories, after adding a suffix _ZZZZ to the end of the file name so that find can select it. Rename the file ending using the modify timestamp of the file.

Script to rename the files using their modification timestamp. rename.sh

#!/usr/bin/env bash
name=$1
# get file modification time, substitute space with underscore, and remove -,:
modtime=`stat $1 | grep Modify | cut -d ' ' -f 2,3 | sed -e 's/\ /_/g' -e 's/[-:]//g' `
#echo $modtime
newname=`echo $1 | sed -e "s/[[:digit:]]\{10\}_ZZZZ$/$modtime/g"`
#echo $newname 
mv $1 $newname

Added the rsync options -r to recurse into directories and -i to show some information.

user1@debian10 /home/user1/test > rsync --backup-dir=OldVersions --suffix=`date +'.%y%m%d%H%M_ZZZZ'` -btri src dest
cd+++++++++ src/
>f+++++++++ src/azvltfexishlm.txt
>f+++++++++ src/dhatkfztklgcan.txt
>f+++++++++ src/feftafvfrdepwezl.txt
>f+++++++++ src/fwclodehxlpg.txt
>f+++++++++ src/ijcjftigjqhxhan.txt
>f+++++++++ src/jdlfsxoinuey.txt
>f+++++++++ src/oljmsfjv.txt
>f+++++++++ src/rbrktqqrtjyxyt.txt
>f+++++++++ src/rqheczjqrjulvlia.txt
>f+++++++++ src/ruyeizqrxstu.txt
>f+++++++++ src/ssuwndmrellunqyq.txt
>f+++++++++ src/vaclfgwqfdihmvis.txt

Ran again after I appended to the files below.

user1@debian10 /home/user1/test > rsync --backup-dir=OldVersions --suffix=`date +'.%y%m%d%H%M_ZZZZ'` -btri src dest
>f.st...... src/azvltfexishlm.txt
>f.st...... src/fwclodehxlpg.txt
>f.st...... src/ijcjftigjqhxhan.txt
>f.st...... src/ruyeizqrxstu.txt

Ran again after I appended to the files below

user1@debian10 /home/user1/test > rsync --backup-dir=OldVersions --suffix=`date +'.%y%m%d%H%M_ZZZZ'` -btri src dest
>f.st...... src/fwclodehxlpg.txt
>f.st...... src/rbrktqqrtjyxyt.txt
>f.st...... src/rqheczjqrjulvlia.txt

Using find to call the script on the backup files:

user1@debian10 /home/user1/test/dest > find . -name "*_ZZZZ" -print -exec ~/bin/rename.sh {} \;
./OldVersions/src/rqheczjqrjulvlia.txt.2110231948_ZZZZ
./OldVersions/src/azvltfexishlm.txt.2110231945_ZZZZ
./OldVersions/src/rbrktqqrtjyxyt.txt.2110231948_ZZZZ
./OldVersions/src/fwclodehxlpg.txt.2110231948_ZZZZ
./OldVersions/src/ruyeizqrxstu.txt.2110231945_ZZZZ
./OldVersions/src/ijcjftigjqhxhan.txt.2110231945_ZZZZ
./OldVersions/src/fwclodehxlpg.txt.2110231945_ZZZZ
./OldVersions/src/ssuwndmrellunqyq.txt.2110231948_ZZZZ

Listing the renamed files.

user1@debian10 /home/user1/test/dest > ls -lR OldVersions/
OldVersions/:
total 4
drwxr-xr-x 2 user1 user1 4096 Oct 23 20:24 src

OldVersions/src:
total 32
-rw-r--r-- 1 user1 user1 1322 Oct 23 19:42 azvltfexishlm.txt.20211023_194252.673598165
-rw-r--r-- 1 user1 user1 2255 Oct 23 19:42 fwclodehxlpg.txt.20211023_194252.673598165
-rw-r--r-- 1 user1 user1 3084 Oct 23 19:45 fwclodehxlpg.txt.20211023_194506.313540547
-rw-r--r-- 1 user1 user1 1178 Oct 23 19:42 ijcjftigjqhxhan.txt.20211023_194252.673598165
-rw-r--r-- 1 user1 user1  485 Oct 23 19:42 rbrktqqrtjyxyt.txt.20211023_194252.673598165
-rw-r--r-- 1 user1 user1 2283 Oct 23 19:42 rqheczjqrjulvlia.txt.20211023_194252.673598165
-rw-r--r-- 1 user1 user1 1579 Oct 23 19:42 ruyeizqrxstu.txt.20211023_194252.673598165
-rw-r--r-- 1 user1 user1 2091 Oct 23 19:42 ssuwndmrellunqyq.txt.20211023_194252.673598165

After each backup script run, then run the rename script. It should be idempotent.

Test scipt for file creation:

#!/usr/bin/env ruby
class Foo
  def initialize()
  end
  def random_string(len)
    pattern = "abcdefghijklmnopqrstuvwxyz"
    accum = ""
    if len == 0
      return ""
    end
    (0..len-1).each do | i |
      index = rand(0..25)
      accum << pattern[index]
    end
    return accum
  end
  def append_data_to_file(fname)
    fout = File.open(fname, "a")
    num_lines = rand(5..30)
    (1..num_lines).each do | lineno |
      rand_string = random_string(rand(24..70))
      fout.puts(rand_string)
    end
    fout.close
  end
  def create_rand_files(path)
    Dir.chdir(path) do 
      num_files = rand(10..20)
      puts "file count #{num_files}"
      (1..num_files).each do | i |
        name_len = rand(8..16)
        rand_name = random_string(name_len) + ".txt"
        puts "file #{i} name #{rand_name}"
        append_data_to_file(rand_name)
      end
    end
  end
  def modify_files(path)
    arr = Dir.entries(path).select { |x| x != "." && x != ".."}.shuffle
    subsize = arr.length / 3
    (1..subsize).each do
      fname = arr.pop
      puts fname
      append_data_to_file(fname)
    end
  end
end
def main
  begin
    if ARGV.length != 2
      puts "use: #{$0} path cmd"
      exit
    end
    path,cmd = ARGV
    b = Foo.new
    case cmd
    when "create"
      b.create_rand_files(path)
    when "modify"
      b.modify_files(path)
    else
      puts "unknown command #{cmd}"
    end
  rescue StandardError => e
    p e
    p e.backtrace.inspect
  end
end
main