CODEWITHSUNDEEP
regexlinuxrsync
D0vev
D0vev

Reputation: 159

Using rsync with RegEx

I am using rsync to sync folders and their content between a Linux server and a network storage to backup files. For this, I am using this line of code:

rsync -rltPuz -k --chmod=ugo+rwx --prune-empty-dirs --exclude=*backup* --exclude=*.zip --exclude=*.zip.bak --password-file=/rsync_pw.txt /source/ user@storage::Kunden/Jobs

This Code is running on the source via crontab. Everything works fine.

But now I have a little problem. My directories are built like this:

I need only to sync all ready folders and their content. I have tried around with --include and --exclude but I did not really got what I needed. Is there a way to tell rsync what I want?

Thanks for your time!

Upvotes: 1

Views: 4271

Answers (2)

diskerror
diskerror

Reputation: 21

Eight years later I find this post after days of pounding on globbing and escaping issues for command option parameters. This was doubly important as my IDE was applying "exclude" options for rsync without quotes or escaping.

CompSci 101: Glob characters ? * [ ] are expanded by the shell before the command is executed. And, they are expanded based on the current working directory. (Yeah, I forget all the places that this applies, too.) This is why it might seem to work in situations.

This includes your option to rsync, --exclude=*.zip. Those parameters need to be either escaped or quoted. So, omitting other options for brevity:

rsync -av --exclude='*backup*' --exclude='*.zip' --exclude='*.zip.bak' /source/ user@storage::Kunden/Jobs

or

rsync -av --exclude=\*backup\* --exclude=\*.zip --exclude=\*.zip.bak /source/ user@storage::Kunden/Jobs

If you are unsure of what the results of an include, exclude, or filter combination is and what is being sent to, say, a production server, you can test your command with the options --dry-run or -n and --debug=filter. You'll get a list of files that are shown or hidden from the planned transfer.

Upvotes: 1

kammala
kammala

Reputation: 427

You can use find /path/to/Jobs -name ready and pipe its output to rsync or use find option -exec and place you rsync call there.

In your example the final command will look like:

find Jobs/ -name 'ready' -exec rsync -rltPuz -k --chmod=ugo+rwx --prune-empty-dirs --exclude=*backup* --exclude=*.zip --exclude=*.zip.bak {}/ dest \;

On my ubuntu it works:

kammala@devuntu:~$ ls -R dest/
dest/:
kammala@devuntu:~$ ls -R Jobs/
Jobs/:
Job1  Job2

Jobs/Job1:
new  ready

Jobs/Job1/new:
new1.txt  new2.txt  some_new_backup.txt

Jobs/Job1/ready:
r1.txt  r2.txt  some_backup_file.txt

Jobs/Job2:
new  ready

Jobs/Job2/new:
new3.txt  new4.txt  zipped_bckp.zip.bak

Jobs/Job2/ready:
r4.txt  r5.txt  r6.txt  some_zipped_file.zip.bak
kammala@devuntu:~$ find Jobs/ -name 'ready' -exec rsync -rltPuz -k --chmod=ugo+rwx --prune-empty-dirs --exclude=*backup* --exclude=*.zip --exclude=*.zip.bak {}/ dest \;
building file list ... 
3 files to consider
./
r1.txt
              0 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=1/3)
r2.txt
              0 100%    0.00kB/s    0:00:00 (xfr#2, to-chk=0/3)
building file list ... 
4 files to consider
./
r4.txt
              0 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=2/4)
r5.txt
              0 100%    0.00kB/s    0:00:00 (xfr#2, to-chk=1/4)
r6.txt
              0 100%    0.00kB/s    0:00:00 (xfr#3, to-chk=0/4)
kammala@devuntu:~$ ls -R dest
dest:
r1.txt  r2.txt  r4.txt  r5.txt  r6.txt

Upvotes: 1

Related Questions