Reputation: 11
I'm trying to parallel copy one large dataset (~7TB) to three external USB drives at the same time. Ideally I want to be able to kick it off, leave it to crack on and come back to check on it every so often.
I understand that it may not be the quickest way to do it, but I really don't want to have to go back to swap drives over and take 2 days to do it.
Here's where I've got to:
parallel --bar rsync ./source/ -r --info=progress2 ::: /dest/one/ /dest/two/
Which works, but I cannot for the life of me figure out how to get a good readout. There are plenty of parallel copying of one dataset for speed, but my limitation is the USB HDD write speed, which is about 200MBps.
I'm absolutely open to suggestions that throw the current idea in the bin.
EDIT
The command I ended up with thanks to Renaud is:
parallel -u rsync ./source/ -r --info=progress2{2} {1} ::: /dest/one/ /dest/two/
Which gives me a progress report of one of the threads which isn't perfect for all applications, but is great for this one!
Upvotes: 1
Views: 243
Reputation: 33685
Which works, but I cannot for the life of me figure out how to get a good readout.
Is progress information of each rsync
what you are asking for?
In that case try:
parallel --ll --tag rsync ./source/ -r --info=progress2 ::: /dest/one/ /dest/two/
Requires version 20220522 or later.
Upvotes: 0