CODEWITHSUNDEEP
linuxbashasynchronous
matt
matt

Reputation: 2352

Copy and then move files asynchronously in Bash

On a Linux system I want to loop through a bunch of files on memory sticks, copy a file to the hard drive then move the file on the memory stick to another directory in the same memory stick.

I want to do it asynchronously just so I can do many at the same time but I need to know when each one finished so I can then move them.

This works well synchronously (assumes all memory sticks are already mounted):

for FILE in /home/drive_*;
do  

    cp $FILE $DESTINATION
    mv $FILE "otherDir/"$FILE

done

So this is fine, it does one file at a time and if the files are large it takes quite sometime. How can I do this asynchronously?

I know I can add & (cp $FILE $DESTINATION &) while copying the file but how can I know when it's done so I can then move it?

Upvotes: 2

Views: 2314

Answers (2)

Jon Surrell
Jon Surrell

Reputation: 9647

Just to be clear, you want to copy then move, so that those tasks are synchronous. At the same time as you do those tasks, you want to continue doing other tasks (more copy then moves).

I believe the following is what you seek:

for FILE in /home/drive_*
do
    (cp $FILE $DESTINATION && mv $FILE "otherDir/"$FILE &)
done

You can see that the cp and mv are sequential, but execution continues.

I should note that I doubt you will see any improvement in performance by using this, it may actually get worse. Disk reads and writes are relatively slow, and the disk probably won't be able to read and write several files at once. I would recommend just sticking with a sequential script.

Upvotes: 4

Doomsday
Doomsday

Reputation: 2668

You only have to group your instructions into a bash function, and then put it in background (asynchronous call).

$ function xxx { sleep $1; echo $2; }
$ xxx 5 "long file" &
[1] 10704
$ xxx 2 "short file"&
[2] 10706
$ wait
<here it is waiting for both process to  be completed>
short file
long file
[1]-  Done                    xxx 5 "long file"
[2]+  Done                    xxx 2 "short file"

So, if I use a function with your example, it will be :

function cp_and_move
{
  cp $1 $2
  mv $1 "otherDir/"$1
}    
for i in /home/drive_*; do  
        cp_and_move $FILE $DESTINATION &
done;
wait

Upvotes: 2

Related Questions