CODEWITHSUNDEEP
unixoptimizationcp
VedhaR
VedhaR

Reputation: 507

Fastest way to copy a large folder in unix

I'm trying to copy a large folder around 1-5gb which contains sub directories with files that range from 1kb-35mbs. I tried using 'cp' but it takes over 10 minutes. Copying and pasting in windows seems to do the job faster.

I'm new to unix and I was wondering if there was a faster alternative or cp, or a way to optimize the way it works. I've read up on buffers but I am so confused as to how they work (so I decided not to use it).

What I am trying to do: I need to transfer files from my H drive to a network drive elsewhere (also delete the files from the network drive so that the new files can be copied to it). It's a really straight forward task which I have working. Just takes about two decades to finish.

P.S. Not sure if I have been told wrong, but working in unix is generally faster than windows right?

Upvotes: 0

Views: 5012

Answers (2)

radiaph
radiaph

Reputation: 4081

You could try using tar or a compressor like zip or gzip to put the whole directory structure into a single archive file, copy the archive file, then unarchive it at the destination. I doubt that would save you time overall, but at least the copying step would be faster.

tar -cvf folder.tar /path/to/original/folder
cp folder.tar /path/to/destination
cd /path/to/destination
tar -xvf folder.tar

-c create
-v verbose
-f file name is... (user put it following)

Upvotes: 2

xpusostomos
xpusostomos

Reputation: 1639

I think tar seems to work quicker because the reading and the writing happen in different processes and don't wait for each other. It also has the benefit that it tends to be smarter at retaining subtleties like links and permissions:

tar cf - path | ( cd /my/new/location ; tar xf - )

Upvotes: 0

Related Questions