Speed up powershell move-item command

Question

I've been using the basic "Move-Item -path G:\folder* -destination K:\folder"

However, I have about 20,000 folders, and about 15 million files. This has been 2 days and so far only a few thousand folders (and the content inside) has been moved. The reason I used this command over the typical move feature in Windows Explorer is that I have files with filenames that are VERY long. I've tried expanding the file name limit in Windows, but it still hits a limit when 'moving' using Windows Explorer. It only seems to work (or appears to work without error) with powershell.

Is there any way to speed this up?

Answer 1

Use robocopy with it's /MT:x parameter. /MT is for multi-threading. Also, it doesn't suffer the character limits you find in the Windows interface. In fact you'll find many robocopy examples on the web just to find long file names. You don't have to abandon the work you've already done. Robocopy is incremental it won't recopy a file that hasn't changed. A command might look something like:

Robocopy.exe <SourceFolder> <DestinationFolder> /E /MT:20 /Log:c:\temp\Robolog.log

Or your example:

Robocopy.exe "G:\folder" "K:\folder" /E /MT:20

Note/Update: Changed from /MIR to /E because some files would have already been moved due to prior work.

This command will effectively run 20 Robocopy.exe processes cooperating to copy the data.

Robocopy has many parameters, so the above example is my guess to what you need. /E means Copy all subfolders including empty ones and will copy all files folders below the source director. Using the /Log or /Log+ will send the output to a file instead of the screen. In my experience saving the program from screen echoing speeds things up. That said, I have had mixed results in the log file itself when using /MT:x. However, the mission here is to get the data over much more quickly...

Incidentally /TEE would allow you to send output to both the screen and log.

Also since it's incremental you can rerun afterward perhaps with different logging and threading options to confirm everything copied. If you elect not to use /MOVE ... You can even craft a command to purge the source files and folders when done.

RoboCopy.exe <EmptyDirectory> <OriginalSourceDirectory> /MIR

Always make sure you test with the /L parameter before executing. /L is list mode so it will show or log what will happen without copying or deleting anything.

Use Robocopy.exe /? for the full documentation.

Answer 2

Try Start-BitsTransfer, here is the documentation.