CODEWITHSUNDEEP
c#performancecopy
user1158903
user1158903

Reputation:

performance of copying directories

I am using C# to copy files from one directory to another directory. I am using code from msdn but its pretty slow taking a minute or so to copy a couple of gigs. It only takes seconds in explorer. http://channel9.msdn.com/Forums/TechOff/257490-How-Copy-directories-in-C Surly there a faster way..:)

        private static void Copy(string sourceDirectory, string targetDirectory)
    {
        DirectoryInfo diSource = new DirectoryInfo(sourceDirectory);
        DirectoryInfo diTarget = new DirectoryInfo(targetDirectory);

        CopyAll(diSource, diTarget);
    }

    private static void CopyAll(DirectoryInfo source, DirectoryInfo target)
    {
        // Check if the target directory exists, if not, create it.
        if (Directory.Exists(target.FullName) == false)
        {
            Directory.CreateDirectory(target.FullName);
        }

        // Copy each file into it's new directory.
        foreach (FileInfo fi in source.GetFiles())
        {
            fi.CopyTo(Path.Combine(target.ToString(), fi.Name), true);
        }

        // Copy each subdirectory using recursion.
        foreach (DirectoryInfo diSourceSubDir in source.GetDirectories())
        {
            DirectoryInfo nextTargetSubDir =
                target.CreateSubdirectory(diSourceSubDir.Name);
            CopyAll(diSourceSubDir, nextTargetSubDir);
        }
    }

Using Parallel I was able to copy 6gigs in under a minute faster than explorer and xcopy. 

private static void CopyAll(string SourcePath, string DestinationPath)
{
    string[] directories = System.IO.Directory.GetDirectories(SourcePath, "*.*", SearchOption.AllDirectories);

    Parallel.ForEach(directories, dirPath =>
    {
        Directory.CreateDirectory(dirPath.Replace(SourcePath, DestinationPath));
    }); 

    string[] files = System.IO.Directory.GetFiles(SourcePath, "*.*", SearchOption.AllDirectories);

    Parallel.ForEach(files, newPath =>
    {
        File.Copy(newPath, newPath.Replace(SourcePath, DestinationPath));
    }); 
}

Upvotes: 0

Views: 4233

Answers (4)

TGarrett
TGarrett

Reputation: 552

There is a big problem with using Parallel.ForEach loops with creating directories, the first thing you need to be aware of, are sub directories nested within other directories, if Parallel creates directories out of order, the code can be thrown due to trying to create directory level 8 while directory level 6 was not even made yet.

Upvotes: 0

Nikhil Agrawal
Nikhil Agrawal

Reputation: 48558

What u are using is recursion. It always slows the system.

Use this as it has no recursion.

void CopyAll (string SourcePath, string DestinationPath)
{
    //Now Create all of the directories
    foreach (string dirPath in Directory.GetDirectories(SourcePath, "*.*", 
    SearchOption.AllDirectories))
    Directory.CreateDirectory(dirPath.Replace(SourcePath, DestinationPath));

    //Copy all the files
    foreach (string newPath in Directory.GetFiles(SourcePath, "*.*", 
    SearchOption.AllDirectories))
    File.Copy(newPath, newPath.Replace(SourcePath, DestinationPath));
}

Upvotes: 5

Jerry Coffin
Jerry Coffin

Reputation: 490108

I suspect @Oded is correct, and you're comparing a copy to a move.

If you want to ensure that what you're doing is the same as the shell does, you could look into SHFileOperation, or (on Vista or newer) IFileOperation. At least as far as I know, you'd have to use SHFileOperation via P/Invoke. IFileOperation is a COM interface.

Upvotes: 0

Tigran
Tigran

Reputation: 62248

Your issue is probabbly on *overwriteé of the file. I see that you call 

fi.CopyTo(Path.Combine(target.ToString(), fi.Name), true);

where true means overwrite the file if it exists.

Based on this I suppose so that it's a case in your app to overwrite already existing files.

The stuff should go faster if you

  • first rename destination directory with some temporary unique name
  • after just copy source to dest (so no overwrite)

if success remove temp directory, if fails remove all copied files and rename temp to its original name.

Should be faster.

Good luck.

Upvotes: 0

Related Questions