CODEWITHSUNDEEP
.netperformancefile-io
Richard Ev
Richard Ev

Reputation: 54117

DirectoryInfo.GetFiles slow when using SearchOption.AllDirectories

I am searching a moderate number (~500) of folders for a large number (~200,000) of files from a .NET application.

I hoped to use DirectoryInfo.GetFiles, passing in SearchOption.AllDirectories. However this approach seems to be a lot slower than writing my own code to iterate through the directories and do GetFiles just passing in a searchPattern.

Related MSDN info:

Has anyone had a similar experience to this?

Upvotes: 9

Views: 5867

Answers (2)

Mike Marynowski
Mike Marynowski

Reputation: 3429

The approach Anton mentioned using FindFirstFile() and related native methods has been implemented as of .NET 4 via DirectoryInfo.EnumerateFiles() so no more need for P/Invoke for this!

Upvotes: 3

Anton Gogolev
Anton Gogolev

Reputation: 115751

These two functions are actually infamous for their performance. The reason is that GetFiles walks entire directory tree and constructs an array of FileInfo objects, and only then returns the result to the caller. Construction of said array involves a lot of memory allocations (I'm sure they use List internally, but still) since the number of entries cannot be known ahead of time.

If you're really into performance, you can P/Invoke into FindFirstFile/FindNextFile/FindClose, abstract them into an IEnumerable<FileInfo> and yield FileInfos one at a time.

Upvotes: 14

Related Questions