CODEWITHSUNDEEP
c#asp.netperformancefull-text-search
user3202862
user3202862

Reputation: 217

Fastest Way of Searching a file within Directories and SubDirectories

I need the fastest way to search for files within directories and subdirectories. The number of files is more than 1.4 Million. File sizes range between 5 and 50 KB. Each folder contains 5 to 10000 files.

At the moment I am using:

foreach (string file in Directory.GetFiles(path, "*.*", SearchOption.AllDirectories))
{
    if (file.Contains(searchkeyword))
    {
        string AnchorText = Path.GetFileNameWithoutExtension(file);
    }
}

Upvotes: 1

Views: 2109

Answers (1)

Jonathan Dickinson
Jonathan Dickinson

Reputation: 9218

Best Solution

I would add an index to that directory and use Windows Search to find files in it. The advantages here are not only that it isn't code that you need to maintain but also that the Windows Indexing Service is able to index the contents of files (.doc, .pdf, .txt, etc.).

Home-Grown Solution

A home-grown cache inside of a database (*Sql, Redis, Mongo, Lucene, whatever you prefer).

  1. Subscribe a FileSystemWatcher to the directory when your application starts.
  2. Then perform an enumeration over the directory once (as per your original code sample) to ensure that no files were added/removed/changed while your application was not running.
  3. Store any data from these two in an appropriately indexed table in a database.

When you need to search for a file simply query the database.

Upvotes: 1

Related Questions