CODEWITHSUNDEEP
c#asp.netc#-4.0file-iowebclient
adam
adam

Reputation: 3578

Downloading and reading the same file by many processes w/ C# 4.0

I need to be able to read a file which should be saved locally, and if the file doesn't exist yet I need to download it. Here's a simplified version of what I'm trying to accomplish:

// See if the file exists locally
if (!File.Exists(physicalPath))
{
    // file doesn't exist, download it
    using (WebClient client = new WebClient())
    {
        client.DownloadFile(url, physicalPath);
    }
}

// open up the file and do stuff ..

Because of what happens later on I cannot use Streams, but I need the file saved to the disk first.

Everything works great for one call, but as you can imagine there will be issues when 1,000 processes run through this function at the same time. Here's what I think will happen when two processes kick off at the same time:

Process 1:

  1. Does the file exist locally? No
  2. Start downloading the file
  3. Downloading the file completes
  4. Read the file

Process 2:

  1. Does the file locally exist? Yes
  2. Read the file
  3. EXCEPTION!!! File being used by another process

So process 2 will see that the file exists, but only because process 1 is actively downloading the file already, but process 1 hasn't finished downloading the file yet. What I'd like process 2 (and all the other processes going on), is to wait for process 1 to finish downloading the file.

What helps us out here is that whatever process 1 downloads will work for process 2, meaning process 2 wouldn't have different data and it doesn't need to also write to the file. If process 1 is downloading a file, it'd be preferred that process 2 does not have to download the file, and it can wait for process 1 to finish.

So to sum up, if process 1 is already downloading the file needed by other processes, how can I hold up other processes from continuing so they wait for process 1 to finish? How could I make process 2 continue once the file is ready? (using C# 4.0)

I should also clarify, speed of this operation is priority #1.

Upvotes: 1

Views: 401

Answers (2)

Mecaveli
Mecaveli

Reputation: 1565

Just use a Mutex to synchronise the Processes. Something like this should do the trick:

public static class FileRepository
{
    public static FileInfo GetFile(string fileName)
    {
        FileInfo MyFile;

        if(!File.Exists(fileName))
        {
            // Use some unique name + filename as Mutex Name
            using(new Mutex(true, fileName))
            {
                // Will block if another Process already downloads the file
                if(Mutex.WaitOne())
                {
                    // Download the File and safe FileInfo in MyFile
                    // ...
                }
            }
        }
        else
        {
            MyFile = new FileInfo(fileName);
        }       

        return MyFile;
    }
}

Upvotes: 0

spender
spender

Reputation: 120518

The whole setup sounds somewhat prone to race conditions that would be tricky to moderate.

I'd make a separate (singleton) process that's responsible for queueing requests and downloading them for the other processes. You could talk to it via some sort of IPC (like WCF).

Upvotes: 2

Related Questions