FileSystemWatcher cannot access files because used in other process

Question

I'm writing a FileSystemWatcher which is to copy images from folder A to folder B, whenever an image is uploaded to folder A. I'm trying to use this as a windows service on the server PC but I'm having some issues where my files are locked when they are to be copied. I think I've found the root to my issue, but I'm not having any luck solving it. So, when I run my windows service it always ends unexpectedly at either the first or the second picture upload. The error message I'm getting says this: The process cannot access the file 'filepath' because it is being used by another process.

Relevant parts of my code:

public void WatchForChanges()
{
    FileSystemWatcher watcher = new FileSystemWatcher();
    watcher.Path = Program.SourceFolder;
    watcher.Created += new FileSystemEventHandler(OnImageAdded);
    watcher.EnableRaisingEvents = true;
    watcher.IncludeSubdirectories = true;
}

public void OnImageAdded(object source, FileSystemEventArgs e)
{
    FileInfo file = new FileInfo(e.FullPath);
    ImageHandler handler = new ImageHandler();
    if (handler.IsImage(file))
    {
        handler.CopyImage(file);
    }
}

and, my CopyImage method, which includes one of my proposed solutions to this problem, utilizing a while loop that catches the error and retries the copying of the image:

public void CopyImage(FileSystemInfo file)
{
    // code that sets folder paths
    // code that sets folder paths

    bool retry = true;
    if (!Directory.Exists(targetFolderPath))
    {
        Directory.CreateDirectory(targetFolderPath);
    }
    while (retry)
    {
        try
        {
            File.Copy(file.FullName, targetPath, true);
            retry = false;
        }
        catch (Exception e)
        {
            Thread.Sleep(2000);
        }
    }
}

but this CopyImage solution just keeps on copying the same file, which is not very ideal in my case. I wish it was enough but sadly I've got a queue of images waiting.

Answer 1

While it seems straightforward, it's really unsatisfactorily complex.

The problem is, the application that's writing the file isn't done with it when you get the notification...and so you have a concurrency problem. There is no great way to know when the file closes. Well..one way is to subscribe to journal events - which is what FileSystemWatcher does - but this is fairly involved and requires a lot of moving parts. Going this route, you can be notified when the file closes. If you're interested, see https://msdn.microsoft.com/en-us/library/windows/desktop/aa363798(v=vs.85).aspx.

I'd divide the work into two parts. I think I'd start a ThreadPool thread to do the work, and have it read it's work from a list that the FileSystemWatcher's event handler writes to. That way, the event handler returns quickly. The ThreadPool thread would go through it's list, attempting to get an exclusive lock (similar to Tommaso's code) on the file. If it can't, it just moves on to the next file. Every time it successfully copies, it removes that file from the list.

You need to be concerned about thread safety...so you'd want to make a static object to coordinate writes to the list. Both the event handler and the ThreadPool thread would hold the lock while writing.

Here's a scaffold of the whole approach:

internal sealed class Copier: IDisposable
{
  static object sync = new object();
  bool quit;
  FileSystemWatcher watcher;
  List<string> work;

  internal Copier( string pathToWatch )
  {
    work = new List<string>();
    watcher = new FileSystemWatcher();
    watcher.Path = pathToWatch;
    watcher.Create += QueueWork;
    ThreadPool.QueueUserWorkItem( TryCopy );
  }

  void Dispose()
  {
    lock( sync ) quit = true;
  }

  void QueueWork( object source, FileSystemEventArgs args )
  {
    lock ( sync )
    {
      work.Add( args.FullPath );
    }
  }

  void TryCopy( object args )
  {
    List<string> localWork;
    while( true )
    {
      lock ( sync ) 
      { 
        if ( quit ) return;  //--> we've been disposed
        localWork = new List<string>( work );
      }
      foreach( var fileName in localWork )
      {
        var locked = true;
        try
        {
          using 
          ( var throwAway = new FileStream
            ( fileName, 
              FileMode.Open,
              FileAccess.Read,
              FileShare.None
            )
          ); //--> no-op  - will throw if we can't get exclusive read        
          locked = false;
        }
        catch { }
        if (!locked )
        {
          File.Copy( fileName, ... );
          lock( sync ) work.Remove( fileName );
        }
      }
    }
  }
}

Not tested - wrote it right here in the answer...but it, or something like it will cover the bases.

Answer 2

The image file is probably being created by another application that uses an exclusive access lock on both reading and writing external processes (for more informations, read this, especially the paragraph related to Microsoft Windows). You have to either:

• stop/kill the process which is using the file;
• wait until the file isn't being used anymore.

Since the other process is probably writing the file in the moment you try to copy it with your application, the first option is by no means recommendable. It could also be an anti-virus checking the new file, and even in this case the first option would not be recommendable.

You could try to integrate the following code into your CopyImage method so that your application will wait until the file will be no longer in use before proceeding:

private Boolean WaitForFile(String filePath)
{
    Int32 tries = 0;

    while (true)
    {
        ++tries;

        Boolean wait = false;
        FileStream stream = null;

        try
        {
            stream = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.None);
            break;
        }
        catch (Exception ex)
        {
            Logger.LogWarning("CopyImage({0}) failed to get an exclusive lock: {1}", filePath, ex.ToString());

            if (tries > 10)
            {
                Logger.LogWarning("CopyImage({0}) skipped the file after 10 tries.", filePath);
                return false;
            }

            wait = true;
        }
        finally
        {
            if (stream != null)
                stream.Close();
        }

        if (wait)
            Thread.Sleep(250);
    }

    Logger.LogWarning("CopyImage({0}) got an exclusive lock after {1} tries.", filePath, tries);

    return true;
}