CODEWITHSUNDEEP
c#bufferfilesystemwatcher
Guilherme de Jesus Santos
Guilherme de Jesus Santos

Reputation: 5053

What is the ideal size for FileSystemWatcher InternalBufferSize?

I have a issue with my FileSystemWatcher.

I have application that needs to monitor a great, really great, amount of files which have been created in a folder, in a short period of time.

When I start developing it, I realize that a lot of files where not been notified, if my buffer was less then 64kb, which is what Microsoft recommends. I tried increasing the buffer size beyond this until I reached a value that worked for me, which is 2621440 bytes!

What could you recommend to use a small size for this case, or what would be the ideal size of buffer?

My example code : 

WATCHER = new FileSystemWatcher(SignerDocument.UnsignedPath, "*.pdf");
WATCHER.InternalBufferSize = 2621440; //Great and expensive buffer 2.5mb size!
WATCHER.IncludeSubdirectories = true;
WATCHER.EnableRaisingEvents = true;
WATCHER.Created += new FileSystemEventHandler(watcher_Created);
WATCHER.Renamed += new RenamedEventHandler(watcher_Renamed);

And what Microsoft say about this in .NET 2.0 :

Increasing buffer size is expensive, as it comes from non paged memory that cannot be swapped out to disk, so keep the buffer as small as possible. To avoid a buffer overflow, use the NotifyFilter and IncludeSubdirectories properties to filter out unwanted change notifications. link : FileSystemWatcher.InternalBufferSize Property

Upvotes: 5

Views: 6741

Answers (2)

user2071475
user2071475

Reputation: 53

You can set the buffer to 4 KB or larger, but it must not exceed 64 KB. If you try to set the InternalBufferSize property to less than 4096 bytes, your value is discarded and the InternalBufferSize property is set to 4096 bytes. For best performance, use a multiple of 4 KB on Intel-based computers.

From: http://msdn.microsoft.com/de-de/library/system.io.filesystemwatcher.internalbuffersize(v=vs.110).aspx

Upvotes: 2

Sedat Kapanoglu
Sedat Kapanoglu

Reputation: 47680

For such a huge workload you might want to opt for "periodic sweep" approach instead of instant notifications. You could for instance scan the directory every 5 seconds and process the added files. If you move the file to another directory after it's processed, your periodic workload might even become minimal.

That is also a safer approach because even if your processing code crashes you can always recover, unlike notifications, your checkpoint wouldn't get lost.

Upvotes: 4

Related Questions