CODEWITHSUNDEEP
c#multithreadingfile-ioparallel-processing
user0000001
user0000001

Reputation: 2233

Parallel file IO while doing heavy CPU operations in C#

I am rather new to C# and trying to determine best practice when it comes to file operations.

My application is using Parallel.ForEach to parse several large files. In the method, I am writing to a single file with StreamWriter but having to lock the object during that process. I anticipated a slow down with that methodology and knew that I would have to have a separate thread for file IO.

Which brings me to my question. Does C# already provide a namespace for non-blocking, thread-safe file IO?

Thanks!

Upvotes: 2

Views: 430

Answers (1)

Servy
Servy

Reputation: 203822

.NET provides operations for asynchronous file IO, and it provides tools for synchronizing asynchronous operations. You have to combine the two tools yourself if you want both features though, not that that's particularly hard. You can create a simple queue to serialize asynchronous operations for you, as shown here. Once you have that you can share a single queue between all of your workers, having them each enqueue a call to WriteAsync to do your writing.

Note that you don't actually need a separate thread for the file writing at all. It can be asynchronous without needing a dedicated thread.

Upvotes: 3

Related Questions