CODEWITHSUNDEEP
c#.netmultithreadingthread-safetyfilestream
Alex
Alex

Reputation: 11137

best way to handle multiple streams to same file

I am trying to create a binary storage. Files once added, can't be deleted. All files are stored in the same master file (storage.bin). i will have a table index with pairs of (key, startFilePos, fileLength).

I am struggling to find the best solution before i start coding. As i am not that familiar with streams under .net, i would like some advice on them.

  1. are multiple streams to the same file (storage.bin) a solution? As files may be added in parallel, once i start receiving a file, i reserve the space on disk for it and start writing that specific file to the reserved block while other same-scenario files are added on other threads.

  2. if (1) is a viable solution, is it the optimal one? if not, could you point to a better one?

  3. what is the best way to allocate threads for every file-add scenarios? It seems like 1 thread/ 1 stream isn't a viable solution. What i was thinking is to have a a thread pool and if the limit is reached, have a new file wait for one to be available.

Upvotes: 2

Views: 1580

Answers (1)

Darin Dimitrov
Darin Dimitrov

Reputation: 1038830

  1. are multiple streams to the same file (storage.bin) a solution? As files may be added in parallel, once i start receiving a file, i reserve the space on disk for it and start writing that specific file to the reserved block while other same-scenario files are added on other threads.

Yes, this could be a solution. As long as the different streams are writing to different non-overlapping segments of the file there will be no problems.

  1. if (1) is a viable solution, is it the optimal one? if not, could you point to a better one?

Use separate files and keep the index in the master file.

  1. what is the best way to allocate threads for every file-add scenarios? It seems like 1 thread/ 1 stream isn't a viable solution. What i was thinking is to have a a thread pool and if the limit is reached, have a new file wait for one to be available

You've just described how a thread pool works. Before rolling your own, make sure you have checked the ThreadPool class built directly into the framework. Or a custom concurrency limiting task scheduler.

Upvotes: 2

Related Questions