CODEWITHSUNDEEP
c#multithreadinglistconcurrency
AAverin
AAverin

Reputation: 3042

C# concurrent List access

There is this class, that allows to write data into stream in one Thread and read in multiple Threads.

The main purpose it support streaming, eg. network or audio stream, reading it and processing in multiple threads allowing for data to come as it goes.

http://www.codeproject.com/Articles/345105/Memory-Stream-Multiplexer-write-and-read-from-many?msg=4665022#xx4665022xx

MemoryStreamMultiplexer writes data chunk by chunk and stores into List<byte[]> _Buffer variable.

This _Buffer is then passed into MemoryStreamReader like this:

new MemoryStreamReader(_Buffer, dataReady, finished);

MemoryStreamReader reads byte[] arrays stored in this _Buffer list one my one, checking that data is present

if (_bufferIndex < _bufferList.Count)
{
  return ReadInternal(buffer, offset, count);
}

ReadInternal gets the next buffer from _Buffer list and reads it afterwards

byte[] currentBuffer = _bufferList[_bufferIndex];

Now, here is the problem: at random, currentBuffer becomes null, that results in NullReferenceException later.

In debug, while currentBuffer is stated to be null, if I check the value in _bufferList[_bufferIndex] - valid array is returned. So this seems to be some memory or concurrency issue.

My thoughts are that at some point List decides to change it's Capacity, re-allocating internal array. While we are still reading the List, it returns null because memory address had already changed but no data was yet copied into it.

The question is - how to workaround this problem?

Upvotes: 2

Views: 1897

Answers (1)

Dariusz
Dariusz

Reputation: 22241

This is a slightly extended producer-consumer problem.

Since you are using C# (gc, memory will get freed automatically) I would suggest creating each Consumer with a separate synchronized queue, then posting the same data array to each of those queues. Since the data will be read only after it is pushed it is perfectly safe for multiple threads to access it simultaneously.

Using a ConcurrentQueue will spare you the worry for synchronization, which will in turn save you a lot of work.

Upvotes: 2

Related Questions