CODEWITHSUNDEEP
c#memorystreammemorystream
Bill Tarbell
Bill Tarbell

Reputation: 5234

memory efficient way to read from COM stream to c# byte[]

My current approach is to read the COM stream into a C# MemoryStream and then call .toArray. However, I believe toArray creates a redundant copy of the data. Is there a better way that has reduced memory usage as the priority?

var memStream = new MemoryStream(10000);
var chunk = new byte[1000];
while (true)
{
 int bytesRead = comStream.read(ref chunk, chunk.Length);
 if (bytesRead == 0)
   break; // eos
 memStream.Write(chunk, 0, bytesRead);
}

//fairly sure this creates a duplicate copy of the data
var array = memStream.ToArray();

//does this also dupe the data?
var array2 = memStream.GetBuffer();

Upvotes: 1

Views: 401

Answers (1)

Marc Gravell
Marc Gravell

Reputation: 1062745

If you know the length of the data before you start consuming it, then: you can allocate a simple byte[] and fill that in your read loop simply by incrementing an offset each read with the number of bytes read (and decrementing your "number of bytes you're allowed to touch). This does depend on having a read overload / API that accepts either an offset or a pointer, though.

If that isn't an option: GetBuffer() is your best bet - it doesn't duplicate the data; rather it hands you the current possibly oversized byte[]. Because it is oversized, you must consider it in combination with the current .Length, perhaps wrapping the length/data pair in either a ArraySegment<byte>, or a Span<byte>/Memory<byte>.

In the "the length is known" scenario, if you're happy to work with oversized buffers, you could also consider a leased array, via ArrayPool<byte>.Shared - rent one of at least that size, fill it, then constrain your segment/span to the populated part (and remember to return it to the pool when you're done).

Upvotes: 4

Related Questions