CODEWITHSUNDEEP
c#.net.net-core
msedi
msedi

Reputation: 1743

Is there a MemoryStream that accepts a Span<T> or Memory<T>?

I'm reading data from many sources (MemoryMappedFiles or FileStream). One problem is though that every single call to read a byte, int or some other type is extremely slow. So I'd like to read a chunk of data into an array and hand this over to a lightweight memory stream and do the reading of the individual components there.

The problem is that the current MemorySrream of .NET does only allow an array in the constructor, but I would need a Stream that is able to handle Span or Memor for example. There is a ReadOnlyMemoryStream as an internal class deeply buried in the .NET source code.

The interesting thing is though that the ReadOnlyMemoryStream is slower than the MemoryStream where I would have thought it shouldn't make a big difference.

Is there any better implementation?

Upvotes: 15

Views: 10442

Answers (3)

Rook
Rook

Reputation: 6155

Once again, late to the party but: you can wrap a Memory<T> with a stream, so long as you're happy to use unsafe code.

The class you want is UnmanagedMemoryStream, and it is supported by all current versions of dotnet. You can use it like this:

var m = new Memory<int>(new int[100]);

unsafe
{
    using var p = m.Pin();
    using var s = new UnmanagedMemoryStream(
        (byte*)p.Pointer, m.Length, m.Length, FileAccess.Write);
    using var w = new BinaryWriter(s);
    w.Write(123);
}

Debug.Assert(m.Span[0] == 123);

(UnmanagedMemoryStreams are read-only unless you specifically request write-access)

This approach works with anything you can get a pointer to, which includes Span<T>. I've done no profiling on performance of this flavour of stream compared to a regular MemoryStream.

As an alternative approach that doesn't use unsafe code, I've done things like use ArrayPool to provide plain byte arrays on demand for reading stuff into from various places, and then wrapped those pools in memorystreams, spans or memories as necessary. This can be slightly faster than using Memory as your base buffer type because there's one less layer of abstraction, though it isn't quite as tidy to use (requiring finally rather than using to guarantee return to the pool, for example). This may not be practical in your specific situation, but it isn't a bad architecture to use for future work if you're not too constrained by the need to use Memory.

Upvotes: 2

Shmil The Cat
Shmil The Cat

Reputation: 4668

Yes, there is such an extension method, provided by Microsoft.Toolkit.HighPerformance package

Upvotes: 7

Blindy
Blindy

Reputation: 67447

If you have a Span, you're already as fast as it gets, just read straight through it instead of plopping a stream on top of it.

A Span even gives you access to really nifty things like straight struct mapping without copies (MemoryMarshal.Cast), span increments (the equivalent to a stream advancing, part of Unsafe.Add), block copies if actually necessary (Unsafe.Copy) etc.

Upvotes: 5

Related Questions