CODEWITHSUNDEEP
c#serialization
akd
akd

Reputation: 6758

C# serializing/deserializing with memory stream System.OutOfMemoryException

I am trying to serialize/de-serialize a stream of around 50MB xml data with the following code and I get System.OutOfMemoryException exception.

   var formatter = new BinaryFormatter();
        var stream = new MemoryStream();
        using (stream)
        {
            formatter.Serialize(stream, source);
            stream.Seek(0, SeekOrigin.Begin);
            return (T) formatter.Deserialize(stream);
        }

I debugged the code and it throws OutOfMemoryException exception on the formatter.Serialize(stream,source) line.

I did some search in it says that limit is 2GB. How can I debug to find out the reason or is there any efficent way of writing this code? Or any tool to watch the memory usage.

Thanks,

Upvotes: 1

Views: 1985

Answers (1)

Marc Gravell
Marc Gravell

Reputation: 1062600

Dealing with 2GB of xml is never going to be efficient. However, to make it work, you could try writing to a FileStream instead of a MemoryStream, since a MemoryStream has a 2GB limit. Alternatively, you could write your own Stream implementation using multiple buffers rather than a single large buffer.

However, I strongly suggest that what you actually want to do here is some combination of:

  • use a different serialization format (xml is a poor choice for large data)
  • don't require it all in one blob
  • have less data

Upvotes: 1

Related Questions