How to correctly handle stream dispose and position when serializing and deserializing

Question

I have a system which can use different serializers (BinaryFormatter, XmlSerializer, Json.Net) to write data to a file. I have wrapped them in my own IStreamSerializer interface and want to make sure that they behave the same in my application context. This is one of my test methods:

[Test]
public void JsonSerializer_RoundtripsMap_Successfully()
{
    Map map = new Map(2, 4, TileType.Grass);
    IStreamSerializer serializer = new JsonSerializer(); // Json.Net

    using (var ms = new MemoryStream())
    {
        serializer.Serialize(ms, map);
        ms.Position = 0;
        Map loaded = serializer.Deserialize<Map>(ms);
        // Asserts...
    }
}

I create a MemoryStream, serialize it and try to read it back, asserting if the returned object is the same. This works correctly for the BinaryFormatter and XmlSerializer. However, Json.Net is throwing an exception when I reset the stream position to zero:

System.ObjectDisposedException : The object was used after being disposed.

Here is my JsonSerializer.Serialize method:

public void Serialize(Stream stream, object data)
{
    var serializer = new Newtonsoft.Json.JsonSerializer();  
    using (var textWriter = new StreamWriter(stream))
    using (var jsonWriter = new JsonTextWriter(textWriter))
    {
        serializer.Serialize(jsonWriter, data);
    }
}

I know that Dispose is called at the end of the using statement and that this is the reason I cannot set the stream position when I return back to my test class method.

How can I make this work correctly? I have tried a lot of possible solutions, but they either ended up corrupting the serialized file or throwing an error like cannot read stream.

This throws a JsonReaderException and breaks the physically written file, but it does pass the test:

public void Serialize(Stream stream, object data)
{
    var serializer = new Newtonsoft.Json.JsonSerializer();
    var textWriter = new StreamWriter(stream);
    var jsonWriter = new JsonTextWriter(textWriter);
    serializer.Serialize(jsonWriter, data);
}

Again, the BinaryFormatter and XmlSerializer both work correctly in my test case. They seem to not dispose the stream when I call formatter.Serialize, but Json.Net just doesn't write correct data anymore if I try it the same way.

Note: My framework can only use a custom version of .Net, that is similar to v3.5.

Answer 1

StreamWriter by default takes ownership of the stream you pass in, so when you dispose of the stream writer it disposes of the stream you passed in, if you use this constructor you can pass in a bool that tells it to not dispose of the stream that is passed in.

private static readonly UTF8Encoding UTF8NoBOM = new UTF8Encoding(encoderShouldEmitUTF8Identifier:false, throwOnInvalidBytes:true);

public void Serialize(Stream stream, object data)
{
    var serializer = new Newtonsoft.Json.JsonSerializer();  
    using (var textWriter = new StreamWriter(stream, UTF8NoBOM, bufferSize:1024, leaveOpen:true))
    using (var jsonWriter = new JsonTextWriter(textWriter))
    {
        serializer.Serialize(jsonWriter, data);
    }
}

You just need to pass in the default values that the old constructor would have passed in for the 2nd and 3rd parameters which would be a UTF8Encoding without byte order marks and 1024 respectively.

^{* I used named parameters because I don't like passing in mystery constants, using named parameters it makes it more obvious what 1024 and true represent.}

---

As a alternate solution if you are not on .NET 4.5 or newer you can use a class like below that passes through all Stream commands except Dispose

public class DisposeBlocker : Stream
{
    private readonly Stream _source;
    private readonly bool _blockDispose;
    private readonly bool _blockClose;

    public DisposeBlocker(Stream source, bool blockDispose = true, bool blockClose = false)
    {
        if(source == null)
            throw new ArgumentNullException(nameof(source));
        _source = source;
        _blockDispose = blockDispose;
        _blockClose = blockClose;
    }

    protected override void Dispose(bool disposing)
    {
        if (!_blockDispose && disposing)
        {
            _source.Dispose();
        }
    }

    public override void Close()
    {
        if (!_blockClose)
        {
            _source.Close();
        }
    }

    public override Task CopyToAsync(Stream destination, int bufferSize, CancellationToken cancellationToken)
    {
        return _source.CopyToAsync(destination, bufferSize, cancellationToken);
    }

    public override void Flush()
    {
        _source.Flush();
    }

    public override Task FlushAsync(CancellationToken cancellationToken)
    {
        return _source.FlushAsync(cancellationToken);
    }

    protected override WaitHandle CreateWaitHandle()
    {
        //Obsolete method, Reference Source states just return the following.
        return new ManualResetEvent(false);
    }

    public override IAsyncResult BeginRead(byte[] buffer, int offset, int count, AsyncCallback callback, object state)
    {
        return _source.BeginRead(buffer, offset, count, callback, state);
    }

    public override int EndRead(IAsyncResult asyncResult)
    {
        return _source.EndRead(asyncResult);
    }

    public override Task<int> ReadAsync(byte[] buffer, int offset, int count, CancellationToken cancellationToken)
    {
        return _source.ReadAsync(buffer, offset, count, cancellationToken);
    }

    public override IAsyncResult BeginWrite(byte[] buffer, int offset, int count, AsyncCallback callback, object state)
    {
        return _source.BeginWrite(buffer, offset, count, callback, state);
    }

    public override void EndWrite(IAsyncResult asyncResult)
    {
        _source.EndWrite(asyncResult);
    }

    public override Task WriteAsync(byte[] buffer, int offset, int count, CancellationToken cancellationToken)
    {
        return _source.WriteAsync(buffer, offset, count, cancellationToken);
    }

    public override long Seek(long offset, SeekOrigin origin)
    {
        return _source.Seek(offset, origin);
    }

    public override void SetLength(long value)
    {
         _source.SetLength(value);
    }

    public override int Read(byte[] buffer, int offset, int count)
    {
        return _source.Read(buffer, offset, count);
    }

    public override int ReadByte()
    {
        return _source.ReadByte();
    }

    public override void Write(byte[] buffer, int offset, int count)
    {
        _source.Write(buffer, offset, count);
    }

    public override void WriteByte(byte value)
    {
        _source.WriteByte(value);
    }

    protected override void ObjectInvariant()
    {
        //Obsolete method, nothing to override.
    }

    public override bool CanRead
    {
        get { return _source.CanRead; }
    }

    public override bool CanSeek
    {
        get { return _source.CanSeek; }
    }

    public override bool CanTimeout
    {
        get { return _source.CanTimeout; }
    }

    public override bool CanWrite
    {
        get { return _source.CanWrite; }
    }

    public override long Length
    {
        get { return _source.Length; }
    }

    public override long Position
    {
        get { return _source.Position; }
        set { _source.Position = value; }
    }

    public override int ReadTimeout
    {
        get { return _source.ReadTimeout; }
        set { _source.ReadTimeout = value; }
    }

    public override int WriteTimeout
    {
        get { return _source.WriteTimeout; }
        set { _source.WriteTimeout = value; }
    }

    public override object InitializeLifetimeService()
    {
        return _source.InitializeLifetimeService();
    }

    public override ObjRef CreateObjRef(Type requestedType)
    {
        return _source.CreateObjRef(requestedType);
    }

    public override string ToString()
    {
        return _source.ToString();
    }

    public override bool Equals(object obj)
    {
        return _source.Equals(obj);
    }

    public override int GetHashCode()
    {
        return _source.GetHashCode();
    }
}

It is used like

public void Serialize(Stream stream, object data)
{
    var serializer = new Newtonsoft.Json.JsonSerializer();  
    using (var textWriter = new StreamWriter(new DisposeBlocker(stream)))
    using (var jsonWriter = new JsonTextWriter(textWriter))
    {
        serializer.Serialize(jsonWriter, data);
    }
}

Answer 2

Based on the comments I figured out this solution:

public void Serialize(Stream stream, object data)
{
    var serializer = new Newtonsoft.Json.JsonSerializer();
    var streamWriter = new StreamWriter(stream);
    serializer.Serialize(streamWriter, data);
    streamWriter.Flush();
}

I found that Json.Net can work with the StreamWriter directly. So now I flush it at the end, but leave it open. As far as my unit tests and some practical tests go, this works.

Is this a valid solution or do I absolutely have to dispose the StreamWriter? Are these kinds of memory leaks a problem or safe to ignore?