CODEWITHSUNDEEP
c#garbage-collectionprotobuf-netnetworkstream
Rob Lu
Rob Lu

Reputation: 115

How to properly handle incoming protobuf message with a NetworkStream?

Using TCPClient's NetworkStream and protobuf-net I send and receive protobuf messages via TCP. Receiving is done with the following method that runs in an own thread:

private void HandleClientComm()
{
    using (NetworkStream stream = m_Stream)
    {
        object o;
        while (true)
        {
            if (stream.CanRead && stream.DataAvailable)
            {
                o = null;
                if (Serializer.NonGeneric.TryDeserializeWithLengthPrefix(stream, PrefixStyle.Base128, Utilities.CommunicationHelper.resolver, out o))
                {
                    if (o != null)
                    {
                        //Do something with the incoming protobuf object
                    }
                }
                Thread.Sleep(1);
            }
        }
    }   
}

This works fine, but I have a problem with the garbage collection. It seems like old protobuf objects are still kept in the memory. Large message lead to System.OutOfMemoryExceptions after a while.

Explicitly calling GC.Collect() before sleeping fixes this problem. But it obviously slows down everything. How do I handle this properly?

Upvotes: 1

Views: 1020

Answers (1)

Marc Gravell
Marc Gravell

Reputation: 1064204

protobuf-net itself isn't going to keep hold of the old messages - indeed, if it was doing that, the GC.Collect wouldn't have helped.

The first thing I can see is that the hot loop waiting on DataAvailable is really expensive; that could perhaps be interfering with GC. The second thing I can see is that you can probably release the object at o before sleeping; as a random thing to try, perhaps:

using (NetworkStream stream = m_Stream)
{
    object o;
    while (Serializer.NonGeneric.TryDeserializeWithLengthPrefix(
        stream, PrefixStyle.Base128,
        Utilities.CommunicationHelper.resolver, out o))
    {
        if (o != null)
        {
            //TODO: Do something with the incoming protobuf object

            // clear o, just to give GC the very best chance while we sleep
            o = null;
        }
        Thread.Sleep(1); // <=== not sure why you want to sleep here, btw
    }
}

Upvotes: 1

Related Questions