Memory leak at ProtoBuf serialization on .net core

Question

I'm trying to find out the reason of problems of my .net core application with protobuf serialization, so i've wrote a test application and... there is some strange. There is a memory leak if i use Task and there is no memory leak if i use Thread.

Example with Task:

internal class Program
{
    private static void Main(string[] args)
    {
        var data = new Person
        {
            Id = 1,
            Name = new string('q', 10000),
            Address = new Address
            {
                Line1 = new string('w', 10000),
                Line2 = new string('e', 10000)
            }
        };



        //for (var i = 0; i < 100; i++)
        //{
        //    var thread = new Thread(() =>
        //    {
        //        while (true)
        //            try
        //            {
        //                var b = ProtoSerialize(data);
        //                Person serializedPerson;
        //                using (Stream stream2 = new MemoryStream(b))
        //                {
        //                    serializedPerson = Serializer.Deserialize<Person>(stream2);
        //                }
        //            }
        //            catch (Exception e)
        //            {
        //                Console.WriteLine(e);
        //            }
        //    });
        //    thread.Start();
        //}

        for (var i = 0; i < 100; i++)
        {
            new Task(() =>
            {
                while (true)
                {
                    var b = ProtoSerialize(data);
                    Person serializedPerson;
                    using (Stream stream2 = new MemoryStream(b))
                    {
                        serializedPerson = Serializer.Deserialize<Person>(stream2);
                    }
                }
            }).Start();
        }

        Console.ReadLine();
    }


    public static byte[] ProtoSerialize<T>(T record) where T : class
    {
        using (var stream = new MemoryStream())
        {
            Serializer.Serialize(stream, record);
            return stream.ToArray();
        }
    }
}


[ProtoContract]
public class Person
{
    [ProtoMember(1)]
    public int Id { get; set; }

    [ProtoMember(2)]
    public string Name { get; set; }

    [ProtoMember(3)]
    public Address Address { get; set; }
}

[ProtoContract]
public class Address
{
    [ProtoMember(1)]
    public string Line1 { get; set; }

    [ProtoMember(2)]
    public string Line2 { get; set; }
}

and screenshot of dotMemory:

so if i use threads (commented code)

            for (var i = 0; i < 100; i++)
        {
            var thread = new Thread(() =>
            {
                while (true)
                    try
                    {
                        var b = ProtoSerialize(data);
                        Person serializedPerson;
                        using (Stream stream2 = new MemoryStream(b))
                        {
                            serializedPerson = Serializer.Deserialize<Person>(stream2);
                        }
                    }
                    catch (Exception e)
                    {
                        Console.WriteLine(e);
                    }
            });
            thread.Start();
        }

Then there is no leak:

So the question is:

1. Am i doing something wrong?

2. Why this happens?

3. How to do wright?

p.s.

protobuf-net is 2.3.2

.net core 2.0

UPD1: call GC.Collect in main thread changes nothing (there is still memory leak)

Answer 1

Well that's fun. protobuf-net does make limited use of some background pools - there is a [ThreadStatic] instance of ProtoReader, for example - although that is a light object by itself; while it does keep a byte[] pool handy, that is not kept referenced by the [ThreadStatic] instance (the buffer is released to the pool before then). So there's nothing that immediately leaps to mind; have you tried doing a few forced GC.Collects to see if anything is actually leaking, vs simply not been collected yet? Have you tried replacing all the protobuf-net with dummy code (Thread.Sleep(100); for example) to see if you're just seeing the weight of the TPL machinery? Do you know (from that tool) what object types are accounting for the overhead?

It is also possible that this is simply a consequence of the very different threading usage here upsetting GC: the Task code is going to be using the ThreadPool, so a limited number of threads. That is neither good nor bad - just: different.