CODEWITHSUNDEEP
c#performancelinq
ManInMoon
ManInMoon

Reputation: 7005

Processing records speed

I realise this is a non-specific code question. But I suspect that people with answers are on this forum.

I am receiving a large amount of records of < 100 bytes via TCP at a rate of 10 per millisecond.

I have to parse and process the data and that takes me 100 microseconds - so I am pretty maxed out.

Does 100 microseconds seem large?

Here is an example of the kind of processing I do with LINQ. It is really convenient - but is it inherently slow?

    public void Process()
    {
        try
        {
            int ptr = PayloadOffset + 1;
            var cPair = MessageData.GetString(ref ptr, 7);
            var orderID = MessageData.GetString(ref ptr, 15);

            if (Book.CPairs.ContainsKey(cPair))
            {
                var cPairGroup = Book.CPairs[cPair];
                if (cPairGroup.BPrices != null)
                {                    
                    cPairGroup.BPrices.ForEach(x => { x.BOrders.RemoveAll(y => y.OrderID.Equals(orderID)); });
                    cPairGroup.BPrices.RemoveAll(x => x.BOrders.Count == 0);
                }
            }
         }
    }

    public class BOrderGroup
    {
        public double Amount;
        public string OrderID;
    }

    public class BPriceGroup
    {
        public double BPrice;
        public List<BOrderGroup> BOrders;
    }
    public class CPairGroup
    {
        public List<BPriceGroup> BPrices;
    }

    public static Dictionary<string, CPairGroup> CPairs;

Upvotes: 0

Views: 116

Answers (1)

nejcs
nejcs

Reputation: 1272

As other have mentioned, LINQ is not inherently slow. But it can be slower than equivalent non-LINQ code (this is why Roslyn team has "Avoid LINQ" guide under coding conventions).

If this is your hot path and you need every microsecond than you should probably implement logic in such a way:

public void Process()
{
    try
    {
        int ptr = PayloadOffset + 1;
        var cPair = MessageData.GetString(ref ptr, 7);
        var orderID = MessageData.GetString(ref ptr, 15);

        if (Book.CPairs.TryGetValue(cPair, out CPairGroup cPairGroup) && cPairGroup != null)
        {
            for (int i = cPairGroup.BPrices.Count - 1; i >= 0; i--)
            {
                var x = cPairGroup.BPrices[i];
                for (int j = x.BOrders.Count - 1; j >= 0; j--)
                {
                    var y = x.BOrders[j];
                    if (y.OrderID.Equals(orderID))
                    {
                        x.BOrders.RemoveAt(j);
                    }
                }
                if (x.BOrders.Count == 0)
                {
                    cPairGroup.BPrices.RemoveAt(i);
                }
            }
        }
    }
}

Main points:

  • Avoid double dictionary lookup by using TryGetValue
  • Single iteration over cPairGroup.BPrices
  • In place modification of structures by iterating backwards
  • This code should not contain any additional heap allocations

Upvotes: 3

Related Questions