improving conversions to binary and back in C#

Question

I'm trying to write a general purpose socket server for a game I'm working on. I know I could very well use already built servers like SmartFox and Photon, but I wan't to go through the pain of creating one myself for learning purposes.

I've come up with a BSON inspired protocol to convert the the basic data types, their arrays, and a special GSObject to binary and arrange them in a way so that it can be put back together into object form on the client end. At the core, the conversion methods utilize the .Net BitConverter class to convert the basic data types to binary. Anyways, the problem is performance, if I loop 50,000 times and convert my GSObject to binary each time it takes about 5500ms (the resulting byte[] is just 192 bytes per conversion). I think think this would be way too slow for an MMO that sends 5-10 position updates per second with a 1000 concurrent users. Yes, I know it's unlikely that a game will have a 1000 users on at the same time, but like I said earlier this is supposed to be a learning process for me, I want to go out of my way and build something that scales well and can handle at least a few thousand users.

So yea, if anyone's aware of other conversion techniques or sees where I'm loosing performance I would appreciate the help.

GSBitConverter.cs

This is the main conversion class, it adds extension methods to main datatypes to convert to the binary format. It uses the BitConverter class to convert the base types. I've shown only the code to convert integer and integer arrays, but the rest of the method are pretty much replicas of those two, they just overload the type.

public static class GSBitConverter
{
    public static byte[] ToGSBinary(this short value)
    {
        return BitConverter.GetBytes(value);
    }

    public static byte[] ToGSBinary(this IEnumerable<short> value)
    {
        List<byte> bytes = new List<byte>();
        short length = (short)value.Count();

        bytes.AddRange(length.ToGSBinary());
        for (int i = 0; i < length; i++)
            bytes.AddRange(value.ElementAt(i).ToGSBinary());

        return bytes.ToArray();
    }

    public static byte[] ToGSBinary(this bool value);
    public static byte[] ToGSBinary(this IEnumerable<bool> value);

    public static byte[] ToGSBinary(this IEnumerable<byte> value);

    public static byte[] ToGSBinary(this int value);
    public static byte[] ToGSBinary(this IEnumerable<int> value);

    public static byte[] ToGSBinary(this long value);
    public static byte[] ToGSBinary(this IEnumerable<long> value);

    public static byte[] ToGSBinary(this float value);
    public static byte[] ToGSBinary(this IEnumerable<float> value);

    public static byte[] ToGSBinary(this double value);
    public static byte[] ToGSBinary(this IEnumerable<double> value);

    public static byte[] ToGSBinary(this string value);
    public static byte[] ToGSBinary(this IEnumerable<string> value);

    public static string GetHexDump(this IEnumerable<byte> value);
}

Program.cs Here's the the object that I'm converting to binary in a loop.

class Program
{
    static void Main(string[] args)
    {
        GSObject obj = new GSObject();
        obj.AttachShort("smallInt", 15);
        obj.AttachInt("medInt", 120700);
        obj.AttachLong("bigInt", 10900800700);
        obj.AttachDouble("doubleVal", Math.PI);
        obj.AttachStringArray("muppetNames", new string[] { "Kermit", "Fozzy", "Piggy", "Animal", "Gonzo" });

        GSObject apple = new GSObject();
        apple.AttachString("name", "Apple");
        apple.AttachString("color", "red");
        apple.AttachBool("inStock", true);
        apple.AttachFloat("price", (float)1.5);

        GSObject lemon = new GSObject();
        apple.AttachString("name", "Lemon");
        apple.AttachString("color", "yellow");
        apple.AttachBool("inStock", false);
        apple.AttachFloat("price", (float)0.8);

        GSObject apricoat = new GSObject();
        apple.AttachString("name", "Apricoat");
        apple.AttachString("color", "orange");
        apple.AttachBool("inStock", true);
        apple.AttachFloat("price", (float)1.9);

        GSObject kiwi = new GSObject();
        apple.AttachString("name", "Kiwi");
        apple.AttachString("color", "green");
        apple.AttachBool("inStock", true);
        apple.AttachFloat("price", (float)2.3);

        GSArray fruits = new GSArray();
        fruits.AddGSObject(apple);
        fruits.AddGSObject(lemon);
        fruits.AddGSObject(apricoat);
        fruits.AddGSObject(kiwi);

        obj.AttachGSArray("fruits", fruits);

        Stopwatch w1 = Stopwatch.StartNew();
        for (int i = 0; i < 50000; i++)
        {
            byte[] b = obj.ToGSBinary();
        }
        w1.Stop();

        Console.WriteLine(BitConverter.IsLittleEndian ? "Little Endian" : "Big Endian");
        Console.WriteLine(w1.ElapsedMilliseconds + "ms");

    }

Here's the code for some of my other classes that are used in the code above. Most of it is repetitive.

GSObject

GSArray

GSWrappedObject

Answer 1

1) ElementAt is very expensive. Use foreach (var v in value) instead of for (int i = 0; i < length; i++) .. .ElementAt(i) ..

2) ToGsBinary methods is expensive because they copy arrays frequently. Use signature void WriteToGsBinary(Stream stream) instead of byte[] ToGsBinary()

3) Add overloads for arrays: void WriteToGsBinary(Stream stream, byte[] values), void WriteToGsBinary(Stream stream, short[] values), etc

Answer 2

My first hunch, without much to go off, would be that a lot of your time is being sunk into constantly re-creating arrays and lists.

I would be inclined to move to a Stream-based approach rather than trying to create arrays constantly. That being, make all the GSBinary methods accept a Stream then write to it rather than making their own arrays, then if you want it in local memory use a MemoryStream at the base and then get your array out of it at the end (Or even better if you're planning this to be a networked application, write directly to the network stream).

As per Chris's comment earlier however the best way to start is to run a profiler such at dotTrace or redgate's ANTS performance profiler to actually find out which step is taking the most time before investing time refactoring something which, while inefficient, may only be a small fraction of the actual time.