CODEWITHSUNDEEP
c#serializationprotobuf-net
Max0999
Max0999

Reputation: 354

Serialize data is too big

I serialize the data for every player server side and it's about 128kb in size. I serialize a [255,255] array of bools which is a must for mapping, what alternatives could I use as I heard gzip would actually increase the size?

I have heard about protobuf-net but it's undocumented and no examples exist on the internet.

Upvotes: 2

Views: 1549

Answers (3)

Guffa
Guffa

Reputation: 700562

If you represent the booleans as bits and serialise as binary, that is only about 8 kilobyte.

If you need is as text, serialise the binary using base64, that will make it about 12 kilobyte.

Flatten the two dimensional array to a one dimensional array, and make a BitArray from it.

Example:

bool[] bools = { true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true };

BitArray bits = new BitArray(bools);
byte[] bytes = new byte[3];
bits.CopyTo(bytes, 0);

Console.WriteLine(BitConverter.ToString(bytes));
Console.WriteLine(Convert.ToBase64String(bytes));

Outut:

FF-FF-0F
//8P

Upvotes: 2

Marc Gravell
Marc Gravell

Reputation: 1063338

The first thing I would do would be: not store that data in a bool[,] - that is pretty inefficient, and a real pain to store. I would write a wrapper that shims it to a flat byte[]:

public sealed class BitGrid
{
    public BitGrid() {
        // 255 * 255 = 32 bytes per row, 255 rows
        bytes = new byte[8160];
    }
    public BitGrid(byte[] data)
    {
        if (data == null) throw new ArgumentNullException("data");
        if (data.Length != 8160) throw new ArgumentException("data");
        this.bytes = data;
    }

    readonly byte[] bytes;

    public bool this[byte x, byte y]
    {
        get
        {
            int xByte = x / 8, xBit = x % 8;
            byte val = bytes[(32 * y) + xByte];
            switch (xBit)
            {
                case 0: return (val & 1) != 0;
                case 1: return (val & 2) != 0;
                case 2: return (val & 4) != 0;
                case 3: return (val & 8) != 0;
                case 4: return (val & 16) != 0;
                case 5: return (val & 32) != 0;
                case 6: return (val & 64) != 0;
                case 7: return (val & 128) != 0;
            }
            throw new InvalidOperationException("oops!");
        }
        set
        {
            int xByte = x / 8, xBit = x % 8;
            int offset = (32 * y) + xByte;
            byte val = bytes[offset];
            if (value)
            {
                switch (xBit)
                {
                    case 0: val |= 1; break;
                    case 1: val |= 2; break;
                    case 2: val |= 4; break;
                    case 3: val |= 8; break;
                    case 4: val |= 16; break;
                    case 5: val |= 32; break;
                    case 6: val |= 64; break;
                    case 7: val |= 128; break;
                }
            }
            else
            {
                switch (xBit)
                {
                    case 0: val &= 254; break;
                    case 1: val &= 253; break;
                    case 2: val &= 251; break;
                    case 3: val &= 247; break;
                    case 4: val &= 239; break;
                    case 5: val &= 223; break;
                    case 6: val &= 191; break;
                    case 7: val &= 127; break;
                }
            }

            bytes[offset] = val;
        }
    }
    public byte[] ToArray()
    {
        return (byte[])bytes.Clone();
    }
}

Then to serialize it, it is just:

byte[] data = grid.ToArray();
// store "data"

and to deserialize, it is just:

byte[] data = ...
grid = new BitGrid(data);

You can save / load a byte[] to/from disk using the File.ReadAllBytes / File.WriteAllBytes methods, or if you have other data to store, then any standard serializer will work fine with a byte[]. This data will always be 8160 bytes - just under 8k.

Upvotes: 5

Matthew Strawbridge
Matthew Strawbridge

Reputation: 20630

You could use a (one-dimensional) BitArray for serialization. This packs the bits into bytes.

Upvotes: 2

Related Questions