Convert Object To Byte Array without Serialization nor Padding

Question

It seems easy but I can't see it easy at all. My question is I have a struct and I need to convert it to byte stream without any additional bytes for types nor padding nor metadata. Assume I have a struct

[StructLayout(LayoutKind.Sequential)]
public struct MyStruct
{
    public ushort a;                 
    public uint b;  
    public uint c;
    public ushort d;
}

Remark: I can't change the pack here to 1(Project constrains), so using the following solution won't work as there's a padding added

int size = Marshal.SizeOf(typeof(T));
byte[] arr = new byte[size];
IntPtr ptr = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(str, ptr, true);
Marshal.Copy(ptr, arr, 0, size);
Marshal.FreeHGlobal(ptr);

Also Remark: I can't use Binary format serialization as there's a metadata added. All what I want simply if a=1 b=2 c=3 d=4 to get binary format in a GENERIC WAY as

    arr {byte[12]}  byte[]
    [0]  1  byte
    [1]  0  byte
    [2]  2  byte
    [3]  0  byte
    [4]  0  byte
    [5]  0  byte
    [6]  3  byte
    [7]  0  byte
    [8]  0  byte
    [9]  0  byte
    [10] 4  byte
    [11] 0  byte

Any help?

Answer 1

Thanks all for your valuable answers/comments! I have found a workaround I will remove the padding manually

        static byte[] getBytes<T>(T str) where T : struct
    {

        int size = Marshal.SizeOf(typeof(T));
        int Pack = str.GetType().StructLayoutAttribute.Pack;
        byte[] arr = new byte[size];
        IntPtr ptr = Marshal.AllocHGlobal(size);
        Marshal.StructureToPtr(str, ptr, true);
        Marshal.Copy(ptr, arr, 0, size);
        Marshal.FreeHGlobal(ptr);
        arr = RemovePadding<T>(arr.ToList(), str, str.GetType().StructLayoutAttribute.Pack);
        return arr;
    }
    static byte[] RemovePadding<T>(List<byte> buffer, T str, int Pack)
    {
        int largestsize = 0;
        int index = 0;
        int RowOfMemory = 0;

        //Get all fields 
        var fields = str.GetType().GetFields();

        //After MSDN
        //The alignment of the type is the size of its largest element (1, 2, 4, 8, etc., bytes) or the specified packing size, whichever is smaller.
        foreach (var item in fields)
        {
            if (Marshal.SizeOf(item.FieldType) > largestsize)
                largestsize = Marshal.SizeOf(item.FieldType);
        }
        if (largestsize < Pack) Pack = largestsize;

        //Find and remove padding from all memory rows 
        foreach (var item in fields)
        {
            int size = Marshal.SizeOf(item.FieldType);
            if (RowOfMemory != 0 && (RowOfMemory + size) > Pack)
            {
                int paddingsize = Math.Abs(Pack - RowOfMemory);
                buffer.RemoveRange(index, paddingsize);
                RowOfMemory = size % Pack;
            }
            else if ((RowOfMemory + size) < Pack)
            {
                RowOfMemory += size;
            }
            else if ((RowOfMemory + size) == Pack)
            {
                RowOfMemory = 0;
            }
            index += size;
        }
        if (RowOfMemory != 0)
        {
            int paddingsize = Math.Abs(Pack - RowOfMemory);
            buffer.RemoveRange(index, paddingsize);
        }
        return buffer.ToArray();
    }

I will not count this as a right answer, may be someone found a better solution.

Answer 2

You can use Explicit layout to define exactly how you want the array to be formed:

    [StructLayout(LayoutKind.Explicit)]
    public struct MyStruct
    {
        [FieldOffset(0)]
        public ushort a;
        [FieldOffset(2)]
        public uint b;
        [FieldOffset(6)]
        public uint c;
        [FieldOffset(10)]
        public ushort d;
    }

But of course its not at all generic

Answer 3

If you don't mind a bit of manual work, you can write your own converter and handle each datatype differently, for example:

public static class StructSerializer
{
    public static byte[] Serialize<T>(T data) where T : struct
    {
        List<byte> result = new List<byte>();
        Type type = data.GetType();
        IEnumerable<FieldInfo> orderedFields = type.GetFields().OrderBy(f => Marshal.OffsetOf(type, f.Name).ToInt32());

        foreach (FieldInfo fieldInfo in orderedFields)
        {
            object value = fieldInfo.GetValue(data);
            MethodInfo conversion = typeof(BitConverter).GetMethod(nameof(BitConverter.GetBytes), new[]{fieldInfo.FieldType});
            if (conversion == null) continue;
            byte[] converted = (byte[])conversion.Invoke(null, new []{value});
            result.AddRange(converted);
        }

        return result.ToArray();
    }
}

---

You didn't specifically mention it, but keep the ByteOrder in mind. You can check the byte order with BitConverter.IsLittleEndian.

Answer 4

Edit: I may have misunderstood the question; if your point is that you want the specific layout - then see the FieldOffset hints that Tim Rutter notes in their answer.

---

unsafe might be the easiest option:

using System;
using System.Runtime.InteropServices;

static class P
{
    private static unsafe void Main()
    {
        MyStruct orig = new MyStruct { a = 1, b = 2, c = 3, d = 4 };

        byte[] raw = new byte[sizeof(MyStruct)];
        // write the empty array to prove it is empty
        Console.WriteLine(BitConverter.ToString(raw));


        // serialize
        fixed (byte* p = raw)
        {
            var typed = (MyStruct*)p;
            *typed = orig;
        }

        // write the serialized data
        Console.WriteLine(BitConverter.ToString(raw));

        // deserialize
        MyStruct clone;
        fixed (byte* p = raw)
        {
            var typed = (MyStruct*)p;
            clone = *typed;
        }

        Console.WriteLine($"a = {clone.a}, b = {clone.b}, c = {clone.c}, d = {clone.d}");
    }
}

with:

[StructLayout(LayoutKind.Sequential)] // see Tim's answer for explicit layout
public struct MyStruct
{
    public ushort a;
    public uint b;
    public uint c;
    public ushort d;
}

However, it isn't very generic. If you need more generic code: the Unsafe type has many utility methods for this kind of thing.