Marshal a std::vector<uint64_t> from C++ to C#

Question

no matter what I try. I appear to get garbage results when I marshal the data across! The data after the marshal copy just contains an array of what looks like uninitialized data. Just pure garbage.

Thanks for your help in advance!

C++

typedef uint64_t TDOHandle;

extern "C" DATAACCESSLAYERDLL_API const TDOHandle * __stdcall DB_GetRecords()
{
    const Database::TDBRecordVector vec = Database::g_Database.GetRecords();
    if (vec.size() > 0)
    {
        return &vec[0];
    }
    return nullptr;
}

C#

The declaration

    [System.Security.SuppressUnmanagedCodeSecurity()]
    [DllImport("DataCore.dll")]
    static private extern IntPtr DB_GetRecords();

//The marshalling process

            IntPtr ptr_records = DB_GetRecords();
        if (ptr_records != null)
        {
            Byte[] recordHandles = new Byte[DB_GetRecordCount()*sizeof (UInt64)];
            Marshal.Copy(ptr_records, recordHandles, 0, recordHandles.Length);

            Int64[] int64Array = new Int64[DB_GetRecordCount()];
            Buffer.BlockCopy(recordHandles, 0, int64Array, 0, recordHandles.Length);
        }

Answer 1

I'll add that there is another way to do it:

public sealed class ULongArrayWithAllocator
{
    // Not necessary, default
    [UnmanagedFunctionPointer(CallingConvention.StdCall)]
    public delegate IntPtr AllocatorDelegate(IntPtr size);

    private GCHandle Handle;

    private ulong[] allocated { get; set; }

    public ulong[] Allocated
    {
        get
        {
            // We free the handle the first time the property is
            // accessed (we are already C#-side when it is accessed)
            if (Handle.IsAllocated)
            {
                Handle.Free();
            }

            return allocated;
        }
    }

    // We could/should implement a full IDisposable interface, but
    // the point of this class is that you use it when you want
    // to let C++ allocate some memory and you want to retrieve it,
    // so you'll access LastAllocated and free the handle
    ~ULongArrayWithAllocator()
    {
        if (Handle.IsAllocated)
        {
            Handle.Free();
        }
    }

    // I'm using IntPtr for size because normally 
    // sizeof(IntPtr) == sizeof(size_t) and vector<>.size() 
    // returns a size_t
    public IntPtr Allocate(IntPtr size)
    {
        if (allocated != null)
        {
            throw new NotSupportedException();
        }

        allocated = new ulong[(long)size];
        Handle = GCHandle.Alloc(allocated, GCHandleType.Pinned);
        return Handle.AddrOfPinnedObject();
    }
}

[DllImport("DataCore.dll", CallingConvention = CallingConvention.StdCall)]
static private extern IntPtr DB_GetRecords(ULongArrayWithAllocator.AllocatorDelegate allocator);

and to use it:

var allocator = new ULongArrayWithAllocator();
DB_GetRecords(allocator.Allocate);

// Here the Handle is freed
ulong[] allocated = allocator.Allocated; 

and C++ side

extern "C" DATAACCESSLAYERDLL_API void __stdcall DB_GetRecords(TDOHandle* (__stdcall *allocator)(size_t)) {
    ...
    // This is a ulong[vec.size()] array, that you can
    // fill C++-side and can retrieve C#-side
    TDOHandle* records = (*allocator)(vec.size());
    ...
}

or something similar :-) You pass a delegate to the C++ function that can allocate memory C#-side :-) And then C# side you can retrieve the last memory that was allocated. It is important that you don't make more than one allocation C++-side in this way in a single call, because you are saving a single LastAllocated reference, that is "protecting" the allocated memory from the GC (so don't do (*allocator)(vec.size());(*allocator)(vec.size());)

Note that it took me 1 hour to write correctly the calling conventions of the function pointers, so this isn't for the faint of heart :-)

Answer 2

You are returning the address of memory owned by a local variable. When the function returns, the local variable is destroyed. Hence the address you returned is now meaningless.

You need to allocate dynamic memory and return that. For instance, allocate it with CoTaskMemAlloc. Then the consuming C# can deallocate it with a call to Marshal.FreeCoTaskMem.

Or allocate the memory using new, but also export a function from your unamanaged code that can deallocate the memory.

For example:

if (vec.size() > 0)
{
    TDOHandle* records = new TDOHandle[vec.size()];
    // code to copy content of vec to records
    return records;
}
return nullptr;

And then you would export another function that exposed the deallocator:

extern "C" DATAACCESSLAYERDLL_API void __stdcall DB_DeleteRecords(
    const TDOHandle * records)
{
    if (records)
        delete[] record;
}

All that said, it seems that you can obtain the array length before you call the function to populate the array. You do that with DB_GetRecordCount(). In that case you should create an array in your managed code, and pass that to the unmanaged code for it to populate. That side steps all the issues of memory management.